examples/azure/DALL-E.ipynb

{
 "cells": [
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Azure DALL·E image generation example\n",
    "\n",
    "This notebook shows how to generate images with the Azure OpenAI service."
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Setup\n",
    "\n",
    "First, we install the necessary dependencies."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "! pip install openai\n",
    "# We need requests to retrieve the generated image\n",
    "! pip install requests\n",
    "# We use Pillow to display the generated image\n",
    "! pip install pillow \n",
    "# (Optional) If you want to use Microsoft Active Directory\n",
    "! pip install azure-identity"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "import openai"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "Additionally, to properly access the Azure OpenAI Service, we need to create the proper resources at the [Azure Portal](https://portal.azure.com) (you can check a detailed guide on how to do this in the [Microsoft Docs](https://learn.microsoft.com/en-us/azure/cognitive-services/openai/how-to/create-resource?pivots=web-portal))\n",
    "\n",
    "Once the resource is created, the first thing we need to use is its endpoint. You can get the endpoint by looking at the *\"Keys and Endpoints\"* section under the *\"Resource Management\"* section. Having this, we will set up the SDK using this information:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "openai.api_base = '' # Add your endpoint here\n",
    "\n",
    "# At the moment DALL·E is only supported by the 2023-06-01-preview API version\n",
    "openai.api_version = '2023-06-01-preview'"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Authentication\n",
    "\n",
    "The Azure OpenAI service supports multiple authentication mechanisms that include API keys and Azure credentials."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "use_azure_active_directory = False"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "#### Authentication using API key\n",
    "\n",
    "To set up the OpenAI SDK to use an *Azure API Key*, we need to set up the `api_type` to `azure` and set `api_key` to a key associated with your endpoint (you can find this key in *\"Keys and Endpoints\"* under *\"Resource Management\"* in the [Azure Portal](https://portal.azure.com))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "if not use_azure_active_directory:\n",
    "    openai.api_type = 'azure'\n",
    "    openai.api_key = os.environ[\"OPENAI_API_KEY\"]"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> Note: In this example, we configured the library to use the Azure API by setting the variables in code. For development, consider setting the environment variables instead:\n",
    "\n",
    "```\n",
    "OPENAI_API_BASE\n",
    "OPENAI_API_KEY\n",
    "OPENAI_API_TYPE\n",
    "OPENAI_API_VERSION\n",
    "```"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Authentication using Microsoft Active Directory\n",
    "Let's now see how we can get a key via Microsoft Active Directory Authentication."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from azure.identity import DefaultAzureCredential\n",
    "\n",
    "if use_azure_active_directory:\n",
    "    default_credential = DefaultAzureCredential()\n",
    "    token = default_credential.get_token(\"https://cognitiveservices.azure.com/.default\")\n",
    "\n",
    "    openai.api_type = 'azure_ad'\n",
    "    openai.api_key = token.token"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A token is valid for a period of time, after which it will expire. To ensure a valid token is sent with every request, you can refresh an expiring token by hooking into requests.auth:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import typing\n",
    "import time\n",
    "import requests\n",
    "\n",
    "if typing.TYPE_CHECKING:\n",
    "    from azure.core.credentials import TokenCredential\n",
    "\n",
    "class TokenRefresh(requests.auth.AuthBase):\n",
    "\n",
    "    def __init__(self, credential: \"TokenCredential\", scopes: typing.List[str]) -> None:\n",
    "        self.credential = credential\n",
    "        self.scopes = scopes\n",
    "        self.cached_token: typing.Optional[str] = None\n",
    "\n",
    "    def __call__(self, req):\n",
    "        if not self.cached_token or self.cached_token.expires_on - time.time() < 300:\n",
    "            self.cached_token = self.credential.get_token(*self.scopes)\n",
    "        req.headers[\"Authorization\"] = f\"Bearer {self.cached_token.token}\"\n",
    "        return req\n",
    "\n",
    "if use_azure_active_directory:\n",
    "    session = requests.Session()\n",
    "    session.auth = TokenRefresh(default_credential, [\"https://cognitiveservices.azure.com/.default\"])\n",
    "\n",
    "    openai.requestssession = session"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Generations\n",
    "\n",
    "With setup and authentication complete, you can now generate images on the Azure OpenAI service and retrieve them from the returned URLs."
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 1. Generate the images\n",
    "\n",
    "The first step in this process is to actually generate the images:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "generation_response = openai.Image.create(\n",
    "    prompt='A cyberpunk monkey hacker dreaming of a beautiful bunch of bananas, digital art',\n",
    "    size='1024x1024',\n",
    "    n=2\n",
    ")\n",
    "\n",
    "print(generation_response)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Having the response from the `Image.create` call, we download from the URL using `requests`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "import requests\n",
    "\n",
    "# First a little setup\n",
    "image_dir = os.path.join(os.curdir, 'images')\n",
    "# If the directory doesn't exist, create it\n",
    "if not os.path.isdir(image_dir):\n",
    "    os.mkdir(image_dir)\n",
    "\n",
    "# With the directory in place, we can initialize the image path (note that filetype should be png)\n",
    "image_path = os.path.join(image_dir, 'generated_image.png')\n",
    "\n",
    "# Now we can retrieve the generated image\n",
    "image_url = generation_response[\"data\"][0][\"url\"]  # extract image URL from response\n",
    "generated_image = requests.get(image_url).content  # download the image\n",
    "with open(image_path, \"wb\") as image_file:\n",
    "    image_file.write(generated_image)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "With the image downloaded, we use the [Pillow](https://pypi.org/project/Pillow/) library to open and display it:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from PIL import Image \n",
    "\n",
    "display(Image.open(image_path))"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.3"
  },
  "vscode": {
   "interpreter": {
    "hash": "3a5103089ab7e7c666b279eeded403fcec76de49a40685dbdfe9f9c78ad97c17"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}