Skip to content

Generate a Python SDK from a Swagger file

Jump to bottom
Laurent Mazuel edited this page Sep 13, 2017 · 35 revisions

This document provides instructions and guidelines on how to generate a Python SDK from a Swagger file.

Table of contents:

Step 1: Generating your first client for local testing

If you're discovering Python for the first time, we have great intros using Notebooks on Azure. Click the "Show me more samples" button to see them.

Once you're comfortable with Python:

  • You need to install Autorest: aka.ms/autorest/install. Minimal Autorest version is "1.2.1". Please refer to the Autorest CLI doc for details about parameters.

  • To call Autorest, you need the following options:

    • Required parameter: --payload-flattening-threshold=2

    • About the generator:

      • If your endpoint is ARM, add --python --azure-arm=true
      • If not, add --python. If your client might ask authentication, add --add-credentials

And that's it! You should now have Python code ready to test. Note that this generation is for testing only and should not be sent to a customer or published to PyPI.

Example

ARM management Swagger:

autorest --version=latest --python --azure-arm=true --payload-flattening-threshold=2 --input-file=myswagger.json

Not-ARM Swagger:

autorest --version=latest --python --payload-flattening-threshold=2 --add-credentials --input-file=myswagger.json

Step 2: Generating your first client that you want to release

Generate a package

First, your Swagger needs to at least be a PR in the official Microsoft Swagger RestAPI repo. This allows you to determine the name of your future package.

  • If your root folder is prefixed by arm-:
    • Your package name will be prefixed with azure-mgmt-
    • Your namespace will start with azure.mgmt
  • If not:
    • Your package name will be prefixed with azure-
    • Your namespace will start with azure

For example, the arm-compute folder will create the package azure-mgmt-compute and will use the namespace azure.mgmt.compute.

Let's assume for now that your Swagger is in arm-compute

To call Autorest, you need the following options:

  • Required parameters:

    --payload-flattening-threshold=2 --license-header=MICROSOFT_MIT_NO_VERSION --namespace=azure.mgmt.compute --package-name=azure-mgmt-compute --package-version=0.1.0

  • About the generator:

    • If your endpoint is ARM, add --python --azure-arm=true
    • If not, add --python. If your client might ask authentication, add --add-credentials

Example

ARM Swagger:

autorest --version=latest --python --azure-arm=true --payload-flattening-threshold=2 --license-header=MICROSOFT_MIT_NO_VERSION --namespace=azure.mgmt.storage --package-name=azure-mgmt-storage --package-version=0.1.0 --input-file=arm-storage/2016-12-01/swagger/storage.json

Testing

We recommend you to test your package. We made a tutorial to help you with this process: https://github.com/Azure/azure-sdk-for-python/wiki/Contributing-to-the-tests

Note this is not mandatory, but testing is important, and Autorest has regularly issues that can only be seen in one specific client in one specific context (so code generation does not protect you from bugs)

Create a PR on the repo

You can create a PR on this repo with your package. If you called Autorest with the --package-name option, you should see in your folder a setup.py file generated by Autorest.

Example: if your package name is azure-mgmt-logic, Autorest will generate a azure-mgmt-logic folder in a way that azure-mgmt-logic/setup.py exists.

In your PR, you need to update the configuration file called swagger_to_sdk_config.json as well.

A typical new entry looks like this:

   "logic": {
      "autorest_options": {
        "input-file": "arm-logic/2016-06-01/swagger/logic.json",
        "namespace": "azure.mgmt.logic",
        "package-version": "2.0.0"
      },
      "output_dir": "azure-mgmt-logic/azure/mgmt/logic",
      "build_dir": "azure-mgmt-logic"
    },

Note that if you use the new MD configuration file of Autorest, your configuration can be simplified to (see next section for the meaning of "generated_relative_base_directory"):

   "logic": {
      "markdown": "arm-logic/readme.md",
      "output_dir": "azure-mgmt-logic/azure/mgmt/logic",
      "build_dir": "azure-mgmt-logic",
      "generated_relative_base_directory": "Generated/Python/azure/mgmt/logic",
    },

The MD should have at least this structure:

    ## Inputs

    ``` yaml 
    input-file:
      - ./2016-12-01/swagger/backupManagement.json
      - ./2016-08-10/swagger/operations.json
    azure-arm: true
    ```

    ## Code Generation

    ### C#

    ```yaml $(csharp)
    csharp:
      namespace: Microsoft.Azure.Management.RecoveryServices.Backup
      output-folder: $(output-folder)Generated/CSharp
    ```

    ### Python

    ```yaml $(python)
    python:
      namespace: azure.mgmt.recoveryservicesbackup
      package-version: 0.1.0
      output-folder: $(output-folder)Generated/Python
      payload-flattening-threshold: 2
      license-header: MICROSOFT_MIT_NO_VERSION
    ```

    ```yaml $(python) && $(create)
    python:
      package-name: azure-mgmt-recoveryservicesbackup
    ```

Important points:

  • prefix all "output-folder" in language section by $(output-folder) to allow the CI to customize the path, and configure "generated_relative_base_directory" accordingly.
  • Separate "package-name" in a different since this parameter should be used only creation of package (not update).

This will be used by the CI to update your package automatically in the future.

Note that the "setup.py" files will be replaced by more optimized files using a CookieCutter template published here.

You must update in your PR the package_service_mapping.json to allow your documentation to be on docs.microsoft.com:

	"azure-mgmt-containerregistry": {
		"service_name": "Container Registry",
		"category ": "Management"
	},

Step 3: Updating your package

You changed the Swagger already used

When you have a PR merged on https://github.com/Azure/azure-rest-api-specs, if your configuration file is up to date, then a PR will be automatically created here. The link will appear in your Swagger PR. You can review the Python PR and comment on it if necessary (even just a LGTM).

New ApiVersion or not using MD file

Open a PR here with the correct updates of the configuration file.

  • If your Swagger PR is not merged yet, that's it: your package will be updated when the Swagger PR is merged.
  • If your Swagger PR is already merged, there is no automatic way to regenerate your package. You can regenerate manually yourself and create a PR for the change, or you can simulate the CI generation (there is a process for that) or simply ask me @lmazuel to do it.

Step 4: Releasing to PyPI

We have no private server to mock PyPI for testing. However, Python is able to install a package "like it was in PyPI" from Github with the following command (example here for azure-mgmt-consumption)

pip install "git+https://github.com/Azure/azure-sdk-for-python#subdirectory=azure-mgmt-consumption&egg=azure-mgmt-consumption"

To insure all packages are released correctly and meets the quality we expect (for instance, there is some tricky configuration details on how to build the package to improve the performance on the client machine), only us in the Python team can release on PyPI for now.

Clone this wiki locally