Model_Serving

This Project is the reposiory for solving AI Engineer Party

Architecture

Procedure

• Download the data and upload it to bigquery.
• Load data and learn with AutoML.
• Store the learned weight in storage.
• Deploy the model. In this case, load the weight from storage and then inference.

Dependencies

• Python 3.6+
• Pytorch 1.0+
• nni(AutoML)
• google-cloud-storage 1.18.0
• google-cloud-bigquery 1.18.0

Quick Tutorial and Usage

1.set your GCE region. In my case us central1

$ gcloud config set compute/zone us-central1

2. install Google Cloud SDK

$ https://cloud.google.com/sdk/docs/quickstarts

3. install google cloud python API

$ pip install --upgrade google-cloud-storage
$ pip install --upgrade google-cloud-bigquery

4. Download MNIST dataset

$ tools/make_data.py

5. Upload on BigQuery

$ bq load --source_formag=CSV -F":" mnist.train data/train.txt.gz\
  "key:integer, image:string,label:integer"
$ bq load --source_formag=CSV -F":" mnist.test data/test.txt.gz\
  "key:integer, image:string,label:integer"

6. Check BirQuery python API operation

from google.cloud import bigquery
client = bigquery.Client()
query = ("SELECT image, label FROM mnist.train")
query_job = client.query(
  query,
) #API request -starts the query
for row in query_job: # API request -fetches results
  print(row)

7. Save data from Bigquery

 python tools/data_from_bq.py

8. Install AutoML nni of Microsoft

pip install --upgrade nni

click here for details(https://github.com/microsoft/nni)

9. Training using AutoML

nnictl create --config config.yml

10. Create Google Storage Bucket to upload saved model state_dict

$ PROJECT_ID = $(gcloud config list project --format "value(core.projcet)")
$ BUCKET="${PROJECT_ID}-ml"

#create bucker
$ gsutil mb -c regional -l us-central1 gs://${BUCKET}

#upload saved model
$ gsutil -m cp -R save_model/model_param.pth gs://${BUCKET}

11. Deploy google functions

$ cd gdeploy
$ gcloud beta functions deploy [function_name] --runtime python37 --trigger-http

12. Test API usgin 'curl' click here for details about curl

$ curl -X  POST\
curl -X POST -H "Content-Type:application/json"  -d '{"url":"[image url]"}'

Reference

• nni AutoML
• google cloud platform
• Tae Hwan Jung repository

Author

• Juntae Kim, Korea University DAVIAN LAB