Deploy model with FastAPI and Heroku

In this blog we will see how we can deploy our model with FastAPI and Heroku. Below technologies will be used:

• FastAPI
• Heroku
• Docker
• Github workflow
• AWS

Our model will classify images of dogs and cats.

First Generating the model

We use the data from https://www.kaggle.com/c/dogs-vs-cats . A model is creted using pretrained resnet model. The kaggle notebook is available at https://github.com/njoysubho/datascience/blob/master/dog-vs-cat.ipynb . Not really a SOTA model but it is good enough for our purposes.

We saved the model in AWS S3. We will use the model in our FastAPI server.

The FastAPI service

Our FastAPI server will be a simple REST API. It has one enpoint `/pet` which will accept an image file and return the prediction. Our code look like this:

As you can see the code is a pretty straight forward. We read the image file and transform it to a tensor. We load the model and make a prediction.

The complete code can be found at https://github.com/njoysubho/fastapi-dog-cat.

Now that we have our model and the FastApi service we can now deploy it.

The Deployment

First an application need to be created on heroku. I named the application `dog-cat-fastapi`.

We can upload the whole code as zip and heroku will automatically deploy the code, however I decided to use docker image as I also wanted to run the code locally.

The dockerFile looks like

A few things to notice here -

• `RUN aws s3 cp s3://datascience-sab/${MODEL_NAME}.pth .` command will pull the model file from S3 bucket.
• In order to access aws S3 it needs credential, for this worklfow we are passing the credential from github secret and set those in ENV.
• When I first deployed the app I could not connect to it , turned out that Heroku does not support the `EXPOSE` command and it sets the random port as env variable named `PORT`an we can use that to expose the port. the belo command will make the app avilable in the port
  `CMD gunicorn -k uvicorn.workers.UvicornWorker — bind 0.0.0.0:$PORT app.main:app`

Finally we make our github workflow below is the step to deploy on heroku

we pass app name, secrets the model name as build args to `heroku container:push` command.

After this we use `heroku container:release` command to release the app.

The complete github workflow can be found https://github.com/njoysubho/fastapi-dog-cat/blob/main/.github/workflows/build-release-deploy.yml.

With this we can deploy our app to heroku.

Below are some improvements that can be done —

• Enhance the model .
• Introduce model versioning.
• Improved logging, tracing and metrics for the service.
• Also clean up docker image and stream line all RUN and pip commands.

Thats it for now. It was really satisfying for me to learn about the fastapi framework and deploy a model on heroku.