Easy Step-by-Step Guide – Deployment of Machine Learning Models on AWS Serverless Architecture

My purpose in this post is to provide an easy step-by-step guide on how to deploy machine learning on an AWS serverless architecture. I will deploy the same web app as discussed here using AWS Fargate, a serverless architecture for containers. Python is the programming language I am using to build and test the application along with the libraries and dependencies. The article will cover the entire workflow starting with building a docker image and uploading it onto the Amazon Elastic Container Registry and then creating a cluster, as well as defining and executing tasks on AWS-managed infrastructure. Once you are done, you will be able to deploy a fully functional containerized web app without any infrastructure provisioning on AWS.  The code for the application is available on my GitHub repository. The application is available at: http://heart-healthy.adataanalyst.com:8501/

Overview of Tasks for Deployment of Machine Learning Models

  • Build a cross-platform web app that incorporates online prediction functionality
  • Create a Dockerfile
  • Build and push a Docker image on Amazon Elastic Container Registry
  • Create and execute a task to deploy the app using AWS Fargate’s serverless infrastructure.

Technologies Used for Deployment of Machine Learning Models


Streamlit is an open-source framework for the rapid development of interactive, highly interactive, and fast machine learning and data science web applications. It is extremely well designed and easy to use. 


The company Docker makes a product (also called Docker) that enables users to create, run and manage containers. LXD and LXC are less famous alternatives to Docker’s containers, while Docker is by far the most popular. Docker helps you build, deploy, and run applications by using containers. An application is packaged in a container with all of its parts, including libraries and other dependencies, and distributed as a single package. The Dockerfile, which is really just a set of instructions, is used to create a docker image. Container orchestration is the whole process of managing hundreds and thousands of containers.

Amazon Web Services (AWS)

Cloud computing can be accessed through Amazon Web Services (AWS). AWS offers over 175 services from data centers around the world. AWS has two services for container orchestration:

  • Amazon Elastic Container Service (ECS) is Amazon’s proprietary container orchestration platform. The concept behind ECS is similar to Kubernetes (both are orchestration platforms). services). ECS is AWS-native, so it can only be used on AWS infrastructure. 
  • Amazon Elastic Kubernetes Service (Amazon EKS) uses Kubernetes, a publicly available open-source project for deployment on many cloud providers. EKS makes it easy to start, run, and scale Kubernetes applications on-premises or in the AWS cloud. AWS is an easy way to run EKS clusters fully managed.

There are many differences between EKS and ECS in terms of pricing, compatibility, and security, though the purpose of both is to orchestrate containerized applications. No one solution is best. The choice depends on the use case.

AWS Fargate — AWS Serverless Architecture for running containers

Building your web application with Fargate is easy. It makes use of AWS Elastic Container Service and Amazon Elastic Kubernetes Service to use a serverless compute engine. It minimizes the need for constant server provisioning and management, lets you buy a specific amount of resources accordingly for each application, and ensures increased security through application isolation by design. It eliminates the need to select instances and scale cluster capacity since Fargate allocates the right amount of computing. Using containers will save you from over-provisioning and paying for additional servers.

AWS Fargate

Follow these simple steps to deploy the app on AWS Serverless Architecture (Fargate):

Step 1: Create a Repository in Elastic Container Registry (ECR)

1. Login to your AWS console and search for Elastic Container Registry


2. Create a new repository

Create a new Repository in ECR

3. Click on button named “Create Repository”

4. Click on “View push commands”

View push commands

Step 2 – Execute Push Commands

Push commands allow us to build a docker image push the image to our newly AWS repository. For this, we need to navigate to our project folder where the Dockerfile and the rest of the code are available on our local computer. Execute the commands by copying and pasting in using AWS CLI:

1. Retrieve an authentication token and authenticate your Docker client to your registry

2. Build your Docker image. We can skip this step if your image is already built

3. After the build completes, tag your image so you can push the image to this repository

4. Push the image to our newly created AWS repository

Push Commands

Step 3 – Confirm the uploaded image

Click on the repository we created and we will see an image URI of the uploaded image in the step above. Copy the image URI as we need it later.

ECR Repository

 Step 4 — Create and Configure a Cluster

1. Click on “Clusters” on left-side menu

2. Click on create cluster

3. Select “Networking only” and click Next step

Create Cluster

4. Configure Cluster (Enter cluster name) and click on Create

Cluster Name

5. Click on “Create”

Cluster Created

Step 5 – Create a new Task definition

A task definition is required to run Docker containers in Amazon ECS.

  1. Click on “Create new task definition”
Create Task Definition

2. Create a new task definition

3. Select “FARGATE” as launch type


4. Select Launch Type Compatibility

5. Fill in the details

Task Definition Name
Task IAM Role

6. Click on “Add Containers” and fill in the details

Container Name

7. Add the port mapping

Port Mapping

8. Click “Create Task” on the bottom right

Create Task

Step 6 – Execute Task Definition

1. Execute the task by clicking “Run Task” under Actions

Run Task

2. Click on “Switch to launch type” to change the type to Fargate

Fargate – Task

3. Select the VPC and Subnet from the dropdown


4. Click on “Run Task” on bottom right

Step 7 – Allow inbound port 8501 from Network settings

We need to port 8501 (used by Streamlit) by creating a new rule.

1. Click on Task

2. Click on ENI Id


3. Click on Security groups

Security Group

4. Scroll down and click on “Edit inbound rules”

TCP 8501

5. Add a Custom TCP rule of port 8501

6. Remove HTTP rule of port 80

We are able to access the application using AWS serverless architecture.

Use the public IP address with port 8501 to access the application as shown below:

Web Application

Optional : Create a sub-domain to the DNS zone

DNS records specify how internet traffic is routed for the domain. For example, you could route traffic for the apex of your domain, such as example.com, to one instance, and route traffic for a subdomain, such as blog.example.com, to a different instance.

  1. On the DNS zone management page, choose Add record.
  2. A record maps a domain, such as example.com, or a subdomain, such as blog.example.com, to a web server’s or instance’s IP address, such as In the Subdomain text box, enter the target subdomain for the record. In the Resolves to the text box, enter the target IP address for the record, select your running instance.

The code for the application is available on my GitHub repository. The application is available at: http://heart-healthy.adataanalyst.com:8501/

Don’t forget to share and follow!

Remember, don’t forget to share this post so that other people can see it too! Also, make sure you subscribe to this blog’s mailing list, follow me on Twitter so that you don’t miss out on any useful posts!

I read all comments, so if you have something to say, something to share, or questions and the like, leave a comment below!

Leave a Reply