Machine Learning — Cloud-Native Model Deployments

In this post, we are going to learn about the cloud-native machine learning model deployments.

Cloud-Native Deployments

First and foremost, let’s understand the meaning of cloud-native deployments. If we are building an application or a service and we can deploy this application or the service on any cloud platform without much ado, it could be called cloud-native deployment. And the way it is made possible is through container technologies such as Docker. It’s required to wrap the applications or the services within the containers and move the container’s images onto the cloud services, such as AWS ECS, AWS EKS, Google Kubernetes Engine, etc. The following are some important aspects of achieving the cloud-native deployments:

  • Containerizing the applications: First and foremost, the application or the service would need to be wrapped within the container.
  • Storing container images in container repositories: The next important thing is to identify the container repository. The main use of the container repository is to store the container images and maintain the versioning of these images. In the development environment, you could use a local container repository such as the Nexus repository or Jfrog repository. On the cloud platform, you could easily find the services that can help in storing the container images. An example of such a service is the AWS elastic container repository (ECR).
  • Using a container orchestration tool for the deployments: The next important thing is to set up a container orchestration development environment. The container orchestration tool such as kubernetes is an important tool adopted by many.

Once we have the container, container repository, and the container orchestration tool, we are ready for the cloud-native machine learning model deployments. In the next section, we will learn how to use some of these technologies for deploying machine learning models on cloud platforms.

Cloud-Native ML Model Deployments

The following diagram represents the cloud-native ML model deployments:

Image title

Fig 1. AWS ECS Cloud-native ML Model Deployments

For the cloud-native ML model deployments, the following are some of the important aspects:

  • Machine learning engineers or data scientists work on building the models, optimizing the models’ performance, generating pickle files, and uploading the pickle files on a cloud storage platform such as AWS S3. This workflow could also be achieved using tools such as GitLab and continuous integration tools such as Jenkins. In the diagram, you will see that the ML engineers build the model and commit and push the model code in the code repository such as GitlLab. As a result, a Jenkins build is triggered. As part of the Jenkins build job, the model is generated and pushed on a cloud storage platform, such as AWS S3.
  • Once the model is ready, the next step is to have the Python developers update the flask-related code and configurations. Refer to the diagram for the details. The developers then push their code in a code repository such as GitLab. As a result, the Jenkins build is triggered. The Jenkins build job results in some of the following:
    • Running the build
    • Building the container image
    • Pushing the container images on a cloud container repository service such as AWS ECR
    • Updating the cloud container orchestration services such as AWS ECS. As soon as the cloud container services are updated, the existing containers are shut down based on some policies. New containers with the latest images uploaded in the container repository are started. As a result, the new container loads the latest pickle file uploaded in cloud storage, such as S3.
    • The new requests are then served with a prediction created using the latest pickle file.


In this post, we learned about how to use cloud services for deploying machine learning models. The main idea is to containerize the service, such as a flask service, which loads pickle files from the cloud storage service and serves the request with the predictions.

This UrIoTNews article is syndicated fromDzone