Deploy Machine Learning Models to Production

Deploy Machine Learning Models to Production: Flask, Streamlit, Docker, Kubernetes, and Google Cloud Platform

Introduction

Machine learning (ML) development does not end when a model achieves high accuracy in a notebook. The real engineering challenge begins when that model must serve predictions reliably to users and applications. A production ML system needs an API, reproducible dependencies, monitoring, scalability, security, and a deployment strategy.

A typical workflow can move from a trained Python model to a web interface with Streamlit, an inference API with Flask, a portable container using Docker, and scalable orchestration through Kubernetes on Google Cloud Platform (GCP).

The basic production path can be represented as:

Model → Flask API → Docker Container → Kubernetes → Google Cloud → End User

🚀 This architecture is useful for engineering students learning MLOps as well as professionals building scalable ML services.

Background Theory

Machine learning systems generally contain several layers:

1. Data layer — collects and prepares input data.
2. Training layer — trains the ML algorithm.
3. Model layer — stores the trained model.
4. Application layer — exposes predictions.
5. Infrastructure layer — packages and runs the application.
6. Orchestration layer — manages containers and scaling.
7. Cloud layer — provides computing, networking, storage, and monitoring.

A production system therefore differs considerably from a Jupyter Notebook.

From Notebook to Production

Consider a simple classification model:

[\hat{y}=f(X;\theta)]

where:

• (X) = input features
• (\theta) = trained model parameters
• (\hat{y}) = predicted output

During experimentation, a developer might simply execute:

prediction = model.predict(X)

Production requires additional engineering:

[\text{Request}
\rightarrow
\text{Validation}
\rightarrow
\text{Preprocessing}
\rightarrow
\text{Inference}
\rightarrow
\text{Response}]

The model is only one component of this pipeline.

Why Containerization Matters

A model may work perfectly on a developer’s machine but fail elsewhere because of:

• Python version differences
• incompatible package versions
• missing system libraries
• incorrect environment variables
• different operating-system dependencies

Docker addresses this problem by packaging the application and its dependencies into a reproducible container.

Definition

Machine learning model deployment is the process of making a trained ML model available to an application, service, user, or automated system so that it can generate predictions using new data.

A production ML deployment can be defined as:

[D = M + API + C + O + Cloud]

where:

• (M) = machine learning model
• (API) = application programming interface
• (C) = containerization
• (O) = orchestration
• Cloud = cloud infrastructure

Flask

Flask is a lightweight Python web framework that can expose a machine learning model through HTTP endpoints.

For example:

POST /predict

could receive:

{
  "age": 35,
  "income": 72000,
  "experience": 8
}

and return:

{
  "prediction": 1,
  "probability": 0.91
}

Streamlit

Streamlit provides a convenient way to build interactive ML interfaces using Python.

Instead of sending JSON manually, a user could interact with:

• text fields
• sliders
• dropdown menus
• buttons
• charts

🎯 Streamlit is particularly useful for demonstrations, internal tools, prototypes, and data applications.

Docker

Docker packages the application into an image that can run consistently across environments.

Kubernetes

Kubernetes manages containerized applications and provides capabilities such as:

• deployment
• scaling
• service discovery
• rolling updates
• workload management
• self-healing

Google Cloud Platform

GCP provides cloud infrastructure on which containers, Kubernetes clusters, storage systems, networking, and monitoring tools can operate.

Step-by-Step Machine Learning Deployment

Step 1: Train and Save the Model

Suppose we train a simple scikit-learn model.

from sklearn.ensemble import RandomForestClassifier
import joblib

model = RandomForestClassifier(
    n_estimators=100,
    random_state=42
)

model.fit(X_train, y_train)

joblib.dump(model, "model.pkl")

The resulting file might be:

model.pkl

The model should ideally be accompanied by the exact preprocessing pipeline used during training.

Step 2: Build the Flask API

Create:

app.py

A simplified API might look like:

from flask import Flask, request, jsonify
import joblib

app = Flask(__name__)

model = joblib.load("model.pkl")

@app.route("/predict", methods=["POST"])
def predict():
    data = request.get_json()

    features = [[
        data["age"],
        data["income"],
        data["experience"]
    ]]

    prediction = model.predict(features)[0]
    probability = model.predict_proba(features)[0].max()

    return jsonify({
        "prediction": int(prediction),
        "probability": float(probability)
    })

@app.route("/health", methods=["GET"])
def health():
    return jsonify({"status": "healthy"})

if __name__ == "__main__":
    app.run(host="0.0.0.0", port=8080)

The /health endpoint is especially important for container orchestration.

Step 3: Add Dependencies

Create:

requirements.txt

For example:

Flask
scikit-learn
joblib
gunicorn

For production, package versions should normally be pinned or otherwise controlled to improve reproducibility.

Step 4: Test the API Locally

Run:

python app.py

Then send a request:

curl -X POST http://localhost:8080/predict \
-H "Content-Type: application/json" \
-d '{"age":35,"income":72000,"experience":8}'

A successful response could look like:

{
  "prediction": 1,
  "probability": 0.91
}

Step 5: Create a Dockerfile

FROM python:3.11-slim

WORKDIR /app

COPY requirements.txt .

RUN pip install --no-cache-dir -r requirements.txt

COPY model.pkl .
COPY app.py .

EXPOSE 8080

CMD ["gunicorn", "--bind", "0.0.0.0:8080", "app:app"]

The resulting architecture is:

Client
   │
   ▼
Flask API
   │
   ▼
ML Model
   │
   ▼
Prediction

Step 6: Build the Docker Image

docker build -t ml-api:v1 .

Run it:

docker run -p 8080:8080 ml-api:v1

Now the model is running inside a container.

Step 7: Add a Streamlit Interface

A simple interface might be:

import streamlit as st
import requests

st.title("🤖 ML Prediction System")

age = st.number_input("Age", 18, 100, 35)
income = st.number_input("Income", 0, 500000, 72000)
experience = st.number_input("Experience", 0, 50, 8)

if st.button("Predict"):
    payload = {
        "age": age,
        "income": income,
        "experience": experience
    }

    response = requests.post(
        "http://localhost:8080/predict",
        json=payload
    )

    result = response.json()

    st.success(f"Prediction: {result['prediction']}")
    st.info(f"Confidence: {result['probability']:.2%}")

The Streamlit application becomes the presentation layer while Flask remains responsible for inference.

Step 8: Push the Container to Google Cloud

A production workflow can use Artifact Registry to store container images.

Conceptually:

Developer
   │
   ▼
Docker Build
   │
   ▼
Container Registry
   │
   ▼
Kubernetes Cluster
   │
   ▼
ML API

Before deploying to GCP, configure the appropriate project, region, authentication, APIs, repository, and IAM permissions.

Step 9: Deploy with Kubernetes

A simplified Kubernetes Deployment could look like:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: ml-api
spec:
  replicas: 3
  selector:
    matchLabels:
      app: ml-api
  template:
    metadata:
      labels:
        app: ml-api
    spec:
      containers:
        - name: ml-api
          image: REGION-docker.pkg.dev/PROJECT_ID/ml-repo/ml-api:v1
          ports:
            - containerPort: 8080
          resources:
            requests:
              cpu: "250m"
              memory: "512Mi"
            limits:
              cpu: "1"
              memory: "1Gi"

Three replicas mean Kubernetes attempts to maintain three running instances.

Step 10: Expose the Application

A Kubernetes Service can expose the deployment:

apiVersion: v1
kind: Service
metadata:
  name: ml-api-service
spec:
  selector:
    app: ml-api
  ports:
    - port: 80
      targetPort: 8080
  type: LoadBalancer

The resulting architecture becomes:

                 ┌──────────────┐
                 │    Users     │
                 └──────┬───────┘
                        │
                        ▼
                ┌───────────────┐
                │ Load Balancer │
                └───────┬───────┘
                        │
             ┌──────────┼──────────┐
             ▼          ▼          ▼
          ┌─────┐    ┌─────┐    ┌─────┐
          │ Pod │    │ Pod │    │ Pod │
          │ ML  │    │ ML  │    │ ML  │
          └─────┘    └─────┘    └─────┘

Comparison

Flask vs. Streamlit

Flask is primarily an API framework.

Streamlit is primarily an application/UI framework.

Therefore:

[\text{Flask} \neq \text{Streamlit}]

They can instead complement each other.

Streamlit UI
     │
     ▼
Flask API
     │
     ▼
ML Model

Docker vs. Kubernetes

Docker answers:

How do I package and run my application consistently?

Kubernetes answers:

How do I manage many running containers reliably?

This distinction is fundamental for understanding modern cloud-native ML systems.

Diagrams and Tables

Production Architecture

                   ┌─────────────────┐
                   │      User       │
                   └────────┬────────┘
                            │
                            ▼
                    ┌──────────────┐
                    │  Streamlit   │
                    │     UI       │
                    └──────┬───────┘
                           │ HTTP
                           ▼
                    ┌──────────────┐
                    │  Flask API   │
                    └──────┬───────┘
                           │
                           ▼
                    ┌──────────────┐
                    │ ML Model     │
                    │ model.pkl    │
                    └──────────────┘

Containerized Architecture

Google Cloud
│
└── Kubernetes
    │
    ├── Pod 1 → Flask + Model
    ├── Pod 2 → Flask + Model
    └── Pod 3 → Flask + Model

Key Production Metrics

Examples

Example 1: Customer Classification

An e-commerce company can deploy a model predicting whether a customer is likely to purchase.

Input:

[X=[Age,Income,Visits,CartValue]]

Output:

[P(Purchase=1|X)=0.87]

The Flask API receives the customer information and returns the probability.

Example 2: Engineering Predictive Maintenance

A manufacturing system can use:

[X=[Temperature,Vibration,Pressure,Speed]]

to estimate machine failure probability.

For example:

{
  "temperature": 83.4,
  "vibration": 7.2,
  "pressure": 4.8,
  "speed": 1800
}

The model might return:

{
  "failure_probability": 0.78
}

⚙️ Engineers can then trigger inspection before catastrophic equipment failure.

Example 3: Financial Risk Classification

A model can estimate a risk category from:

• transaction history
• account characteristics
• customer behavior
• payment patterns

The API can serve the model to multiple internal applications without embedding the ML model into each application.

Real-World Application

Production ML deployment is particularly valuable when prediction must be consumed repeatedly.

Healthcare Systems

ML APIs can support medical image classification, risk prediction, or operational forecasting. Such systems require strict validation, privacy, security, and human oversight.

Manufacturing

Predictive-maintenance models can process sensor data continuously.

A simplified relationship is:

[Risk=f(T,V,P,S)]

where:

• (T) = temperature
• (V) = vibration
• (P) = pressure
• (S) = speed

Transportation

Models can estimate:

• travel time
• traffic congestion
• fuel consumption
• maintenance requirements

E-Commerce

Production models can power:

• recommendation engines
• customer segmentation
• demand forecasting
• fraud detection
• search ranking

Engineering Design

ML models can be exposed through web applications so engineers can evaluate designs without directly interacting with the underlying Python environment.

Common Mistakes

Mistake 1: Deploying the Notebook Directly

A notebook is an experimentation environment, not necessarily a production service.

Solution: Separate training, inference, API, and infrastructure components.

Mistake 2: Ignoring Preprocessing

Suppose training uses:

[X’=\frac{X-\mu}{\sigma}]

but production sends raw (X).

The model may produce unreliable predictions.

Solution: Package preprocessing with the model pipeline.

Mistake 3: Using Development Servers

Flask’s development server should not normally be treated as a production application server.

Solution: Use a production WSGI server such as Gunicorn behind an appropriate production architecture.

Mistake 4: No Health Endpoint

Kubernetes needs to know whether an application is functioning.

Solution:

GET /health

should return a meaningful health response.

Mistake 5: No Resource Limits

A model can consume excessive CPU or memory.

Solution: Define Kubernetes resource requests and limits.

Mistake 6: Hard-Coding Secrets

Never place API keys or credentials directly into source code.

🔐 Use appropriate secret-management mechanisms and IAM controls.

Challenges and Solutions

Scaling

If one container processes (R) requests/second and traffic reaches (T) requests/second, a rough initial replica estimate is:

[N \geq \frac{T}{R}]

For example, if:

[T=300 \text{ requests/s}]

and each replica handles approximately:

[R=100 \text{ requests/s}]

then:

[N \geq 3]

In real systems, additional capacity should normally be reserved for variability and failures.

Case Study

Production Predictive-Maintenance Platform

Consider a hypothetical industrial company operating hundreds of machines.

The engineering team trains a failure-prediction model using historical sensor measurements.

The original model runs inside a Python notebook.

The team converts the system into:

Sensor Data
     ↓
Preprocessing
     ↓
ML Model
     ↓
Flask API
     ↓
Docker
     ↓
Kubernetes
     ↓
Google Cloud
     ↓
Engineering Dashboard

Deployment Strategy

The team initially deploys three replicas.

Each replica contains:

Python
Flask
Gunicorn
Model
Dependencies

Kubernetes distributes requests between replicas.

If traffic increases, additional replicas can be created.

If one pod fails, Kubernetes can replace it.

Engineering Outcome

The main improvement is not simply model accuracy.

The organization gains:

• repeatable deployment
• centralized inference
• scalable infrastructure
• controlled versions
• easier monitoring
• improved operational reliability

This illustrates a fundamental MLOps principle:

A production ML model is a software system, not merely a trained algorithm.

Essential Tips

1. Version Everything

Version:

• source code
• Docker images
• models
• configuration
• dependencies

For example:

ml-api:v1
ml-api:v2
ml-api:v3

2. Keep Training Separate from Inference

Training may require GPUs and large datasets.

Inference may require low latency and predictable resource consumption.

Separating them often produces a cleaner architecture.

3. Monitor the Model, Not Just the Server

CPU utilization can be normal while model quality deteriorates.

Monitor:

[\text{System Health} + \text{Data Quality} + \text{Model Quality}]

4. Automate Deployment

A mature workflow can be:

Git Push
   ↓
Tests
   ↓
Build Docker Image
   ↓
Security Checks
   ↓
Container Registry
   ↓
Kubernetes Deployment
   ↓
Monitoring

5. Design for Failure

Production infrastructure should assume that:

• containers can crash
• networks can fail
• dependencies can become unavailable
• traffic can suddenly increase

Resilience should therefore be designed rather than assumed.

FAQs

What is the difference between Flask and Streamlit?

Flask is commonly used to build APIs and backend services, while Streamlit is designed primarily for interactive Python-based web applications and data/ML interfaces. They can be used together.

Why should I use Docker for an ML model?

Docker packages the model, application, runtime, and dependencies into a reproducible environment. This significantly reduces environment-related deployment problems.

Do I need Kubernetes for every ML project?

No. A small prototype may not need Kubernetes. Kubernetes becomes more valuable when you require orchestration, multiple replicas, automated scaling, resilience, and more sophisticated deployment management.

Why use Google Cloud Platform?

GCP provides cloud infrastructure and managed services that can support scalable ML applications, container registries, Kubernetes environments, networking, storage, monitoring, and security.

Should the model be inside the Docker image?

For relatively small and stable models, packaging the model with the application image can be practical. For larger or frequently updated models, external model storage and controlled model loading may be preferable.

How can I scale an ML API?

You can increase the number of application replicas and distribute requests among them. Kubernetes can help automate this process based on appropriate workload and resource metrics.

How do I monitor a deployed ML model?

Monitor infrastructure metrics such as CPU, memory, latency, and errors, but also monitor ML-specific metrics such as prediction distributions, data quality, drift, and—when labels become available—model performance.

Is Streamlit suitable for enterprise production?

It can be useful for internal applications, dashboards, demonstrations, and certain production workloads. However, applications requiring complex APIs, highly customized frontends, or large-scale public traffic may benefit from a more specialized frontend/backend architecture.

Conclusion

Deploying a machine learning model to production requires much more than saving a .pkl file and calling predict().

A practical production architecture can combine Flask for inference APIs, Streamlit for interactive interfaces, Docker for reproducible packaging, Kubernetes for orchestration, and Google Cloud Platform for scalable cloud infrastructure.

The complete engineering pipeline can be summarized as:

[\boxed{
Model
\rightarrow
API
\rightarrow
Container
\rightarrow
Orchestration
\rightarrow
Cloud
\rightarrow
Monitoring}]

🚀 The most important lesson is that successful ML deployment sits at the intersection of machine learning, software engineering, DevOps, cloud computing, and systems engineering.

For students, this workflow provides a practical path from an ML experiment to a deployable application. For professionals, it provides the foundation for building systems that can handle real users, changing workloads, software updates, and production failures.

Ultimately, the goal is not simply to make a model predict.

The goal is to make it reliable, scalable, observable, maintainable, secure, and useful in the real world.