Python WSGI/ASGI #

Running Python-based web applications behind the Caddy web server is one of the best standards for modern production deployment. The Python web development ecosystem is divided into two main server interface standards: WSGI (Web Server Gateway Interface) for traditional synchronous frameworks like Django and Flask, and ASGI (Asynchronous Server Gateway Interface) for modern asynchronous/real-time frameworks like FastAPI and Sanic. Because Caddy is written in Go and can’t natively execute Python code, you need an intermediary in the form of a Python Application Server (Gunicorn for WSGI, and Uvicorn for ASGI). Caddy acts at the front gate as a reverse proxy handling automatic HTTPS protocols, data compression, and high-performance static file serving, while Gunicorn/Uvicorn manages Python worker processes in the background. We’ll discuss the WSGI and ASGI integration architectures, practice setting up Flask with Gunicorn and FastAPI with Uvicorn, compose production Django configurations, create service management using systemd, and automate zero-downtime deployment.

Python Deployment Architecture #

Understanding the request data flow is very important when designing your Python server infrastructure.

Caddy stands as the front-line fortress directly receiving internet traffic (ports 80 and 443). When a dynamic page request arrives, Caddy decrypts the SSL/TLS connection, then forwards the request locally through a UNIX socket file to the Python Application Server. Gunicorn or Uvicorn acts as a process manager distributing that request to several worker processes running your Django, Flask, or FastAPI framework code.

flowchart TD
    Internet["Internet (Port 443)"] -->|"HTTPS"| Caddy["Caddy Server"]
    
    Caddy -->|"Static / Media Files"| Disk["Read directly from Disk"]
    Caddy -->|"Dynamic Files / API"| Socket["Send to the UNIX Socket (via unix//run/gunicorn.sock)"]
    
    subgraph Backend["Python Backend"]
        Socket --> AppServer["Gunicorn / Uvicorn Pool (Application Server)"]
        AppServer --> AppPython["Django / FastAPI App (Python Code)"]
    end
    
    Disk --> Response["HTML Response"]
    AppPython --> Response

    style Caddy stroke:#0288d1,stroke-width:2px
    style Backend stroke:#37474f,stroke-width:1px,stroke-dasharray:5,5

This separation ensures the very fast Caddy binary handles all the heavy network I/O operations, while the Python runtime is only focused on processing application logic.

Flask + Gunicorn (WSGI) #

Flask is a very popular synchronous micro-framework (WSGI) known for its simplicity. In production environments, you run Flask under Gunicorn supervision.

1. Preparing a Local Virtual Environment #

You must isolate your project’s Python library dependencies using a virtual environment (venv) to avoid version conflicts with the system’s global libraries:

# Create a virtual environment directory in the project folder
python3 -m venv /var/www/myapp/venv
source /var/www/myapp/venv/bin/activate

# Install Flask and Gunicorn
pip install Flask gunicorn

2. Simple Flask Application Code (app.py) #

# /var/www/myapp/app.py
from flask import Flask, jsonify

app = Flask(__name__)

# Health check endpoint for server status verification by Caddy
@app.get('/health')
def health():
    return jsonify(status='UP', services='healthy')

@app.route('/')
def hello():
    return "<h1>Hello from Flask behind Caddy!</h1>"

if __name__ == '__main__':
    app.run()

3. Running Gunicorn via a UNIX Socket #

Run Gunicorn by defining the worker count and a safe UNIX socket path:

cd /var/www/myapp
# Run Gunicorn with 4 worker processes bound to a socket file
/var/www/myapp/venv/bin/gunicorn \
    --workers 4 \
    --bind unix:/run/myapp/gunicorn.sock \
    --access-logfile /var/log/myapp/access.log \
    --error-logfile /var/log/myapp/error.log \
    --timeout 120 \
    app:app

FastAPI + Uvicorn (ASGI) #

FastAPI is a modern asynchronous framework (ASGI) designed for building high-performance APIs. Because FastAPI is asynchronous, you can’t use plain Gunicorn. You use Uvicorn as workers under Gunicorn management.

1. Installing ASGI Dependencies #

pip install fastapi "uvicorn[standard]" gunicorn

2. FastAPI Application Code (main.py) #

# /var/www/fastapi-app/main.py
from fastapi import FastAPI
from pydantic import BaseModel

app = FastAPI()

class UserItem(BaseModel):
    username: str
    email: str

@app.get("/health")
async def health_check():
    return {"status": "healthy"}

@app.get("/api/users/{user_id}")
async def get_user(user_id: int):
    return {"user_id": user_id, "role": "member"}

@app.post("/api/users")
async def create_user(user: UserItem):
    return {"message": "User created successfully", "data": user}

3. Running ASGI with Uvicorn Workers in Gunicorn #

You combine Gunicorn’s process management advantages with Uvicorn’s asynchronous event-loop performance by defining a custom worker class:

cd /var/www/fastapi-app
# Using UvicornWorker for asynchronous handling
/var/www/fastapi-app/venv/bin/gunicorn \
    --workers 4 \
    --worker-class uvicorn.workers.UvicornWorker \
    --bind unix:/run/fastapi-app/uvicorn.sock \
    --access-logfile /var/log/fastapi/access.log \
    --timeout 120 \
    main:app

Django Production Setup #

Django is a full-stack WSGI framework requiring special attention when entering production environments regarding static assets handling and proxy security header settings.

1. Production settings.py Adjustments #

You must turn off debug mode and tell Django it runs behind the Caddy HTTPS reverse proxy:

# settings.py
DEBUG = False

# Limit the domains allowed to access Django
ALLOWED_HOSTS = ['example.com', 'www.example.com']

# IMPORTANT: Tell Django to trust the secure proxy headers from Caddy.
# Without this, Django assumes the connection is insecure and triggers an HTTPS redirect loop.
SECURE_PROXY_SSL_HEADER = ('HTTP_X_FORWARDED_PROTO', 'https')
USE_X_FORWARDED_HOST = True

# Session cookie and CSRF token security
SESSION_COOKIE_SECURE = True
CSRF_COOKIE_SECURE = True

# Specify the static file storage location when compiled
STATIC_ROOT = '/var/www/django-app/staticfiles'
STATIC_URL = '/static/'

2. Compiling Django Static Assets #

Run the static asset collection command so all CSS/JS files from the Django admin modules are copied to the target directory:

cd /var/www/django-app
# Compile static assets to STATIC_ROOT
/var/www/django-app/venv/bin/python manage.py collectstatic --noinput
# Run database migrations
/var/www/django-app/venv/bin/python manage.py migrate

Caddyfile for Python Apps #

After the Python application server (Gunicorn/Uvicorn) is running and listening on a local UNIX socket, you compose the production Caddyfile. You configure Caddy to serve Django’s /static/ folder directly from disk, while all other requests are routed to the Python application socket:

# Production Caddyfile configuration for Python Apps
example.com {
    log {
        output file /var/log/caddy/python-access.log
        format json
    }
    
    encode zstd gzip
    
    # HTTP security headers
    header {
        Strict-Transport-Security "max-age=31536000; includeSubDomains"
        X-Content-Type-Options "nosniff"
        X-Frame-Options "SAMEORIGIN"
        -Server
    }
    
    # 1. Serve Django static files directly by Caddy (Very Fast)
    @static path /static/* /media/*
    handle @static {
        root * /var/www/django-app
        file_server
        # Aggressive 1-year cache for maximum browser performance
        header Cache-Control "public, max-age=31536000, immutable"
    }
    
    # 2. Forward dynamic requests to the Gunicorn UNIX socket
    handle {
        reverse_proxy unix//run/django-app/gunicorn.sock {
            # Add essential proxy headers for Django/FastAPI
            header_up X-Real-IP {remote_host}
            header_up X-Forwarded-For {remote_host}
            header_up X-Forwarded-Proto {scheme}
            header_up Host {host}
            
            # Health Check Configuration
            health_uri /health
            health_interval 15s
            health_timeout 5s
        }
    }
}

Systemd Service for Gunicorn #

So the Python application server automatically restarts when the Linux OS boots (boot/reboot) or after an unexpected crash, you must create a systemd service definition file:

# Systemd Service File: /etc/systemd/system/myapp.service
[Unit]
Description=Python Application Server Service (Gunicorn)
After=network.target

[Service]
# Run the process under the www-data user for security
User=www-data
Group=www-data
WorkingDirectory=/var/www/myapp

# Load the secret configuration file (.env) in an isolated folder
EnvironmentFile=/etc/myapp/env
Environment="PATH=/var/www/myapp/venv/bin"

# Automatically prepare the temporary UNIX socket directory in /run/
RuntimeDirectory=myapp
RuntimeDirectoryMode=0755

# Main Gunicorn execution command
ExecStart=/var/www/myapp/venv/bin/gunicorn \
    --workers 5 \
    --bind unix:/run/myapp/gunicorn.sock \
    --pid /run/myapp/gunicorn.pid \
    --access-logfile /var/log/myapp/gunicorn-access.log \
    --error-logfile /var/log/myapp/gunicorn-error.log \
    --timeout 120 \
    app:app

# Use the HUP signal to trigger safe configuration reloads without interruption
ExecReload=/bin/kill -s HUP $MAINPID
KillMode=mixed
TimeoutStopSec=5
PrivateTmp=true

# Automatic restart policy on crashes
Restart=on-failure
RestartSec=5

[Install]
WantedBy=multi-user.target

Run the following commands in the terminal to register and activate the new service:

sudo systemctl daemon-reload
sudo systemctl enable myapp.service
sudo systemctl start myapp.service
sudo systemctl status myapp.service

Determining the Number of Gunicorn Workers #

Determining the right number of worker processes in Gunicorn is very important for optimizing your Python application’s performance. Because Python is natively limited by the Global Interpreter Lock (GIL) in synchronous programming, you must run several independent worker processes to maximize your server’s multi-core CPU utilization.

The General Gunicorn Worker Formula (WSGI/Synchronous) #

For synchronous applications like standard Flask or Django, the recommended empirical formula is:

[\text{Worker Count} = (2 \times \text{CPU Core Count}) + 1]

For example:

  • If your server VM has 2 vCPU cores: [(2 \times 2) + 1 = 5 \text{ workers}]

The ASGI Worker Formula (Asynchronous / FastAPI) #

For asynchronous frameworks like FastAPI based on non-blocking event loops, one worker process can efficiently handle thousands of concurrent connections. Therefore, you don’t need too many worker processes. The optimal count is:

[\text{Worker Count (ASGI)} = \text{CPU Core Count}]

Python Deployment Updates (Graceful Reload) #

Every time you update your Python application code on the server, crudely stopping the service using systemctl restart cuts off active user connections and triggers errors in their browsers.

Gunicorn supports the Graceful Reload feature using the Unix system SIGHUP signal. When Gunicorn receives a reload command, the master process walks through the worker child processes one by one, stops the old worker after its active connections finish, and creates new workers carrying the updated application code in the background without closing the main connection socket.

Here’s a safe deployment automation bash script you can install on your production server:

#!/bin/bash
# deploy-python.sh — Python zero-downtime deployment script

APP_DIR="/var/www/myapp"
SERVICE_NAME="myapp"

echo "=== Starting the Python Deployment Process ==="

# 1. Enter the project directory and pull the latest Git updates
cd "$APP_DIR" || exit 1
git pull origin main

# 2. Update the dependencies inside the virtual environment
echo "Updating library dependencies..."
./venv/bin/pip install -r requirements.txt --quiet

# 3. Run Django database schema migrations
echo "Running database migrations..."
./venv/bin/python manage.py migrate --noinput

# 4. Recompile Django static assets
echo "Compiling static files..."
./venv/bin/python manage.py collectstatic --noinput --quiet

# 5. Trigger systemd to do a graceful reload (SIGHUP) to Gunicorn
echo "Triggering the Gunicorn Graceful Reload..."
sudo systemctl reload "$SERVICE_NAME"

# 6. Give a 3-second pause so the initialization process finishes
sleep 3

# 7. Test the API health via Caddy
echo "Testing the API health status..."
HTTP_STATUS=$(curl -sf -o /dev/null -w "%{http_code}" https://example.com/health)

if [ "$HTTP_STATUS" != "200" ]; then
    echo "✗ Health test failed with status: $HTTP_STATUS!"
    echo "The deployment detected a runtime error. Please check gunicorn-error.log."
    exit 1
fi

echo "✓ The Python deployment process completed without downtime!"

Django REST Framework CORS Configuration in Caddy #

When you build a separate API architecture using Django REST Framework (DRF) as the backend and React/Vue as the frontend on different domains, you must handle the CORS (Cross-Origin Resource Sharing) policy.

Handling CORS directly on the Caddy edge server side frees you from installing additional Python libraries like django-cors-headers in your Python codebase:

# Caddy configuration for the Django REST Framework API
api.example.com {
    log {
        output file /var/log/caddy/django-api-access.log
        format json
    }
    
    encode zstd gzip
    
    # 1. Handle OPTIONS preflight requests from frontend browsers
    @options method OPTIONS
    handle @options {
        header Access-Control-Allow-Origin      "https://app.example.com"
        header Access-Control-Allow-Methods     "GET, POST, PUT, DELETE, PATCH, OPTIONS"
        header Access-Control-Allow-Headers     "Content-Type, Authorization, X-CSRFToken"
        header Access-Control-Allow-Credentials "true"
        header Access-Control-Max-Age           "86400"
        respond "" 204
    }
    
    # 2. Insert CORS headers for dynamic API transactions
    header Access-Control-Allow-Origin      "https://app.example.com"
    header Access-Control-Allow-Credentials "true"
    
    # 3. Stream requests to the Django Gunicorn socket
    reverse_proxy unix//run/django-api/gunicorn.sock {
        header_up X-Real-IP {remote_host}
        header_up X-Forwarded-Proto {scheme}
    }
}

Python Request Processing Flow Diagram by Caddy #

To visualize how a request is routed by Caddy through the UNIX socket gate to your Python application, look at the following flowchart:

flowchart TD
    A["Client Browser Request Arrives\n(e.g., GET /api/users)"] --> B["1. Caddy decrypts TLS\nand applies security headers"]
    
    B --> C{"2. Is the route intended\nfor static files?"}
    
    C -- "Yes" --> D["3. Caddy directly serves the file from the folder\n/var/www/django-app/staticfiles"]
    D --> E["Done"]
    
    C -- "No" --> F["4. Route the request to reverse_proxy\ntarget unix//run/myapp/gunicorn.sock"]
    
    F --> G["5. Gunicorn Master Process\n(Receives the request at the socket buffer)"]
    G --> H{"6. Is the framework type\nWSGI or ASGI?"}
    
    H -- "WSGI (Django/Flask)" --> I["7. Route the request to synchronous workers\n(Standard Python thread execution process)"]
    H -- "ASGI (FastAPI)" --> J["8. Route the request to the UvicornWorker\n(Asynchronous event-loop execution process)"]
    
    I --> K["9. The application processes the request\n(PostgreSQL database access)"]
    J --> K
    
    K --> L["10. Return the response binary output to Gunicorn"]
    L --> M["11. Gunicorn streams the data back to Caddy"]
    M --> N["12. Caddy compresses the response (Gzip/Zstd)\nand sends it to the Client browser"]
    N --> E

Summary #

  • UNIX Socket Communication: Always use UNIX socket connections (unix//run/myapp/gunicorn.sock) for Gunicorn-Caddy integration for faster, safer local memory performance compared to TCP sockets.
  • ASGI Worker Class: Configure Gunicorn with the Uvicorn worker class (--worker-class uvicorn.workers.UvicornWorker) to optimally serve asynchronous FastAPI applications.
  • Django SSL Configuration: Don’t forget to add the SECURE_PROXY_SSL_HEADER = ('HTTP_X_FORWARDED_PROTO', 'https') parameter in Django’s settings.py file so HTTPS links are detected correctly.
  • Static Asset Isolation: Configure Caddy to read the /static/ folder directly from local disk storage to avoid wasting Gunicorn resources on static files.
  • Worker Capacity Tuning: Adjust the WSGI worker count using the (2 × CPU Cores) + 1 formula to optimize server CPU utilization without wasting RAM memory.
  • Graceful Reloading: Use the UNIX HUP signal (systemctl reload myapp) on Gunicorn deployments to apply application code updates without dead pauses (zero-downtime).
  • API CORS Centralization: Leverage the Caddyfile to insert centralized CORS header configuration to secure your Django REST Framework API backend.

← Previous: PHP-FPM   Next: WebSocket →

About | Author | Content Scope | Editorial Policy | Privacy Policy | Disclaimer | Contact