Weighted #

In managing large-scale computing infrastructure, it’s rare to find a cluster where every backend server has identical hardware specs. Some servers may run on high-spec physical machines (bare-metal), while others run on medium VMs, or small containers in the cloud. To maximize this hardware efficiency, Caddy provides the Weighted Load Balancing feature (weight-based load balancing). We’ll learn about the weight distribution concept, the load ratio division formula, combining it with the Least Connections algorithm, and its implementation in Canary Release and Blue-Green Deployment strategies.

The Weighted Load Balancing Concept #

The Weighted Load Balancing policy lets you, as a network administrator, assign a numeric weight value to each backend (upstream) server. This weight acts as the determinant of the portion or percentage of traffic that should be routed to that server.

The higher a server’s weight value, the more requests Caddy routes to it. Mathematically, the probability or traffic ratio received by server $i$ is calculated with the formula:

$$\text{Traffic Ratio}_i = \frac{\text{Weight}i}{\sum{j=1}^{n} \text{Weight}_j}$$

For example, we have 3 servers with the following weight configuration:

• Server A (Weight: 5) (Enterprise VM)
• Server B (Weight: 3) (Standard VM)
• Server C (Weight: 2) (Micro VM)

The total weight sum is $5 + 3 + 2 = 10$. The traffic ratios Caddy will route are:

• Server A receives: $\frac{5}{10} = 50%$ of total requests.
• Server B receives: $\frac{3}{10} = 30%$ of total requests.
• Server C receives: $\frac{2}{10} = 20%$ of total requests.

Here’s a visualization of the load distribution flow based on those weight ratios:

flowchart TD
    Reqs["10 Incoming Requests"] --> Caddy{"Caddy Proxy"}
    
    Caddy -->|"Weight: 5 (50% of routes)"| S_A["Server A (Enterprise VM)"]
    Caddy -->|"Weight: 3 (30% of routes)"| S_B["Server B (Standard VM)"]
    Caddy -->|"Weight: 2 (20% of routes)"| S_C["Server C (Micro VM)"]

    style S_A stroke:#43a047,stroke-width:2px
    style S_B stroke:#0288d1,stroke-width:2px
    style S_C stroke:#e53935,stroke-width:2px

This weight division effectively protects the small server (Server C) from overload danger while still leveraging the full processing power of the big server (Server A).

The Smooth Weighted Round Robin (SWRR) Algorithm #

In a traditional Weighted Round Robin (plain WRR) implementation, if you have Server A with weight 5, Server B with weight 1, and Server C with weight 1, the scheduler sends 5 consecutive requests to Server A, then 1 to Server B, and 1 to Server C. This pattern triggers a Bursty Load problem, where Server A gets instantly bombarded by consecutive request surges that can spike response latency immediately.

To solve this, Caddy internally adopts the Smooth Weighted Round Robin (SWRR) algorithm (the same algorithm Nginx implements). SWRR dynamically distributes requests in an alternating fashion so the load is split smoothly.

The algorithm works by tracking three variables for each upstream:

1. Weight: The static weight you configure in the Caddyfile.
2. EffectiveWeight: The actual weight that can be dynamically lowered if Caddy detects the backend starting to respond slowly.
3. CurrentWeight: The dynamic weight that changes every time a new request arrives.

Each time a new request comes in, the SWRR circuit runs the following steps:

• For every upstream: CurrentWeight = CurrentWeight + EffectiveWeight.
• Select the upstream with the highest CurrentWeight as the target.
• Subtract the cluster’s total EffectiveWeight from the selected CurrentWeight: $$\text{CurrentWeight}{\text{selected}} = \text{CurrentWeight}{\text{selected}} - \sum \text{EffectiveWeight}$$

Mathematical Trace of the SWRR Algorithm #

Let’s trace the SWRR math for 7 consecutive requests on a 3-server cluster with weights A:5, B:1, C:1 (Total weight = 7):

• Distribution Result: A, A, B, A, C, A, A
• Analysis: After request #7 finishes, the CurrentWeight values cleanly return to the initial 0, 0, 0 state. We can see that the 5 requests to Server A aren’t sent consecutively at the start, but are smoothly interspersed with Server B on request #3 and Server C on request #5.

Weighted Configuration in Caddy #

In the Caddyfile, you can apply weights to upstreams by writing the weight option individually for each dial target behind the reverse_proxy directive.

There are two syntax writing styles supported by Caddy:

1. Structured Syntax (Highly Recommended) #

Write each backend using the to directive and insert the weight attribute inside:

# Structured writing example with weights
app.example.com {
    reverse_proxy {
        # Enterprise VM
        to backend-1:3000 weight 5
        
        # Standard VM
        to backend-2:3000 weight 3
        
        # Micro VM
        to backend-3:3000 weight 2
        
        lb_policy round_robin
    }
}

2. Inline Syntax (Shortcut) #

Write the weight directly using query-style URL notation on the dial address (commonly used for concise single-line configurations):

# Inline shortcut writing example
app.example.com {
    reverse_proxy backend-1:3000?weight=5 backend-2:3000?weight=3 backend-3:3000?weight=2
}

Integration with Least Connections (Weighted Least Connections) #

When you combine the weight parameter with the Least Connections (least_conn) algorithm, Caddy no longer just looks for the server with the absolutely smallest connection count. Caddy evaluates the load ratio by dividing the active connection count by each server’s weight value.

The target server with the smallest ratio value is selected to receive the next request:

$$\text{Load Ratio} = \frac{\text{Active Connections}}{\text{Weight}}$$

Load Ratio Calculation Simulation #

Imagine we have 2 active servers:

• Server A (Weight: 2): Currently processing 10 active connections.
• Server B (Weight: 5): Currently processing 15 active connections.

A new client sends a request to Caddy. Let’s compare where the request will be routed:

1. Server A’s Load Ratio: $\frac{10}{2} = 5.0$
2. Server B’s Load Ratio: $\frac{15}{5} = 3.0$

• Caddy’s Decision: The new request will be routed to Server B, even though in absolute numbers Server B is processing more connections (15 connections) than Server A (10 connections). This is logical because Server B has 2.5 times stronger hardware (weight 5 vs 2), so its actual spare capacity is still much larger than Server A’s.

# Weighted Least Connections configuration
app.example.com {
    reverse_proxy {
        to backend-1:3000 weight 2
        to backend-2:3000 weight 5
        
        lb_policy least_conn
    }
}

Adjusting Weight Values During Heavy Traffic #

In industry practice, static weight values are often less than optimal when backend workload types shift dynamically. For example, Server A (strong VM with weight 10) handles complex database queries, while Server B (small VM with weight 2) serves static JSON responses. During peak traffic, Server A’s database experiences query contention, causing CPU utilization to spike drastically even though its active connection count is still below the weight division ratio.

To overcome this, the DevOps team must design dynamic weight adjustments based on the per-core CPU load factor. You can leverage an external script that reads backend CPU utilization in real time and updates Caddy’s weights gracefully through the JSON REST API.

# Example cluster with dynamic weight tuning options
prod.example.com {
    reverse_proxy {
        # Database VM (Weight adjusted dynamically by the monitor daemon)
        to db-app-1:3000 weight 8
        
        # Cache / Static API VM
        to cache-app-2:3000 weight 2
        
        lb_policy least_conn
    }
}

Use Case 1: Heterogeneous Clusters #

In the real world, cloud budget savings often force you to mix various VM types. For example, you place your company’s local physical servers (On-Premise) as the main backends, plus a few small cloud VMs (AWS EC2) as auxiliary backends handling traffic overflow.

# Heterogeneous cluster configuration
prod.example.com {
    reverse_proxy {
        # On-Premise Physical Server (Very strong)
        to 192.168.10.50:8080 weight 10
        
        # AWS EC2 Instance 1 (Medium VM)
        to 10.0.1.15:8080 weight 3
        
        # AWS EC2 Instance 2 (Medium VM)
        to 10.0.1.16:8080 weight 3
        
        lb_policy least_conn
    }
}

With this configuration, 62.5% of traffic ($\frac{10}{16}$) is processed on your company’s local servers for free, while the rest is split evenly to the paid cloud VMs only when the load starts rising.

Use Case 2: Canary Deployments (Gradual Feature Releases) #

Canary Deployment is a new application feature release strategy where you release the new version (v2) code to only a small subset of users first to test its stability in production, while the majority of users stay routed to the stable version (v1) code.

flowchart TD
    User["User Traffic"] --> Caddy{"Caddy Proxy\n(Canary Split)"}
    
    Caddy -->|"Weight: 95 (95% of routes)"| V1["Stable Cluster (App v1)"]
    Caddy -->|"Weight: 5 (5% of routes)"| V2["Canary Cluster (App v2)"]

    style V1 stroke:#0288d1,stroke-width:2px
    style V2 stroke:#e53935,stroke-dasharray: 5,5

Here’s an example Canary Release implementation using weight division in the Caddyfile:

# Canary Deployment 95% vs 5%
app.example.com {
    reverse_proxy {
        # Production Server v1 (Stable)
        to app-v1-node1:3000 weight 95
        
        # New Server v2 (Canary)
        to app-v2-canary:3000 weight 5
        
        lb_policy round_robin
    }
}

Canary Transition Automation Script Using the Caddy REST API #

To automate gradual new feature releases without manual intervention, you can write a bash script that gradually raises the Canary server’s route weight. This script dynamically modifies the JSON array at the Caddy API path /config/apps/http/servers/srv0/routes/0/handle/0/routes/0/handle/0/upstreams:

#!/bin/bash
# canary_rollout.sh

CADDY_API_UPSTREAMS="http://localhost:2019/config/apps/http/servers/srv0/routes/0/handle/0/routes/0/handle/0/upstreams"

# Array of Canary percentage stages (weights of v1 vs v2)
# Stage 1: 95 vs 5, Stage 2: 80 vs 20, Stage 3: 50 vs 50, Stage 4: 0 vs 100
stages=("95:5" "80:20" "50:50" "0:100")

check_health() {
    # Check the 5xx error rate in the last minute of Caddy logs
    error_count=$(tail -n 100 /var/log/caddy/access.log | grep -c '"status":5')
    if [ "$error_count" -gt 5 ]; then
        echo "WARNING: Detected $error_count errors. Canceling the release (Rollback)!"
        # Instant rollback to 100% v1
        curl -X PUT "$CADDY_API_UPSTREAMS" \
          -H "Content-Type: application/json" \
          -d '[{"dial":"app-v1-node1:3000","weight":100},{"dial":"app-v2-canary:3000","weight":0}]'
        exit 1
    fi
}

for stage in "${stages[@]}"; do
    w1=$(echo $stage | cut -d':' -f1)
    w2=$(echo $stage | cut -d':' -f2)
    
    echo "Starting Canary Stage: v1=$w1%, v2=$w2%"
    
    # Update the weights dynamically in Caddy's memory
    curl -s -X PUT "$CADDY_API_UPSTREAMS" \
      -H "Content-Type: application/json" \
      -d '[{"dial":"app-v1-node1:3000","weight":'$w1'},{"dial":"app-v2-canary:3000","weight":'$w2'}]' \
      -o /dev/null
      
    # Wait 5 minutes for stability observation
    sleep 300
    check_health
done

echo "App v2 Canary Release Completed Successfully!"

Use Case 3: Blue-Green Deployments (Zero-Downtime Migration) #

Blue-Green Deployment is an application release strategy where you maintain two identical physical environments simultaneously: the Blue environment (the current version actively serving users) and the Green environment (the new version where you deploy and test code without disturbing users).

After testing in the Green environment completes successfully, you do an instant switch (cutover) of the main Caddy gateway route.

# Step 1: 100% traffic on Blue
# to blue-server:3000 weight 100
# to green-server:3000 weight 0   // Disabled

# Step 2: 50-50 transition (Gradual load split)
app.example.com {
    reverse_proxy {
        to blue-server:3000 weight 50
        to green-server:3000 weight 50
        
        lb_policy round_robin
    }
}

# Step 3: 100% Green active, Blue is shut down
# to blue-server:3000 weight 0
# to green-server:3000 weight 100

Database Schema Synchronization (Expand and Contract Pattern) #

The biggest challenge in doing zero-downtime Blue-Green migration is keeping the database compatible while both application versions (v1 and v2) access the same database simultaneously during the transition.

You must apply the Expand and Contract (Parallel Run) pattern for database schema migration:

1. Expand: Add new columns to the database without removing old columns. Change the v2 application code to write to both columns (old and new), while the v1 application keeps writing to the old column.
2. Synchronization: Run a background script to copy historical data from the old column to the new column.
3. Contract: After Caddy’s routes fully switch to the Green environment (100% v2), you deploy a patch to stop writing to the old column. Finally, you safely drop the old column from the database.

Summary #

• Key Definition: The Weighted Load Balancing policy distributes traffic based on numeric weight values attached to each backend server.
• SWRR Algorithm: Caddy uses the Smooth Weighted Round Robin algorithm to distribute requests in an alternating fashion, avoiding bursty loads.
• Weighted Least Connections: Caddy divides the active connection count by the weight to find the smallest load ratio, optimizing heterogeneous hardware clusters.
• Canary Deployment: Very effective for safely releasing new application features by routing a small traffic percentage (e.g., 5%) to new server instances.
• Blue-Green Transition: Facilitates zero-downtime migration by gradually shifting traffic loads from the old cluster to the new cluster using the expand and contract database schema pattern.
• Caddyfile Syntax: Supports structured writing using the weight subdirective on to blocks, or instant inline writing using URL query parameters.

← Previous: IP Hash   Next: Active Health Check →