WebSocket #
WebSocket is a high-performance full-duplex two-way communication protocol running over a single TCP connection. Unlike the conventional HTTP protocol, which is stateless and based on a request-response interaction model, WebSocket allows servers to proactively send data (push notifications) to client browsers in real time without waiting for a request first. This communication relationship is initiated using an HTTP handshake upgrade mechanism before the connection is finally taken over entirely for low-level binary or text data transmission. The Caddy web server offers outstanding built-in support for this protocol. By default, Caddy handles the WebSocket handshake and data streaming process automatically without requiring any additional configuration. We’ll thoroughly examine how the WebSocket handshake works, program a Node.js WebSocket server, configure Socket.io, apply path-based routing, compose authentication during the handshake process, do load balancing with sticky sessions, configure system timeout limits, and tune OS resource capacity limits.
How the WebSocket Upgrade Works #
The WebSocket connection initialization process doesn’t directly use raw sockets, but leverages the existing HTTP port infrastructure (ports 80/443) to guarantee compatibility through network firewalls.
This transition process is divided into four main steps:
Client Handshake Request: The client browser sends a regular HTTP GET request to the web server, but includes special marker headers requesting a protocol upgrade:
GET /chat HTTP/1.1 Host: example.com Upgrade: websocket Connection: Upgrade Sec-WebSocket-Key: dGhlIHNhbXBsZSBub25jZQ== Sec-WebSocket-Version: 13Forwarding by Caddy: Caddy detects the
Upgrade: websocketheader, then forwards it intact to your backend application server.Upgrade Approval by the Backend: The backend application validates the security key token, then returns an approval response with the special HTTP status 101 Switching Protocols:
HTTP/1.1 101 Switching Protocols Upgrade: websocket Connection: Upgrade Sec-WebSocket-Accept: s3pPLMBiTxaQ9kYGzzhZRbK+xOo=Socket Hijacking: Immediately after the 101 status is sent, the HTTP connection ends. Both Caddy and the backend hijack that TCP socket to keep it permanently open. Subsequent data flows using very lightweight WebSocket binary frames without HTTP header overhead load.
Basic Configuration #
Because Caddy is designed with a modern architecture natively understanding today’s web protocols, you don’t need to add any custom parameters to the reverse_proxy configuration in the Caddyfile:
# Basic Caddy WebSocket configuration (Works automatically)
example.com {
# Caddy automatically detects the Upgrade header
# and changes the routing mode into a WebSocket binary tunnel
reverse_proxy localhost:3000
}
Node.js WebSocket Server (ws library) #
To see this interaction in real life, let’s create a simple WebSocket server in Node.js using the ws library running internally on port 3000:
// server.js (Node.js WebSocket backend)
const http = require('http');
const WebSocket = require('ws');
// 1. Create a basic HTTP server to handle Caddy health checks
const server = http.createServer((req, res) => {
if (req.url === '/health') {
res.writeHead(200, { 'Content-Type': 'application/json' });
res.end(JSON.stringify({ status: 'UP', activeConnections: wss.clients.size }));
} else {
res.writeHead(404);
res.end();
}
});
// 2. Create a WebSocket server bound to the /ws path
const wss = new WebSocket.Server({ server, path: '/ws' });
wss.on('connection', (ws, req) => {
// Read the X-Real-IP header forwarded by Caddy
const clientIp = req.headers['x-real-ip'] || req.socket.remoteAddress;
console.log(`[WebSocket] New connection established from IP: ${clientIp}`);
ws.on('message', (message) => {
console.log(`[WebSocket] Received message: ${message}`);
// Broadcast the message back to all connected clients
wss.clients.forEach((client) => {
if (client.readyState === WebSocket.OPEN) {
client.send(`Broadcast: ${message}`);
}
});
});
ws.on('close', () => {
console.log('[WebSocket] Connection closed by the client');
});
// Send a welcome message when the connection succeeds
ws.send(JSON.stringify({ type: 'system', data: 'Successfully connected to the server!' }));
});
// Bind the server to localhost port 3000
server.listen(3000, '127.0.0.1', () => {
console.log('WebSocket server running internally at http://127.0.0.1:3000');
});
Socket.io with Caddy #
The Socket.io library uses HTTP long-polling as its initial handshake method before upgrading to a real WebSocket. You must ensure all Socket.io URL segment routes (/socket.io/*) are directed to the same backend intact:
# Caddyfile configuration for Socket.io
example.com {
encode zstd gzip
# Special routing for Socket.io handshake and WebSocket traffic
handle /socket.io/* {
reverse_proxy localhost:3000 {
header_up X-Real-IP {remote_host}
header_up X-Forwarded-Proto {scheme}
}
}
# Static file routing for our web chat frontend
handle {
root * /var/www/chat-app/public
file_server
}
}
On the Node.js application server side, you must also configure a suitable heartbeat timeout so Caddy doesn’t consider the connection dropped while clients are idle:
// Socket.io initialization with ping interval adjustments
const io = require('socket.io')(server, {
transports: ['websocket', 'polling'],
pingTimeout: 60000, // Give a 60-second timeout before considering a disconnect
pingInterval: 25000 // Send a ping heartbeat signal every 25 seconds
});
WebSocket with Path-Based Routing #
You can leverage Caddy’s modular architecture to split WebSocket connection routes to different backend microservices based on URL paths. For example, you separate chat handling, system notifications, and live feed data:
# Modular WebSocket routing configuration
example.com {
# 1. Chat routes go to port 3001
handle /ws/chat {
reverse_proxy localhost:3001
}
# 2. Notification routes go to port 3002
handle /ws/notifications {
reverse_proxy localhost:3002
}
# 3. Live Feed routes go to port 3003
handle /ws/feed {
reverse_proxy localhost:3003
}
# 4. Regular web traffic goes to the main server
handle {
root * /var/www/html
file_server
}
}
WebSocket Authentication #
The standard WebSocket protocol in browsers doesn’t allow sending additional custom HTTP headers (like the Authorization: Bearer *** header when triggering the new WebSocket() initialization command.
Therefore, you must perform the authentication process using one of two alternative methods during the initial handshake phase:
1. The Query Parameter Method (The Most Popular Option) #
The browser sends the token as a query parameter in the URL (e.g., wss://example.com/ws?token=JWT_TOKEN). The backend server reads this parameter when the connection arrives:
// Authentication handling on the backend Node.js side
wss.on('connection', (ws, req) => {
const url = new URL(req.url, 'http://localhost');
const token = url.searchParams.get('token');
if (!validateJWT(token)) {
// Close the connection with the Close status code: Policy Violation (1008)
ws.close(1008, 'Invalid or expired token!');
return;
}
// Successfully authenticated
ws.username = getUsernameFromToken(token);
});
2. The Centralized Authentication Method on the Caddy Gateway Side #
If you want Caddy to filter security before the request is forwarded to the backend, you can apply a basic authentication directive at the Caddyfile level:
# Basic authentication before the WebSocket handshake upgrade
example.com {
@ws_route path /ws/*
handle @ws_route {
# Only allow browsers including valid basic authentication
basicauth {
admin $2a$14$our-admin-bcrypt-hash-key
}
reverse_proxy localhost:3000
}
handle {
reverse_proxy localhost:3000
}
}
WebSocket Load Balancing with Sticky Sessions #
WebSocket connections are stateful (storing active connection status data in server RAM). Once a handshake succeeds on Server A, the client browser must keep communicating with the same Server A until the connection closes. If data packets are routed to Server B mid-way, Server B gets confused because it doesn’t have that client’s session state memory data.
If you deploy several WebSocket backend server nodes behind a Caddy load balancer, you must configure the Sticky Sessions policy (session persistence). The easiest way in Caddy is using the IP Hash or Cookie policy:
# WebSocket cluster load balancing with Sticky Sessions
example.com {
reverse_proxy {
# Define all our production WebSocket server nodes
to ws-node-1:3000 ws-node-2:3000 ws-node-3:3000
# IP Hash guarantees browsers with the same IP address
# are always routed to the same backend node server
lb_policy ip_hash
# Instance health testing
health_uri /health
health_interval 10s
header_up X-Real-IP {remote_host}
header_up X-Forwarded-Proto {scheme}
}
}
[!TIP] Use Redis Pub/Sub for Unlimited Scalability. The Sticky Sessions approach has limitations if one backend server crashes or is restarted. Users bound to that server get disconnected and shifted to another empty node, triggering the loss of unsaved chat history. The best solution at large production scale is making your WebSocket application stateless using Redis Pub/Sub as a centralized Shared State Message Broker. That way, Caddy is free to route users to any node randomly (round-robin) without fearing chat data synchronization loss.
Timeout Configuration for WebSocket #
The Caddy web server by default has a data write timeout limit configuration for dropping slow, inactive connections to save resources. You must be careful when composing global timeout configurations so you don’t cut off active WebSocket connections that can last for days:
# Safe global timeout configuration for WebSocket
{
servers {
timeouts {
# Limit the initial request body reading time
read_body 10s
# Limit the initial request header reading time
read_header 10s
# IMPORTANT: Don't set the write timeout (leave it 0 / default).
# Setting a static write timeout (e.g., 60s) forcibly
# drops all active WebSocket connections every 60 seconds.
write 0
}
}
}
example.com {
reverse_proxy localhost:3000 {
transport http {
dial_timeout 5s
# Avoid setting response_header_timeout for WebSocket
}
}
}
Tuning Linux OS Resource Limits (ulimit) #
WebSocket connections hold TCP connection sockets permanently open on your Caddy server. In the Linux OS, every active network connection is represented as a file (File Descriptor).
By default, Linux limits the maximum open files per process to just 1024 files (soft limit). If your server serves more than 1000 concurrent WebSocket users, Caddy fails to serve new users and triggers the too many open files error log.
You must raise the File Descriptor limit on your production OS:
1. Modifying the Caddy Systemd Configuration #
Edit the Caddy systemd service override file:
sudo systemctl edit caddy
Add the following lines inside the editor configuration file that appears:
[Service]
# Raise the maximum file descriptor limit for the Caddy process (Soft & Hard Limit)
LimitNOFILE=65536
Save the file, then reload the systemd daemon:
sudo systemctl daemon-reload
sudo systemctl restart caddy
2. Verifying Caddy’s Active Limits #
You can check whether the Caddy process resource limit has been successfully raised using the command:
# Find the Caddy process PID
PID_CADDY=$(pgrep caddy)
# Read that process's limits file
cat /proc/$PID_CADDY/limits | grep "Max open files"
# The output should show: Max open files 65536 65536
Client-Side WebSocket Heartbeat and Reconnect #
To ensure the connection stays alive through user router firewalls that often unilaterally cut idle connections, you must implement ping/pong heartbeat and auto-reconnect in your JavaScript browser client code:
// Browser Client JavaScript Utility
class SecureWebSocketClient {
constructor(url) {
this.url = url;
this.ws = null;
this.reconnectInterval = 1000; // Start with a 1-second reconnect delay
this.heartbeatTimer = null;
this.connect();
}
connect() {
console.log(`[WS Client] Trying to connect to ${this.url}...`);
this.ws = new WebSocket(this.url);
this.ws.onopen = () => {
console.log('[WS Client] Successfully connected to WebSocket!');
this.reconnectInterval = 1000; // Reset the delay back to 1 second
this.startHeartbeat();
};
this.ws.onmessage = (event) => {
const data = JSON.parse(event.data);
console.log('[WS Client] Received data:', data);
};
this.ws.onclose = () => {
console.log(`[WS Client] Connection lost. Trying to reconnect in ${this.reconnectInterval}ms...`);
this.stopHeartbeat();
// Auto-reconnect using exponential backoff timing
setTimeout(() => this.connect(), this.reconnectInterval);
this.reconnectInterval = Math.min(this.reconnectInterval * 2, 30000); // Maximum 30-second delay
};
this.ws.onerror = (error) => {
console.error('[WS Client] An error occurred:', error);
};
}
startHeartbeat() {
// Send a ping signal every 25 seconds to the Caddy server
this.heartbeatTimer = setInterval(() => {
if (this.ws.readyState === WebSocket.OPEN) {
this.ws.send(JSON.stringify({ type: 'ping', data: 'heartbeat' }));
}
}, 25000);
}
stopHeartbeat() {
clearInterval(this.heartbeatTimer);
}
}
// Initialize the client connection using the secure wss protocol (WebSocket Secure)
const appSocket = new SecureWebSocketClient('wss://example.com/ws?token=OUR_JWT_TOKEN');
WebSocket Protocol Upgrade Handshake Flow Diagram #
To visualize how the transition process flows from a regular HTTP connection to a two-way WebSocket binary tunnel path in Caddy, look at the following sequence diagram:
sequenceDiagram
autonumber
participant Client as Client Browser (JS)
participant Caddy as Caddy Server (Edge)
participant Backend as Node.js WebSocket App
Client->>Caddy: HTTP GET /ws (Upgrade Request + Sec-WebSocket-Key)
Note over Caddy: Detect the Upgrade: websocket header
Caddy->>Backend: Forward HTTP GET /ws (Upgrade Request)
Note over Backend: Validate the Auth Token & Generate the Accept Key
Backend-->>Caddy: HTTP 101 Switching Protocols (Upgrade Confirmed)
Caddy-->>Client: HTTP 101 Switching Protocols
Note over Caddy: Caddy & Backend hijack the TCP socket in RAM
Note over Client: The connection changes into a WebSocket Tunnel (Stateful)
par Two-Way Transmission (Full-Duplex)
Client->>Caddy: Send Data Frames (Binary / Text)
Caddy->>Backend: Forward Data Frames
and
Backend->>Caddy: Send Data Frames (Proactively)
Caddy->>Client: Forward Data Frames (Real-time Push)
endSummary #
- Native Support: Caddy automatically detects and transparently forwards WebSocket handshake traffic without requiring additional flags or options in the Caddyfile.
- Validation Status: The HTTP 101 Switching Protocols response status in Caddy’s access log confirms the WebSocket upgrade handshake was successfully performed.
- Sticky Sessions: Use the
lb_policy ip_hashdistribution policy when doing multi-server WebSocket load balancing if the application doesn’t use a shared Redis database.- Timeout Tuning: Leave the global
write_timeoutparameter at its default0value (unlimited) so active WebSocket connections aren’t forcibly dropped unilaterally.- Secure Authentication: Apply authentication token validation via query parameters or cookies during initial handshake connection initialization.
- OS File Descriptors: Raise the OS maximum open file capacity limit (
LimitNOFILE=65536) on the Caddy systemd service to serve thousands of concurrent connections.- Client Auto-Reconnect: Implement ping-pong heartbeat and exponential reconnection mechanisms on the JavaScript browser side to face network instability disruptions.