Log Output #

Output is the final destination of the entire log data stream produced by the Caddy server after passing through the processing and formatting stages by the encoder module. In the log recording lifecycle, correct output configuration plays a crucial role in maintaining system stability and operational efficiency. Mistakes in determining the output target or negligence in setting storage policies can be fatal, from a full server disk triggering a whole-system crash, to losing important audit data during a security incident. Caddy offers very flexible built-in support for various output target types: from writing to local files with automatic log rotation features, standard output streams (stdout and stderr) ideal for containerization, the discard target for throwing away unimportant log traffic, to direct network streaming (network sockets) to centralized log collection servers. We’ll learn the characteristics of each output type, how to configure aggressive log rotation, logging strategies in container and Kubernetes environments, buffering handling for high performance, and modular log file separation techniques.

Available Output Types in Caddy #

Caddy divides log output targets into five main output modules. Understanding the performance characteristics and purposes of these five types is essential so you can choose the logging strategy best suited to your server infrastructure architecture.

Here’s a brief summary of the output types natively supported by Caddy:

• file — Writes log lines into physical files on local storage media. Very suitable for traditional deployments on Virtual Machines (VM) or Bare Metal because it comes with automatic rotation, size limits, compression, and file retention management systems.
• stdout — Streams log data to Standard Output (File Descriptor 1). This is the golden standard for container applications because container runtimes like Docker and Kubernetes capture this stream to hand over to node-level logging agents.
• stderr — Streams log data to Standard Error (File Descriptor 2). This is Caddy’s global default output if you don’t explicitly define an output directive.
• discard — Throws away all incoming log entries without processing or writing them to any medium. Very useful for ignoring logging on high-traffic endpoints with no audit value.
• net — Sends logs in real time over network connections (TCP or UDP sockets) to remote log collection servers like Syslog daemons, Logstash, Fluentd, or Grafana Loki.

For a clearer comparison picture, let’s look at the table below comparing each output type’s suitability based on operational metrics:

Output to Local Files (File Logging) #

Writing logs into local files on disk is the most common approach in traditional server architectures. Here’s a basic example of writing the log directive with a file target in the Caddyfile:

# Basic local file logging configuration example
example.com {
    log {
        output file /var/log/caddy/access.log
        format json
    }
    reverse_proxy localhost:8080
}

The Importance of Automatic File Rotation (Log Rotation) #

Continuously writing logs to a single unbounded file is one of the main causes of production server operational failures. Over time, access log files keep growing to tens or hundreds of gigabytes, consuming all hard disk space capacity, and eventually causing the OS kernel to forcibly stop the Caddy server process (out-of-space crash).

Caddy elegantly solves this problem by providing an automatic file rotation module (log rotation) integrated directly inside the output file module. You don’t need third-party utilities like the Linux built-in logrotate.

Here’s an example of complete, safe log file rotation management configuration in the Caddyfile:

# Complete configuration example with log rotation management
example.com {
    log {
        output file /var/log/caddy/access.log {
            # 1. Rotate the file if its size reaches 100 Megabytes.
            # Caddy renames the active file to a timestamped name,
            # and automatically creates a new empty access.log file.
            roll_size 100mb
            
            # 2. Limit the number of stored log archive files to a maximum of 7 files.
            # The 8th oldest archive file is deleted automatically.
            roll_keep 7
            
            # 3. Delete log archive files older than 30 days.
            # Helps data retention compliance without burdening the disk.
            roll_keep_days 30
            
            # 4. By default, Caddy compresses rotated log files into .gz files.
            # If you want to disable gzip compression for CPU speed,
            # you can enable the option below (not recommended).
            # roll_uncompressed
        }
        format json
    }
    reverse_proxy localhost:8080
}

How Caddy’s Rotation File Naming Works #

When the roll_size criteria is met, Caddy closes the active log file /var/log/caddy/access.log, then renames it using a local ISO-8601 timestamp numbering format like the following:

/var/log/caddy/access-2026-06-16T18-44-15.000.log.gz

Because files are archived in compressed .gz format by default, archive file sizes usually shrink by 80-90% from their original size. This significantly saves your disk storage capacity.

Writing Logs to stdout and stderr (Containerization) #

In the modern cloud and container-based infrastructure era, the practice of writing logs to local files inside containers is considered an anti-pattern. Containers are designed to be disposable (ephemeral), meaning data inside the container filesystem is lost when the container is restarted or rebuilt.

Based on The Twelve-Factor App methodology for building modern applications, logs must be treated as event streams. Applications shouldn’t be burdened with the responsibility of managing their own log files. Instead, applications should write all their logs raw to stdout (standard output) or stderr (standard error).

Caddy strongly follows this philosophy. You can direct Caddy logs directly to the container terminal stdout:

# Cloud-native logging configuration example for Docker / Kubernetes
example.com {
    log {
        output stdout
        format json
    }
    reverse_proxy localhost:8080
}

When you run Caddy inside Docker using the configuration above, the Docker runtime captures all JSON text streams from Caddy’s stdout in real time. Docker then hands that log data to the log driver configured on the Docker host (e.g., json-file, journald, or syslog).

Here’s a docker-compose.yml configuration example applying log size limits at the Docker host level:

version: '3.8'

services:
  caddy:
    image: caddy:2.8.4-alpine
    restart: always
    ports:
      - "80:80"
      - "443:443"
    volumes:
      - ./Caddyfile:/etc/caddy/Caddyfile
      - caddy_data:/data
      - caddy_config:/config
    # Set the log driver at the Docker level to prevent the host disk from filling up
    logging:
      driver: "json-file"
      options:
        max-size: "50m"
        max-file: "3"

volumes:
  caddy_data:
  caddy_config:

You can monitor the Caddy container logs in real time from the host terminal with the following commands:

# Monitor the last 100 log lines and keep following (follow)
docker logs caddy --tail 100 -f

# Monitor logs and parse the JSON neatly using the jq utility
docker logs caddy -f 2>&1 | jq .

The discard Output: Disabling Logging #

Not all HTTP requests have audit value. In large-scale production environments, your Caddy server is often bombarded by thousands of automated requests from external systems, such as:

1. Health check calls from Load Balancers (e.g., AWS ALB or HAProxy) checking server health every few seconds.
2. Small static asset requests (like favicon files, CSS files, icon images) repeatedly accessed by client browsers.

If these health check requests keep getting recorded to the main access log file, your log file size bloats quickly with junk information. This makes searching for real human user transaction logs difficult.

Caddy provides the discard output module acting as a “blackhole” to throw away logs without processing them at all. You can combine named matchers with the log directive to filter this log traffic:

# Example of ignoring logging for LB health checks and turning off global logs
example.com {
    # Create a named matcher to detect healthcheck routes
    @healthcheck {
        path /healthz
        method GET
    }

    # Apply modular handling:
    # Health check routes are processed without being recorded in the main access log file
    handle @healthcheck {
        log {
            output discard
        }
        respond "OK" 200
    }

    # Requests to routes other than health checks are recorded to the main file
    handle {
        log {
            output file /var/log/caddy/access.log {
                roll_size 50mb
                roll_keep 3
            }
            format json
        }
        reverse_proxy localhost:8080
    }
}

With the modular tactic above, Caddy server I/O performance stays optimally maintained because log writing is only focused on important business transactions.

The net Output: Network Log Streaming (Remote Logging) #

In large-scale infrastructure consisting of dozens to hundreds of Caddy servers under autoscaling groups, managing local log files on every VM is a nightmare for operations teams. You must manually log into each VM to check logs, or create complicated cron scripts to periodically copy log files.

The best solution is sending Caddy logs in real time directly over the network to a centralized log aggregator. Caddy provides the net output module that can stream logs via TCP or UDP protocols.

1. Remote Logging Using the UDP Protocol #

UDP is a connectionless protocol. Caddy sends log packets without waiting for confirmation of whether the packet reached the log collection server.

# Example of streaming logs via UDP to a remote Syslog server
{
    log {
        # Format: output net udp/<host>:<port>
        # Very fast, no connection overhead, but there's a risk of data loss if the network is unstable
        output net udp/syslog-server.local:514
        format json
    }
}

2. Remote Logging Using the TCP Protocol #

TCP is a connection-oriented protocol guaranteeing safe, ordered packet data delivery.

# Example of streaming logs via TCP to a remote Logstash / Loki
{
    log {
        # Format: output net tcp/<host>:<port>
        # Guarantees all logs are delivered, but can trigger slight latency if the destination server is slow
        output net tcp/logstash.local:5000
        format json
    }
}

Local Linux Syslog Integration Configuration #

If you want Caddy to send logs to a local syslog daemon running on the same Linux server, you can configure the rsyslog Syslog to listen on a local UDP port:

# Add the following configuration to the /etc/rsyslog.d/caddy.conf file:
# 1. Load the UDP module and run the server on local port 514
$ModLoad imudp
$UDPServerRun 514

# 2. Capture logs from the caddy program and store them in a separate file
:programname, isequal, "caddy" /var/log/caddy/syslog-caddy.log
& stop

After rsyslog is restarted (systemctl restart rsyslog), all log streams from Caddy via output net udp/127.0.0.1:514 are safely recorded into the server’s central syslog file.

Buffering and Flush Intervals #

Writing data to physical storage media (SSD/HDD) for every generated log line is a very computationally expensive I/O operation. If the Caddy server serves thousands of requests per second, disk I/O becomes the main bottleneck limiting server response speed.

To improve write performance, Caddy internally applies memory buffering techniques. Caddy doesn’t immediately write logs to disk the moment a request finishes processing. Instead, Caddy collects several log lines first in a temporary RAM memory buffer. After the buffer size reaches a certain limit or after the flush interval time is reached, Caddy writes the entire log pile to disk at once in a single I/O operation.

However, this strategy has a trade-off:

[!WARNING] The Risk of Data Loss with Memory Buffering. Because some of the newest log data is still held in RAM before being flushed to disk, there’s a risk of losing logs if the server suddenly experiences a sudden power outage, a kernel failure (kernel panic), or the Caddy process is forcibly killed using the SIGKILL signal. Log data in the RAM buffer is lost forever and never recorded to disk.

For critical applications requiring audit compliance with no tolerance for data loss, consider streaming logs to stdout or using reliable TCP-protocol remote logging.

Logging to Multiple Files Using Modular Handling (Handle) #

In modern web architecture, you often face scenarios where you want to separate log file destinations based on application traffic route characteristics. For example:

• API traffic (/api/*) should be recorded to the special /var/log/caddy/api-access.log file with long rotation retention (e.g., 90 days) for business transaction audit needs.
• Static asset or regular web page traffic is recorded to the /var/log/caddy/web-access.log file with aggressive rotation (only 7 days retention) to save disk capacity.

The Caddyfile lets you define separate log blocks inside modular handle handling blocks to achieve this log file isolation:

# Example of separating logs to different files using modular handles
example.com {
    # Special handling for API routes
    handle /api/* {
        log {
            output file /var/log/caddy/api-access.log {
                roll_size 200mb
                roll_keep 14
                roll_keep_days 90 # Long-term audit retention
            }
            format json
        }
        reverse_proxy api-backend:8080
    }
    
    # Default handling for other routes
    handle {
        log {
            output file /var/log/caddy/web-access.log {
                roll_size 50mb
                roll_keep 3
                roll_keep_days 7 # Short-term retention
            }
            format json
        }
        file_server { root /var/www/html }
    }
}

Capacity Planning: Estimating Disk Needs for Logs #

Before deploying the Caddy server to a large-scale production environment, you must do storage capacity calculations to design the server’s hard disk specifications. You can estimate Caddy’s daily log disk space consumption with the following practical steps:

1. Calculating the Average Size per Log Line #

You can sample 100 active access log lines and calculate their average byte size using the following shell command on your Linux server:

# Calculate the average size per log entry line (in bytes)
head -100 /var/log/caddy/access.log | \
    awk '{sum += length($0)} END {print "Avg entry size:", sum/NR, "bytes"}'

Usually, one Caddy JSON access log line is about 400 to 700 bytes (depending on how much HTTP header data is recorded).

2. Daily Disk Space Requirement Estimation Formula #

Use the simple mathematical formula below to estimate daily log growth:

[\text{Daily Capacity (Bytes)} = \text{Requests Per Second (RPS)} \times 86400 \text{ seconds} \times \text{Average Entry Size (Bytes)}]

For example, a real-world scenario:

• Average server traffic: 500 requests per second (RPS).
• Average size per log entry: 500 bytes.
• Estimated raw data per day: [500 \times 86400 \times 500 = 21,600,000,000 \text{ bytes} \approx 21.6 \text{ GB per day}]
• If you enable rotation with gzip compression (assuming an 80% compression ratio), real consumption shrinks to: [21.6 \text{ GB} \times 0.20 \approx 4.32 \text{ GB per day}]

Output Strategies by Environment (Development vs Staging vs Production) #

For ready-to-use practical guidance, here’s a summary of Caddy log output configuration best practices tailored to each work environment’s characteristics:

1. Local Development Environment #

On a local computer, you prioritize ease of reading debug processes directly on the terminal console.

# Caddyfile Configuration - Local Development Environment
{
    # Enable global debug level
    debug
}

localhost:8080 {
    log {
        output stderr    # Output to the terminal stderr
        format console   # Human-friendly colored text format
        level debug      # Record up to detailed debug info
    }
    file_server { root ./public }
}

2. Staging Environment #

Staging is a production replica. You want logs to already be in JSON format and recorded to files, but with a higher verbosity level to make it easier for QA to detect integration bugs.

# Caddyfile Configuration - Staging Environment
staging.example.com {
    log {
        output file /var/log/caddy/staging-access.log {
            roll_size 50mb
            roll_keep 3
            roll_keep_days 7
        }
        format json
        level info
    }
    reverse_proxy staging-app:8000
}

3. Large-Scale Production Environment #

In production, you prioritize high performance, long-term retention for security compliance, and protection from full disks.

# Caddyfile Configuration - Production Environment
example.com {
    log {
        output file /var/log/caddy/production-access.log {
            roll_size 500mb     # Larger file size so rotation isn't too frequent at peak
            roll_keep 14        # 2-week archive retention
            roll_keep_days 90   # 3 months of historical log retention
        }
        # Use json format with sensitive data cleanup filters
        format json {
            filter request>headers>Authorization delete
            filter request>headers>Cookie delete
            filter request>client_ip ip_mask 16 32
        }
        level warn # Only record warning levels and above to save I/O (optional)
    }
    reverse_proxy production-app:8000
}

Caddy Log Entry Writing Flow Diagram #

For a visualization of the log data journey from internal processing modules to distribution across various output media, let’s study the flowchart below:

flowchart TD
    A["Formatted Log Event\n(JSON / Structured Text)"] --> B{"Configured Output Type"}
    
    B -->|"output file"| C["File Writer Module"]
    B -->|"output stdout / stderr"| D["Stream Writer Module"]
    B -->|"output net"| E["Network Writer Module"]
    B -->|"output discard"| F["Blackhole Module"]
    
    C --> C1{"Is the file size > roll_size?"}
    C1 -- "Yes" --> C2["File Rotation Process\n(Rename & .gz compression)"]
    C2 --> C3["Retention Evaluation\n(Limit roll_keep / roll_keep_days)"]
    C1 -- "No" --> C4["Write to physical disk storage"]
    C3 --> C4
    
    D --> D1["Text stream enters the shell terminal"]
    D1 --> D2["Captured by the host Log Driver\n(e.g., Docker Engine / systemd)"]
    
    E --> E1["Wrap the data into network packets"]
    E1 --> E2{"Protocol Choice?"}
    E2 -- "UDP" --> E3["Fast send without confirmation\n(Fire and Forget)"]
    E2 -- "TCP" --> E4["Secure handshake connection\n(Guaranteed Delivery)"]
    
    F --> F1["Data instantly deleted from RAM\n(Without consuming I/O resources)"]

The distribution pipeline above shows the flexibility of Caddy’s modular logging architecture, where the final handling of log streams is entirely tailored to the needs of the system architecture you’re building.

Summary #

• Disk Management: Always configure roll_size, roll_keep, and roll_keep_days on output file to prevent the risk of a full server disk triggering a system crash.
• Container Standard: Use output stdout in container environments (Docker/Kubernetes) so logs can be managed externally by the host runtime log driver.
• I/O Optimization: Leverage output discard paired with named matchers to ignore load balancer health check request logging, saving disk I/O resources.
• Centralized Distribution: Send logs in real time to remote log aggregators using output net with the TCP protocol choice (reliability) or UDP (speed).
• Buffering Performance: Caddy buffers logs in RAM memory for high performance, but be aware of the risk of losing the newest logs during a sudden power outage.
• Targeted Log Separation: Use modular handle blocks to separate access log writing destinations (e.g., API logs separated from regular web logs).
• Capacity Calculation: Do daily log disk space consumption estimates by calculating the average size per JSON log line before deploying the server to production.

← Previous: Format Log   Next: Plugins & Modules →