Access Log #

The access log is a structured chronological record of every HTTP request received and processed by the Caddy web server. Access logs act as the black box of your web infrastructure — recording who requested resources, what pages were accessed, when transactions happened, what response status was returned, how long processing took, and even the size of transmitted data. Having well-configured access logs is crucial for monitoring response latency performance, doing post-incident security audits, detecting cyber attacks in real time, and meeting regulatory compliance standards (compliance audits). Caddy provides a very powerful, modular built-in log directive that natively records information in structured JSON format friendly to modern log collection engines. We’ll discuss in depth the strategic role of access logs, practice modular per-site logging configuration, dissect the anatomy of Caddy’s JSON log schema, learn log filtering tactics to save disk space, and compose log sampling configurations to balance server performance under high traffic.

The Strategic Role of Access Logs in Production Environments #

In industry-scale server management, ignoring or disabling access logs for performance reasons is a very risky decision. Access logs aren’t just junk text lines filling hard disk capacity; they’re an invaluable raw data source for your operations and business.

There are four main roles of access logs in production environments:

1. Security Audit Trails #

If your server is breached or suffers security exploit attempts (like SQL injection or authentication bypass), access logs are the only silent witness recording the attacker’s IP address, precise attack time, the User-Agent used, and the URL parameter payloads attempted.

2. Application Performance Monitoring #

Caddy records request processing duration metrics down to the microsecond level. By periodically analyzing access logs, you can detect if a certain API endpoint suddenly experiences latency spikes (slow responses), helping you take database optimization or server capacity addition actions before end users feel application slowness.

3. Business and User Behavior Analysis #

Although client-side tracking tools like Google Analytics exist, those tools are often blocked by client browser ad-blocker extensions. Access logs record pure traffic at the server network level, providing 100% accurate data on hit counts, visitor operating systems, and geographic access distribution.

4. Automatic Threat Detection Integration #

Automated security tools like Fail2ban or your internal monitoring scripts rely on dynamically reading access logs to detect suspicious access patterns (like thousands of 404 error requests within seconds indicating directory brute force attacks), then triggering automatic blocking at the OS firewall level.

The log Directive in Caddy: Modular and Per-Site Logging #

Caddy has a very modular approach to log handling. Unlike Nginx, which by default combines all virtual hosts’ logs into one global access.log file before manual separation, Caddy lets you define logging rules independently indexed directly inside each server block (per-site logging).

The basic syntax of the log directive is:

log [<name>] {
    output <writer_module> ...
    format <encoder_module> ...
    level  <level_name>
}

Production Per-Site Logging Configuration Example #

# Example: Separate Logging for Different Subdomains
api.example.com {
    # 1. Enable access logging specifically for the API domain
    # The log is stored in a separate file in JSON format
    log {
        output file /var/log/caddy/api_access.log
        format json
    }
    
    reverse_proxy localhost:8080
}

dashboard.example.com {
    # 2. Enable access logging specifically for the Dashboard domain
    log {
        output file /var/log/caddy/dashboard_access.log
        format json
    }
    
    root * /var/www/dashboard
    file_server
}

By separating log files like above, the API developer team can focus on monitoring the api_access.log file without being disturbed by static image file traffic records from the dashboard subdomain.

The Advantages and Structure of Caddy’s JSON Log Format #

By default, if you don’t explicitly specify a format module, Caddy records log data in structured JSON format. This design decision is a big leap compared to traditional web servers still using plain text line formats (Common Log Format or CLF).

Why Structured JSON? #

Plain Line Format (Apache/Nginx CLF):
  127.0.0.1 - - [16/Jun/2026:18:30:00 +0700] "GET /api/data HTTP/1.1" 200 2326
  ✗ DON'T use this in modern systems: Requires complex regex for parsing.
  ✗ Error-prone if the user-agent contains spaces or double quote characters.

Structured JSON Format (Caddy):
  {"level":"info","ts":1781609400,"request":{"remote_ip":"127.0.0.1","proto":"HTTP/1.1","method":"GET","uri":"/api/data"},"status":200,"size":2326}
  ✓ CORRECT: Can be read directly by all programming languages without regex.
  ✓ Ready to send to log aggregators (Loki, Elasticsearch, Datadog) without extra parsers.

Anatomy of Caddy’s Main JSON Log Fields #

Let’s dissect the structure of a real JSON log file produced by Caddy:

{
  "level": "info",
  "ts": 1781609400.123456,
  "logger": "http.log.access.log0",
  "msg": "handled request",
  "request": {
    "remote_ip": "203.0.113.50",
    "remote_port": "54321",
    "client_ip": "203.0.113.50",
    "proto": "HTTP/2.0",
    "method": "GET",
    "host": "api.example.com",
    "uri": "/v1/users?limit=10",
    "headers": {
      "User-Agent": ["Mozilla/5.0 ..."],
      "Accept": ["application/json"]
    },
    "tls": {
      "resumed": false,
      "version": 772,
      "cipher_suite": 4865,
      "proto": "h2",
      "server_name": "api.example.com"
    }
  },
  "user_id": "",
  "duration": 0.045231,
  "size": 1240,
  "status": 200,
  "resp_headers": {
    "Content-Type": ["application/json"],
    "Server": ["Caddy"]
  }
}

Here’s the detailed explanation of the fields generated in the JSON log payload above:

Caddy Adapter Compilation Results (Caddy Adapt) for Logs #

When you trigger the caddy adapt command to convert your declarative Caddyfile into Caddy’s native JSON configuration file, the adapter module automatically composes a global logging block combined with per-route configuration.

Here’s an illustration snippet of the Caddy JSON compilation format resulting from adapting our access log logging block:

{
  "logging": {
    "logs": {
      "log0": {
        "writer": {
          "output": "file",
          "filename": "/var/log/caddy/api_access.log"
        },
        "encoder": {
          "format": "json"
        },
        "include": [
          "http.log.access.log0"
        ]
      }
    }
  }
}

In the JSON representation above, we see Caddy maps the internal identifier log0 into a local binary writer format, proving how structured and modular the entire logging system is under the Caddy runtime hood.

Log Filtering #

On high-traffic websites, access log files can balloon to tens of gigabytes within days. Most of the log file contents are usually filled by static asset calls (like .png, .jpg, .css, or .js files) or periodic health check requests from the front load balancer, which aren’t highly valuable for security audits.

You can configure route filters in Caddy to exclude those non-essential requests from access log recording, significantly saving hard disk capacity.

Log Filter Writing Syntax in the Caddyfile #

# Example: Production Log Filter Configuration
example.com {
    # 1. Define a named matcher for routes you DON'T want recorded in the log
    # We filter static assets and health check routes
    @non_loggable {
        path *.png *.jpg *.jpeg *.gif *.svg *.css *.js *.ico
        path /healthz /ping
    }

    # 2. Configure the log with the filter option
    log {
        output file /var/log/caddy/access.log
        format json
        
        # ✓ CORRECT: Skip logging for requests matching the @non_loggable matcher
        exclude @non_loggable
    }

    root * /var/www/html
    file_server
}

With the configuration above, if a client requests the /logo.png file or calls the /healthz route, Caddy still processes and serves the file with a 200 OK status, but Caddy won’t write a single line to the /var/log/caddy/access.log file. This saves your disk I/O write capacity.

Log Sampling for High-Traffic Infrastructure #

If your site serves extreme traffic (e.g., tens of thousands of requests per second), writing access logs for every single transaction to local storage can cause serious disk I/O performance degradation (disk bottleneck).

To handle this scenario without completely disabling access logs, Caddy provides the Log Sampling feature. This feature lets you set rules to only write a percentage of logs to the file randomly or based on quota multiples.

Log Sampling Caddyfile Configuration #

# Example: Log Sampling for Extreme Traffic
example.com {
    log {
        output file /var/log/caddy/access.log
        format json
        
        # Sampling Configuration:
        # Only record 10% of the total successful access logs (200 status)
        # Parameters: observation interval & maximum log limit written per interval
        sampling {
            # Sampling time interval (e.g., 1 second)
            interval 1s
            
            # The maximum first logs always written completely per interval
            first 100
            
            # The log recording multiple after the "first" limit is exceeded
            # A value of 10 means: Caddy only records 1 of every 10 subsequent requests
            thereafter 10
        }
    }

    reverse_proxy localhost:8080
}

Sampling Logic #

In the configuration above, within every 1-second duration:

1. The first 100 incoming requests are always recorded 100% completely without any being skipped.
2. After request #100 is exceeded within the same second, Caddy only records requests #110, #120, #130, and so on.
3. When the next second starts, the counter resets back to zero.

This tactic guarantees you still have sufficiently accurate representative statistical data for calculating your backend’s average latency performance without endangering the server’s disk I/O health.

Integration with Log Aggregators (Grafana Loki & ELK Stack) #

Leaving JSON log data piling up in individual VM server local files makes it hard to monitor system health if your VMs scale to dozens of units (scaling groups). The industry best practice is sending log data to a centralized log aggregator.

Because Caddy logs are already natively structured JSON format, you don’t need complicated Grok regex pattern integration on your log aggregator.

1. Integration with Grafana Loki (Using Promtail) #

You can install Promtail on your Caddy server to read the /var/log/caddy/access.log file in real time and forward it to Grafana Loki:

# Example promtail-config.yaml configuration
server:
  http_listen_port: 9080

clients:
  - url: http://loki-server:3100/loki/api/v1/push

scrape_configs:
  - job_name: caddy-logs
    static_configs:
      - targets: [localhost]
        labels:
          job: caddy-access
          env: production
          __path__: /var/log/caddy/*.log
    pipeline_stages:
      # Because Caddy logs are JSON, we only need to define a JSON stage
      - json:
          expressions:
            status: status
            duration: duration
            method: request.method
            uri: request.uri

2. Integration with the ELK Stack (Using Filebeat) #

If you use Elasticsearch and Logstash, Filebeat can be configured to directly parse Caddy log entries as native JSON documents:

# Example filebeat.yml configuration
filebeat.inputs:
  - type: log
    enabled: true
    paths:
      - /var/log/caddy/access.log
    # Parse the JSON format directly in Filebeat
    json.keys_under_root: true
    json.overwrite_keys: true
    json.add_error_key: true

The Request Lifecycle Against Access Log Recording #

To avoid misunderstanding about when log data is written and what variables are recorded, you must learn the log data processing lifecycle in Caddy. Access logs aren’t written at the start of a request arrival, but right after the response is fully sent to the client browser.

Let’s look at the visual flow below using a flowchart:

flowchart TD
    A["1. Client HTTP Request Arrives"] --> B["2. Caddy records the start time (ts)"]
    B --> C["3. Caddy processes the request through middleware/backend"]
    C --> D["4. The backend returns the response data"]
    D --> E["5. Caddy finishes sending the response to the Client"]
    
    E --> F["6. Calculate the processing duration (duration = ts_end - ts)"]
    F --> G{"7. Does the request match the 'exclude' filter?"}
    
    G -- "Yes" --> H["8. Skip recording (Request is not written to the log)"]
    G -- "No" --> I{"9. Is the 'sampling' feature active?"}
    
    I -- "Yes" --> J{"Does the request pass the sampling selection?"}
    I -- "No" --> K["11. Compose the complete JSON log structure"]
    
    J -- "Yes" --> K
    J -- "No" --> H
    
    K --> L["12. Send the JSON log binary to the Output module\n(e.g., write to the access.log file)"]

Summary #

• Main Function: Access logs record the chronological transactions of HTTP clients on the server, essential for cyber security audits, latency analysis, and Fail2ban integration.
• JSON Advantages: Caddy natively records access logs in data-rich structured JSON format, friendly to modern log aggregator libraries.
• Per-Site Logging: The log directive is written directly inside each server block to independently isolate log file storage.
• Log Filtering: Use the exclude directive combined with named matchers to skip static asset logging, saving disk capacity usage.
• Log Sampling: Limit log writing volume on high-traffic infrastructure using the sampling option to minimize hard disk I/O bottlenecks.
• Modern Integration: Caddy JSON data is ready to send directly to Grafana Loki or Elasticsearch without needing complicated additional regex parsing.
• Adapter Compilation: The native JSON logging pipeline can be observed structurally by triggering the caddy adapt utility on your Caddyfile configuration.
• Writing Timing: Access logs are always written right after the entire response is fully sent back to the client to calculate processing duration time precisely.

← Previous: Logging   Next: Error Log →