Caddy DNS #

By default, the Caddy web server uses the HTTP-01 challenge method to request and renew free SSL/TLS certificates from Let’s Encrypt or ZeroSSL. This method is very practical because it works automatically without requiring additional configuration, but it has significant operational limitations: your server must be directly connected to the public internet and have HTTP port 80 open so the certificate authority (CA) server can reach it back to verify domain ownership. Serious problems arise when you want to install SSL certificates on internal intranet servers behind corporate firewalls, local servers on home networks (behind NAT providers), or when you need a single certificate to dynamically protect all subdomains (wildcard certificates like *.example.com). To handle these scenarios, Caddy provides the caddy-dns plugin allowing you to switch to the DNS-01 challenge method. Through this method, the domain ownership verification process is done entirely by modifying the TXT record on your domain name server (DNS Provider) programmatically via API. We’ll thoroughly discuss the advantages of the DNS-01 challenge method, map out the supported DNS providers, practice secure API credential setup for Cloudflare and AWS Route53, compose issuer redundancy strategies, implement custom DNS delegation techniques, and investigate DNS record propagation failure troubleshooting.

Why Choose the DNS-01 Challenge? #

To determine the right TLS security strategy, you must understand the fundamental differences and operational trade-offs between the HTTP-01 challenge verification method and the DNS-01 challenge.

1. HTTP-01 Challenge (Classic Approach) #

• How It Works: The CA sends a random token to Caddy. Caddy places that token at the special path / .well-known/acme-challenge/ on the web server. The CA then accesses that URL via port 80 to verify.
• Advantages: Very easy, works instantly without requiring external API token configuration.
• Disadvantages: Demands port 80 fully open to the internet. Doesn’t support wildcard certificate creation (*.example.com), and is impossible to use on isolated servers (e.g., internal database servers).

2. DNS-01 Challenge (DNS-API-Based Approach) #

• How It Works: The CA sends a random token to Caddy. Caddy logs into your DNS account via API, then writes a new TXT record named _acme-challenge.example.com containing that token. The CA reads that DNS record from the internet, verifies the data match, then issues the certificate. After success, Caddy removes that TXT record again.
• Advantages: Supports wildcard certificate issuance, doesn’t require open inbound ports (80/443) to the public internet, and is very safe for securing internal servers.
• Disadvantages: Requires you to manage DNS provider API token credentials on the server, and the issuance process takes a bit longer because you must wait for DNS propagation time (propagation delay).

Here’s a brief comparison of usage suitability for this method based on deployment scenarios:

List of Supported DNS Providers #

The caddy-dns module isn’t managed as one big single plugin. Caddy separates it into independent per-provider modules to keep your Caddy binary file size efficient. You only need to compile the DNS provider module you use using xcaddy.

Here’s the list of official repository modules for several popular DNS providers:

• Cloudflare — github.com/caddy-dns/cloudflare
• AWS Route53 — github.com/caddy-dns/route53
• Google Cloud DNS — github.com/caddy-dns/googleclouddns
• DigitalOcean — github.com/caddy-dns/digitalocean
• Azure DNS — github.com/caddy-dns/azure
• DuckDNS — github.com/caddy-dns/duckdns
• Porkbun — github.com/caddy-dns/porkbun

Step-by-Step Guide: Cloudflare DNS #

Cloudflare is one of the most widely used DNS providers because of its stability and fast DNS record propagation speed.

Step 1: Compile Caddy with the Cloudflare Plugin #

Start by compiling a custom Caddy binary that has combined the Cloudflare DNS plugin:

xcaddy build --with github.com/caddy-dns/cloudflare

Step 2: Create a Scoped API Token in Cloudflare #

[!WARNING] NEVER use your Cloudflare account’s Global API Key. Using a Global API Key grants full access rights to delete all domains, modify billing, and damage your entire account if that key leaks from the server. Always create a scoped API Token with minimal access rights only allowed to manage specific domain DNS.

How to create a scoped API Token in the Cloudflare Dashboard:

1. Log into the Cloudflare Dashboard → click the My Profile icon in the top right → select API Tokens.
2. Click Create Token → choose the Edit zone DNS template.
3. Adjust the access Permissions:
   • Zone - DNS - Edit (Mandatory for writing TXT records).
   • Zone - Zone - Read (Mandatory so Caddy can find our domain’s Zone ID).
4. Determine the resource scope (Zone Resources):
   • Include - Specific zone - select your target domain (e.g., example.com).
5. Click Continue to summary then Create Token. Copy that token and store it safely.

Step 3: Configure Environment Variables Securely #

You must inject that token through OS environment variables so it isn’t written raw (hardcoded) in your Caddyfile configuration file:

# Add to the server shell environment variables
export CLOUDFLARE_API_TOKEN="our-secret-cloudflare-token-key"

Step 4: Compose the Caddyfile for Wildcard Certificates #

# Global options block
{
    email [email protected]
}

# Wildcard (*.example.com) and root domain (example.com) handling block
*.example.com example.com {
    tls {
        # Instruct Caddy to use the Cloudflare DNS challenge
        # taking the token value from the environment variable
        dns cloudflare {env.CLOUDFLARE_API_TOKEN}
    }
    
    # Custom subdomain routing using named matchers
    @app host app.example.com
    @api host api.example.com
    
    handle @app {
        reverse_proxy localhost:3000
    }
    
    handle @api {
        reverse_proxy localhost:8080
    }
    
    # Main page handling for the root domain
    handle {
        root * /var/www/html
        file_server
    }
}

AWS Route53 Configuration #

If your server infrastructure runs inside the AWS (Amazon Web Services) ecosystem, AWS Route53 is the ideal DNS provider choice.

1. Custom Binary Compilation #

xcaddy build --with github.com/caddy-dns/route53

2. Minimum IAM Policy for Route53 #

For security reasons, the AWS credentials used by Caddy may only have access to manage the related Hosted Zone. Here’s a JSON IAM Policy document with least privilege access rights that you must create and attach to Caddy’s IAM User or IAM Role:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "route53:GetChange",
                "route53:ChangeResourceRecordSets",
                "route53:ListResourceRecordSets"
            ],
            "Resource": [
                "arn:aws:route53:::hostedzone/ID_OF_OUR_DOMAIN_HOSTED_ZONE",
                "arn:aws:route53:::change/*"
            ]
        },
        {
            "Effect": "Allow",
            "Action": "route53:ListHostedZonesByName",
            "Resource": "*"
        }
    ]
}

3. AWS Route53 Caddyfile Configuration #

Caddy supports loading AWS credentials through several methods. If Caddy runs inside an AWS EC2 instance, Caddy can leverage the IAM Instance Profile feature automatically without writing any credentials in the Caddyfile. If running outside AWS, you load credentials via environment variables:

# Caddyfile configuration with Route53 DNS
*.example.com example.com {
    tls {
        dns route53 {
            # Option 1: If running on EC2 with an Instance Profile,
            # leave this parameter block empty. Caddy automatically takes the RAM role.
            
            # Option 2: Write explicitly using environment variables (for non-AWS hosts)
            access_key_id     {env.AWS_ACCESS_KEY_ID}
            secret_access_key {env.AWS_SECRET_ACCESS_KEY}
            region            {env.AWS_REGION}
        }
    }
    reverse_proxy localhost:3000
}

Using the DNS Challenge on Internal Networks (Intranet) #

A very popular use case of the DNS-01 challenge is providing official HTTPS encryption for internal web servers isolated from the internet. For example, your internal metrics visualization server running on the monitor.company.com subdomain in the office local network:

# Private intranet server security configuration
monitor.company.com {
    tls {
        # Request an official Let's Encrypt certificate via DNS.
        # The Let's Encrypt CA can verify our domain ownership through public DNS,
        # even though the physical monitor.company.com server has no public IP
        # and can't be accessed from outside the office.
        dns cloudflare {env.CLOUDFLARE_API_TOKEN}
    }
    
    # Additional security layer: Only allow physical access from the office LAN subnet
    @external not remote_ip 10.0.0.0/8 192.168.0.0/16
    respond @external "Access Denied: Internal Network Only" 403
    
    reverse_proxy localhost:9090
}

With this tactic, your employees’ browsers no longer show the annoying red “Self-Signed Certificate / Connection Not Private” warning, while also guaranteeing local data traffic security from network eavesdropping dangers.

Issuer Redundancy & Fallback (Let’s Encrypt + ZeroSSL) #

Public certificate authorities can sometimes experience unscheduled maintenance (downtime) or hit certificate submission rate limits (rate limit exceed). To ensure reliable certificate availability, you can configure Caddy to use two different issuers sequentially. Caddy tries contacting the first issuer (e.g., Let’s Encrypt), and if that fails, Caddy automatically switches to the second issuer (ZeroSSL) as backup:

# TLS issuer redundancy configuration example
*.example.com example.com {
    tls {
        # Issuer 1: Let's Encrypt (Default)
        issuer acme {
            ca https://acme-v02.api.letsencrypt.org/directory
            dns cloudflare {env.CLOUDFLARE_API_TOKEN}
        }
        
        # Issuer 2: ZeroSSL (Backup)
        issuer acme {
            ca https://acme.zerossl.com/v2/DV90
            eab_key_id  {env.ZEROSSL_KEY_ID} # EAB credentials for ZeroSSL
            eab_mac_key {env.ZEROSSL_MAC_KEY}
            dns cloudflare {env.CLOUDFLARE_API_TOKEN}
        }
    }
    reverse_proxy localhost:3000
}

DNS Challenge Delegation (CNAME Redirection) #

In enterprise security architectures, DNS administrator teams often strictly forbid embedding DNS API tokens with modification access rights on public edge web servers. The main company.com domain is considered too valuable to place its API token on production virtual machines.

To break this deadlock, you can apply the DNS Challenge Delegation tactic using CNAME records. You delegate the ACME token verification process from the main domain to a backup domain with lower security value (e.g., company-dns.com dedicated specifically for validation).

Configuration Flow: #

1. On the Main Domain DNS Server (company.com): Create a CNAME record redirecting ACME verification requests to the backup domain:

   _acme-challenge.app.company.com CNAME _acme-challenge.app.company-dns.com
   

2. On the Backup Domain DNS Server (company-dns.com): Use a Cloudflare API Token only allowed to manage the company-dns.com zone.

3. In your Caddy Server’s Caddyfile: Configure Caddy to ignore the main domain when interacting with the DNS provider API, and instruct it to modify the backup domain DNS instead:

# CNAME DNS Challenge Delegation Configuration
app.company.com {
    tls {
        # Instruct Caddy to write TXT records on company-dns.com
        # even though Caddy is requesting a certificate for app.company.com
        dns cloudflare {env.CF_DELEGATE_API_TOKEN} {
            # Target zone override parameter
            # (Depends on the custom provider plugin driver implementation)
        }
    }
    reverse_proxy localhost:3000
}

(Note: Most modern ACME client drivers automatically follow CNAME routes transparently when searching for the API endpoint to write TXT tokens).

Troubleshooting DNS Validation Problems #

The DNS-01 challenge involves more external parties (DNS servers, cache resolvers, global internet networks) than HTTP-01. This sometimes triggers operational failures. Here’s a troubleshooting guide for DNS validation failures:

1. Problem: Validation Fails Due to Caching Obstacles (Propagation Timeout) #

• Symptoms: Caddy successfully writes the token to Cloudflare, but Let’s Encrypt returns a “No TXT record found” error.
• Cause: Caddy or the Let’s Encrypt testing server queries the TXT record status to the DNS server before the record is truly propagated to all global replica DNS servers.
• Solution: You can force Caddy to use fast external public DNS resolvers and not use local cache to verify propagation before declaring readiness for Let’s Encrypt testing. Add the resolver parameter inside the tls block:

*.example.com {
    tls {
        dns cloudflare {env.CLOUDFLARE_API_TOKEN}
        
        # Use Google & Cloudflare DNS resolvers to check propagation
        resolvers 1.1.1.1 8.8.8.8
    }
}

2. Using the dig Command for Manual Debugging #

Before requesting a certificate, you can see whether the TXT record has been correctly written on the name server using the terminal command:

# Query our domain's ACME TXT record directly to a public resolver
dig TXT _acme-challenge.example.com @1.1.1.1

# The correct output should display the random token line:
# _acme-challenge.example.com. 120 IN TXT "h7y1K9lPqRstUvwXyz..."

ACME DNS-01 Challenge Authentication Flow Diagram #

To understand the orderly network interaction sequence between Caddy, the DNS provider, and the Certificate Authority (CA) during the DNS-01 challenge process, look at the flowchart visualization below:

flowchart TD
    A["1. Caddy Requests a New Certificate\n(Send an Order to the Let's Encrypt CA)"] --> B["2. The CA Sends the ACME Challenge\n(Random challenge token that must be written)"]
    
    B --> C["3. Caddy Calls the DNS API Driver\n(Send a record writing request via the Provider API)"]
    C --> D["4. The DNS Provider Writes the TXT Record\n(Create the _acme-challenge.example.com record)"]
    
    D --> E["5. Caddy Does Self-Polling\n(Test the record status via the 1.1.1.1 resolvers)"]
    
    E --> E1{"6. Is the record detected?"}
    E1 -- "No" --> E2["Wait (Propagation Delay)\nand repeat the check"]
    E2 --> E
    E1 -- "Yes" --> F["7. Caddy Notifies the CA\n(Declares the record is ready for validation)"]
    
    F --> G["8. The CA Does DNS Validation\n(Reads the TXT record from the public internet)"]
    
    G --> G1{"9. Is the token valid?"}
    G1 -- "No" --> G2["Process Fails\n(Log the error & retry)"]
    G1 -- "Yes" --> H["10. The CA Issues the SSL Certificate\n(The certificate is sent back to Caddy)"]
    
    H --> I["11. Caddy Cleans Up the DNS Record\n(Calls the DNS API to delete the TXT token)"]
    I --> J["12. The Certificate is Stored in Storage\n(HTTPS is active on the Caddy server)"]

Summary #

• Wildcard Solution: The DNS-01 challenge is the only official ACME method for issuing free wildcard certificates from Let’s Encrypt.
• Network Isolation: This method lets internal servers behind firewalls or private NAT obtain public SSL certificates without opening port 80 to the internet.
• Least Privilege Principle: Always use a scoped Cloudflare API Token locked specifically to certain domains, not your account’s Global API Key.
• Token Security: Secure DNS API tokens in environment variable files (EnvironmentFile with 600 access permissions), never write them directly in the Caddyfile.
• AWS IAM Bypass: Leverage the IAM Instance Profile on EC2 instances so Caddy can access Route53 without needing physical credential configuration.
• Quick Verification: Use the resolvers 1.1.1.1 8.8.8.8 parameter to avoid validation process failures caused by slow local DNS propagation.
• CNAME Delegation: Apply the DNS Challenge Delegation CNAME technique if internal security policies forbid placing API tokens on public edge servers.

← Previous: Popular Plugins   Next: Creating Plugins →