DNS Challenge #

The DNS-01 Challenge is one of the most reliable domain ownership verification methods defined in the ACME protocol. Unlike the HTTP-01 or TLS-ALPN-01 challenges, which prove domain control through direct network connections to a web server on ports 80 or 443, the DNS challenge works entirely at the domain name infrastructure layer (Domain Name System). By presenting a cryptographic verification token as a text record (TXT record) in your domain’s DNS zone, you can prove domain ownership to the Certificate Authority (CA) without opening any inbound ports from the outside internet.

This architectural advantage makes DNS-01 the favorite method for various advanced deployment scenarios. It’s the only method CAs approve for issuing wildcard certificates (*.example.com). Additionally, the DNS challenge is an absolute solution for web servers running on private networks (like corporate intranets, internal Kubernetes clusters, or development servers behind strict NAT) to automatically obtain trusted public SSL/TLS certificates.

How the DNS-01 Challenge Works in Detail #

Although the entire verification process runs automatically in the background by Caddy, understanding this protocol handshake timeline is very helpful when you need to diagnose verification failures.

sequenceDiagram
    autonumber
    participant C as Caddy Server
    participant API as DNS Provider API (e.g. Cloudflare)
    participant CA as CA Server (Let's Encrypt)
    participant DNS as Public DNS Nameserver

    C->>CA: POST /newOrder (Request certificate for *.example.com)
    CA-->>C: JWS Response (Here's the challenge token: H3t8xK_qP2...)
    C->>API: POST /dns_records (Create TXT record _acme-challenge.example.com)
    API->>DNS: Write TXT record to Nameserver
    Note over C, DNS: Caddy waits for DNS propagation (caching check)
    CA->>DNS: Query TXT _acme-challenge.example.com
    DNS-->>CA: TXT record response (H3t8xK_qP2...)
    Note over CA: CA verifies the cryptographic token match
    CA-->>C: Issue TLS Certificate (DER Format)
    C->>API: DELETE /dns_records (Remove temporary TXT record)

Process Timeline Details: #

1. Initial Request (T+0): Caddy contacts the CA’s ACME directory and requests a new certificate order for the domain, e.g., *.example.com.
2. Token Delivery (T+1): The CA responds by providing a unique verification token and asking Caddy to create a TXT record with the hostname _acme-challenge.example.com.
3. Notification to the DNS API (T+2): Caddy calls your DNS provider’s API (like Cloudflare or AWS Route 53) to create that TXT record with the signed token value.
4. Record Propagation (T+3): The DNS provider publishes the new record on their array of authoritative nameservers.
5. Self Propagation Check (T+4): Before telling the CA to verify, Caddy intelligently performs a local DNS query to make sure the record is readable on the internet.
6. CA Verification Query (T+n): Once the record is detected, Caddy tells the CA the challenge is ready to verify. The CA then independently performs a public DNS query to read that TXT record.
7. Certificate Issuance (T+n+1): If the token matches, the domain authorization status becomes valid. The CA issues the TLS certificate and sends it to Caddy.
8. DNS Zone Cleanup (T+n+2): Caddy calls the DNS API again to remove the temporary TXT record so your DNS zone stays clean and tidy.

Understanding the Main Obstacle: Propagation Delay #

The biggest challenge in implementing the DNS challenge is Propagation Delay. Unlike HTTP-01, which is instant once the token is written to disk, DNS record changes take time to spread across the internet infrastructure.

Factors Affecting DNS Propagation Time:

1. TTL Caching Mechanism:
   Every DNS record has a Time-To-Live (TTL) value. Public caching resolvers
   (like Google DNS 8.8.8.8 or Cloudflare 1.1.1.1) store previous query results
   for the remaining TTL time. If the CA queries before the record propagates,
   they get an empty answer and reject the challenge.
   
2. DNS Provider Replication Speed:
   Global providers (like Cloudflare) use Anycast architecture that replicates
   record changes to hundreds of their nameservers worldwide in < 10 seconds.
   Other, slower providers may need 1 to 5 minutes to replicate data.
   
3. Multi-Perspective Validation (MPV):
   Because Let's Encrypt checks DNS records from several global satellite locations
   simultaneously, all of your domain's authoritative nameservers must have finished
   replicating the new record data for the challenge to be declared valid.

Configuring Propagation Parameters in the Caddyfile #

Caddy provides configuration options inside the tls directive to control how Caddy handles DNS propagation delay:

# Caddyfile configuration with advanced propagation parameters
*.example.com {
    tls {
        dns cloudflare {env.CLOUDFLARE_API_TOKEN}
        
        # Maximum wait time for Caddy to detect the record before giving up (default: 2m)
        # Increase this value if your DNS provider replicates data slowly
        propagation_timeout 5m
        
        # Static delay before Caddy starts calling the DNS resolver for the first check
        propagation_delay 30s
        
        # Specify specific public DNS resolvers to check propagation
        # Highly recommended to avoid the OS's local cache
        resolvers 1.1.1.1 8.8.8.8
    }
    
    reverse_proxy localhost:8080
}

Split-Horizon DNS and Local Resolution #

In corporate internal network architectures, you often implement Split-Horizon DNS (or Split-Brain DNS). This is a condition where a public domain (e.g., app.company.com) points to a private IP (like 10.0.0.5) when accessed from inside the office, but points to a public IP when accessed from the outside internet.

When Caddy performs its self-propagation verification inside the corporate local network, it queries your internal DNS server. If your internal DNS server replicates slowly or has special protection rules, Caddy may think the TXT record doesn’t exist yet and cause a timeout failure.

The solution is configuring the resolvers option in the Caddyfile to use external nameserver IPs (your domain’s real nameservers, like Cloudflare nameservers or 1.1.1.1) so Caddy’s checks pass quickly and accurately:

# Using external resolvers to traverse Split-Horizon DNS
app.company.com {
    tls {
        dns cloudflare {env.CF_API_TOKEN}
        # bypass the corporate local DNS, check directly against Cloudflare/Google DNS
        resolvers 1.1.1.1 8.8.8.8
    }
    reverse_proxy localhost:8080
}

Advanced Tactic: DNS Challenge Delegation (CNAME Redirection) #

In strict corporate environments, the DNS zone for the main production domain (e.g., company.com) is usually tightly locked for security reasons. Giving DNS API credentials with write access to this main zone on production Caddy web servers can pose a security risk that security teams won’t accept.

To solve this security dilemma, you can use the DNS Challenge Delegation tactic with CNAME records. The ACME protocol defines that when the CA looks for a TXT record at _acme-challenge.example.com and finds a CNAME record, the CA follows that CNAME pointer to the new target and looks for the TXT record at the destination domain.

DNS Challenge Delegation Flow via CNAME:

Main DNS Zone (company.com) - Tightly Locked, No API Access:
  - Create a permanent CNAME:
    _acme-challenge.production.company.com  CNAME  production.challenge.company-ops.com

Secondary DNS Zone (company-ops.com) - API Access Allowed for Caddy:
  - Caddy calls the DNS API only for the company-ops.com zone.
  - Caddy creates a TXT record at:
    production.challenge.company-ops.com  TXT  "cryptographic_verification_token"

Validation Result:
  - Let's Encrypt looks for TXT at _acme-challenge.production.company.com.
  - The CA is redirected to production.challenge.company-ops.com.
  - The CA successfully reads the verification token.
  - A certificate is issued for production.company.com without exposing the main API key!

Caddyfile DNS Delegation Configuration: #

In your Caddyfile, just define your DNS plugin pointed at the secondary zone (for example, a Cloudflare token holding permission for the company-ops.com zone):

# Caddyfile on the production server
production.company.com {
    tls {
        # The Cloudflare token below only has permission to edit the company-ops.com zone
        dns cloudflare {env.CF_CHALLENGE_ZONE_TOKEN}
    }
    
    reverse_proxy localhost:8080
}

Main DNS Provider Configuration Guide #

Here are detailed configuration examples for several of the largest DNS service providers supported by Caddy:

1. AWS Route 53 #

Route 53 uses AWS IAM identity architecture. We recommend injecting credentials using standard AWS environment variables:

# AWS Route 53 Caddyfile
*.example.com {
    tls {
        dns route53 {
            # Optional: set the retry count if the AWS API is busy
            max_retries 10
        }
    }
    reverse_proxy localhost:3000
}

# Inject these environment variables before running Caddy
export AWS_ACCESS_KEY_ID="AKIAIOSFODNN7EXAMPLE"
export AWS_SECRET_ACCESS_KEY="wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY"
export AWS_REGION="us-east-1"

2. Google Cloud DNS #

GCP DNS requires a JSON key file from a Service Account with the DNS Administrator role.

Steps to Create a Service Account in GCP: #

1. Run the commands in Cloud Shell or on a local computer:

# Create a new Service Account
gcloud iam service-accounts create caddy-dns-sa --display-name "Caddy DNS SA"

# Grant the dns.admin role to the Service Account
gcloud projects add-iam-policy-binding PROJECT_ID \
    --member "serviceAccount:caddy-dns-sa@PROJECT_ID.iam.gserviceaccount.com" \
    --role "roles/dns.admin"

# Download the Service Account JSON key to a local file
gcloud iam service-accounts keys create /etc/caddy/gcp-dns-key.json \
    --iam-account caddy-dns-sa@PROJECT_ID.iam.gserviceaccount.com

Google Cloud DNS Caddyfile: #

# Google Cloud DNS Caddyfile
*.example.com {
    tls {
        dns googleclouddns {
            gcp_project "our-gcp-project-name"
        }
    }
    reverse_proxy localhost:8000
}

# Point Caddy to the Service Account JSON file location
export GOOGLE_APPLICATION_CREDENTIALS="/etc/caddy/gcp-dns-key.json"

3. Azure DNS #

Azure requires complete information about an Azure Active Directory Service Principal:

# Azure DNS Caddyfile
*.example.com {
    tls {
        dns azure {
            tenant_id       {env.AZURE_TENANT_ID}
            client_id       {env.AZURE_CLIENT_ID}
            client_secret   {env.AZURE_CLIENT_SECRET}
            subscription_id {env.AZURE_SUBSCRIPTION_ID}
            resource_group  "our-dns-resource-group"
        }
    }
    reverse_proxy localhost:5000
}

DNS Provider Performance Comparison Table #

The DNS provider choice greatly affects the initial certificate issuance speed on your server. Here’s a comparison table of several popular providers:

API Credential Security Practices #

DNS API credentials are the gateway keys to your entire domain. If these credentials leak, outsiders can change your email traffic (MX records), redirect your site to phishing servers, or completely take down your services.

Credential Security Policies: #

• Very Strict File Permissions: Store all credential environment variables in a separate configuration file (e.g., /etc/caddy/caddy.env) and set its file permissions so only root and the caddy user can read it:

# Create a protected credentials file
sudo touch /etc/caddy/caddy.env
sudo chown caddy:caddy /etc/caddy/caddy.env
sudo chmod 600 /etc/caddy/caddy.env

• Use Docker Secrets or Kubernetes Secrets: In container orchestration environments, don’t write API keys in raw YAML manifest files. Use the platform’s built-in secret management feature to inject values securely into the Caddy container at runtime.

DNS Challenge Troubleshooting #

1. Caddy Fails to Create the TXT Record in DNS #

• Log Symptoms: Error message failed to create TXT record or Authentication Error.
• Cause: The API token permissions granted in the provider dashboard are insufficient (for example, forgetting to grant Edit access or selecting the wrong domain zone).
• Solution: Recheck the token configuration in the DNS provider dashboard. For Cloudflare, verify with a manual curl query command to ensure your token has write access to the relevant zone.

2. The ACME Challenge Times Out #

• Log Symptoms: Caddy repeatedly shows waiting for propagation... logs until finally producing a timeout error.
• Cause: The provider’s DNS nameservers replicate slowly, or Caddy uses the OS’s local DNS resolver, which caches an old empty query result.
• Solution: Add the resolvers 1.1.1.1 8.8.8.8 configuration to the tls block in the Caddyfile so Caddy checks propagation directly against public resolvers that refresh data quickly, and raise the propagation_timeout parameter to 10m.

Summary #

• DNS-01 Challenge — The ACME verification method using TXT records in the domain’s DNS zone to prove administrative control.
• No Open Ports — DNS-01 doesn’t require inbound HTTP (80) or HTTPS (443) ports open to the server, making it the ideal solution for servers on private/internal networks.
• The Only Option for Wildcards — The DNS challenge is the only method Certificate Authorities allow for issuing wildcard certificates (*.domain.com).
• Propagation Delay — The wait time for DNS data replication across global nameservers. Configure the propagation_timeout and custom resolvers parameters in the Caddyfile to prevent timeouts.
• DNS Delegation — The tactic of redirecting ACME verification to a secondary DNS zone using CNAME to keep the main domain’s API credentials secure.
• Credential Security — Protect the environment variable file containing DNS API tokens with 600 file permissions and limit token permission scope to only the relevant DNS zone edit level.

← Previous: Wildcard Certificate   Next: Web Server →