xcaddy #

xcaddy is the official command-line tool developed by the Caddy team to simplify the process of building custom Caddy binaries equipped with additional modules or plugins. Caddy is designed with a very flexible modular architecture where almost every feature — from HTTP protocol handling, DNS challenge certificate provisioning, to data storage systems — runs as an independent module. However, unlike some traditional web servers that load modules dynamically from external library files while the application runs (runtime plugins), Caddy uses a compile-time plugins approach. This requires you to recompile Caddy’s source code together with the plugin source code every time you want to add or remove a feature. xcaddy exists to simplify this complex compilation process into one simple command-line invocation by automating Go modules dependency handling behind the scenes. We’ll thoroughly examine the advantages of the compile-time module architecture, learn how to install xcaddy, practice building custom binaries for one or many plugins, do precise version management, compose Docker multi-stage build workflows, integrate it into CI/CD pipelines, and do cross-compilation for various hardware architectures.

Architecture Concept: Compile-Time vs Runtime Plugins #

To understand why you need a tool like xcaddy, you must first understand the fundamental comparison between modules loaded while the application runs (runtime/dynamic plugins) and modules combined at compile time (compile-time/static plugins).

Several other popular web servers use dynamic plugins architecture. The advantage is users can install or disable modules by editing plain text configuration files without touching the main web server binary. However, this approach has fairly heavy technical consequences:

  1. ABI Compatibility: Dynamic modules must be compiled using exactly the same compiler version as the main web server binary. If there’s a minor compiler difference, the module refuses to load (Application Binary Interface mismatch).
  2. Performance (Overhead): Communication between dynamic modules in RAM often requires a bridge layer (IPC / Inter-Process Communication) that can add micro-latency to every HTTP request handling.
  3. Stability: A memory allocation error (segfault) in one bad dynamic module can directly collapse the stability of the entire main web server binary process.

Caddy chooses the compile-time plugins approach natively supported by the Go programming language. In this architecture, all third-party plugin source code is combined directly into the Caddy codebase before the Go compiler creates the final binary file. This approach provides several outstanding advantages:

  • Single Self-Contained Binary: The compilation result is a single binary file without dependencies on external library files (.so on Linux or .dll on Windows). You only need to move this one binary file to deploy the Caddy server to the target machine.
  • Type Safety: The Go compiler checks all code dependencies and data types from the start. If there’s an API version mismatch between Caddy and a plugin, the compilation process immediately fails early, preventing unexpected runtime crashes in production environments.
  • Maximum Performance: Communication between modules happens directly as native Go function calls within the same isolated memory. This eliminates external communication overhead latency.

The main weakness of this approach is that every time you want to update or add a new plugin, you must redo the compilation process. This is where xcaddy plays a crucial role by automating the entire workflow of writing Go glue code and invoking the Go compiler for you.

Installing xcaddy #

Before using xcaddy, you must install it on your development machine first. Because xcaddy’s job is triggering Go code compilation, your machine must have an active Go compiler (Go SDK) installation with a minimum version matching Caddy’s requirements (Go version 1.21 or above recommended).

Here are three xcaddy installation methods you can choose:

If your machine already has the Go SDK installed, this is the fastest and cleanest method to download and compile the latest xcaddy binary directly from its official repository:

# Compile and install xcaddy to your $GOPATH/bin or $HOME/go/bin directory
go install github.com/caddyserver/xcaddy/cmd/xcaddy@latest

# Make sure the Go bin directory is already in your system PATH, then verify the installation:
xcaddy version
# Example output: xcaddy v0.4.1 go1.22.4 darwin/arm64

Method 2: Using a System Package Manager #

If you prefer automatic package management through the operating system, several distributions provide custom repositories for xcaddy:

  • For macOS (Using Homebrew):
    brew install xcaddy
  • For Debian/Ubuntu (Using the Official Caddy Repository):
    # Add the official Caddy repository if not already installed
sudo apt install -y debian-keyring debian-archive-keyring apt-transport-https
curl -1sLf 'https://dl.cloudsmith.io/public/caddy/xcaddy/gpg.key' | sudo gpg --dearmor -o /usr/share/keyrings/caddy-xcaddy-archive-keyring.gpg
curl -1sLf 'https://dl.cloudsmith.io/public/caddy/xcaddy/config.deb.txt?term=exact' | sudo tee /etc/apt/sources.list.d/caddy-xcaddy.list
sudo apt update
sudo apt install xcaddy

Method 3: Downloading the Pre-Compiled Binary Directly #

If you don’t want to install the Go SDK or a package manager on your computer (e.g., when you just want to download the binary instantly), you can manually download the xcaddy binary archive file from the official GitHub releases page: xcaddy Releases. Extract the archive file and move the xcaddy binary into your system PATH folder (like /usr/local/bin).

Compiling Caddy Using xcaddy #

After xcaddy is installed and the Go compiler is accessible by the system, you’re ready to build a custom Caddy binary.

1. Building the Default Binary (Without Additional Plugins) #

To simply ensure your compilation system works properly, you can run the build command without any parameters. This command downloads the latest stable release Caddy source code and compiles it into a binary file named caddy in your active directory:

# Run the default compilation
xcaddy build

2. Building a Binary with One Additional Plugin #

The most common scenario is the need to add a DNS challenge provider module (e.g., Cloudflare) so Caddy can request wildcard SSL certificates. You use the --with option followed by the Go plugin repository module name:

# Compile the latest Caddy with the Cloudflare DNS plugin
xcaddy build --with github.com/caddy-dns/cloudflare

# Check the binary file compilation result in the active directory
ls -lh caddy
# -rwxr-xr-x 1 user staff 38M Jun 16 18:47 caddy

# Verify whether the cloudflare module is integrated into the new binary
./caddy list-modules | grep dns.providers.cloudflare
# Confirmation output: dns.providers.cloudflare

3. Building a Binary with Many Plugins at Once #

You’re not limited to just one --with option. You can add as many --with parameters as needed to combine various plugins into a single Caddy binary:

# Compile Caddy with a combination of four popular third-party plugins
xcaddy build \
    --with github.com/caddy-dns/cloudflare \
    --with github.com/mholt/caddy-ratelimit \
    --with github.com/caddyserver/cache-handler \
    --with github.com/greenpau/caddy-security

# Verify the presence of all compiled modules in one command
./caddy list-modules | grep -E "cloudflare|rate_limit|cache|security"

Precise Plugin Version Management #

By default, if you don’t specify a version, xcaddy takes the latest stable release tag version (@latest) of Caddy and each declared plugin. However, in production environments requiring high stability, you must set versions precisely to avoid compilation failures or sudden binary behavior changes due to plugin code updates with breaking changes.

xcaddy supports several flexible version specification syntaxes:

1. Specifying Specific Caddy and Plugin Versions #

You can add semantic tag version markers (@vX.Y.Z) both on the main Caddy version argument and on the plugin repository name arguments:

# Compile Caddy version 2.8.4 with the Cloudflare plugin version 0.0.8
xcaddy build v2.8.4 \
    --with github.com/caddy-dns/[email protected]

2. Compiling Using a Specific Branch or Commit Hash #

If you need the latest feature from a plugin not yet officially released as a tag, you can refer directly to the Git repository’s branch name or commit hash ID:

# Referring to the main branch (main/master)
xcaddy build --with github.com/example/caddy-plugin@main

# Referring to a specific commit hash (12-character SHA-1) for deterministic security
xcaddy build --with github.com/example/caddy-plugin@abc123def456

3. Replacing with Local Source Code (Local Development Module) #

If you’re a Caddy plugin developer writing custom module code on a local computer, you can instruct xcaddy to replace the online repository module path with your local directory path. This lets you test your custom module in real time without uploading the code to GitHub first:

# Format: --with <import_path>=<local_path>
xcaddy build \
    --with github.com/user/my-custom-plugin=/Users/user/Projects/my-custom-plugin

Graceful Binary Upgrade Steps in Production #

One of the biggest operational challenges on production servers is how to update or replace the active web server binary without stopping user request handling services (zero-downtime upgrade).

You’re strictly forbidden from directly overwriting the active Caddy binary file crudely (e.g., overwriting /usr/bin/caddy while the Caddy process is running). This triggers a filesystem text file busy error and risks corrupting your web server process memory.

Here’s a guide of safe steps (graceful upgrade) for replacing the Caddy binary on a Linux production server:

Step 1: Compile the New Binary #

Do the custom binary compilation process using xcaddy in an isolated directory or on your staging machine:

xcaddy build v2.8.4 --with github.com/caddy-dns/cloudflare

Step 2: Validate the Configuration File Using the New Binary #

Before replacing the system binary, you must ensure the new binary you built can read your production Caddyfile configuration file without triggering parsing errors:

# Validate the Caddyfile using the new local custom binary
./caddy validate --config /etc/caddy/Caddyfile
# Make sure the output displays: "Valid configuration"

Step 3: Back Up the Old Server Binary #

Backup is the best defense step. If there’s unusual performance after the upgrade, you can do an instant rollback within seconds:

# Back up the active binary
sudo cp /usr/bin/caddy /usr/bin/caddy.bak

Step 4: Safely Overwrite the Binary File #

Copy the new binary by overwriting the binary in the system directory. To minimize Linux filesystem lock risks, use the move (mv) command or forced copy (cp with the overwrite option):

# Move the new binary to the system bin directory
sudo cp caddy /usr/bin/caddy

# Give the correct execute permissions
sudo chmod +x /usr/bin/caddy

Step 5: Trigger a Graceful Configuration Reload #

Caddy has an advanced port sharing architecture where the new instance can run and bind network socket ports (like ports 80 and 443) in parallel before the old instance closes its connections. This is done through an internal reload signal:

# If using systemd, run the service reload
sudo systemctl reload caddy

# Or trigger a direct reload through the binary
caddy reload --config /etc/caddy/Caddyfile

Systemd sends the SIGUSR1 signal to the old Caddy master process. The old Caddy safely hands over the TCP sockets to the new process in the background, lets existing user connections finish processing (draining connections), then automatically stops itself without dropping a single new user HTTP request.

Docker Multi-Stage Builds with xcaddy #

Installing the Go SDK and xcaddy directly inside production servers or your deployment virtual machine (VM) is often considered less secure and burdens system storage space. The best approach to keep the system clean is using isolated Docker containers through the Multi-Stage Build feature.

The Caddy developer team provides a special builder Docker image named caddy:builder already equipped with a Go compiler installation and a ready-to-use xcaddy tool. You can compose a custom Dockerfile as follows:

# Stage 1: Isolated Compilation Pipeline using the official builder image
FROM caddy:2.8.4-builder AS builder

# Specify the Caddy version and install the needed plugins
RUN xcaddy build \
    --with github.com/caddy-dns/cloudflare \
    --with github.com/mholt/caddy-ratelimit

# Stage 2: Creating the clean, minimal final image
FROM caddy:2.8.4-alpine

# Copy only the custom compiled Caddy binary from Stage 1
# This action overwrites the default Caddy binary in the alpine image
COPY --from=builder /usr/bin/caddy /usr/bin/caddy

# Expose standard web ports
EXPOSE 80 443 2019

# Caddy automatically runs using our new custom binary

With the Dockerfile structure above, the final Docker image result stays small (~40 Megabytes) because the Go compiler and source code dependencies weighing hundreds of megabytes remain permanently in the discarded Stage 1.

Building in CI/CD Pipelines (GitHub Actions) #

To guarantee custom binary reproducibility and ease developer team collaboration, you should automate the Caddy compilation process using a CI/CD pipeline like GitHub Actions. Every time you change the plugin list in your repository configuration, GitHub Actions does an automatic compilation and provides a ready-to-download custom binary file.

Here’s a very reliable, recommended GitHub Actions configuration file .github/workflows/build-caddy.yml:

name: Build Custom Caddy Binary

on:
  push:
    branches: [ main ]
  workflow_dispatch: # Allows manual triggering from the GitHub dashboard

jobs:
  build-caddy:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout Repository
        uses: actions/checkout@v4
      
      - name: Set up Go SDK Environment
        uses: actions/setup-go@v5
        with:
          go-version: '1.22'
          cache: true # Enables automatic caching for Go modules build cache
      
      - name: Install xcaddy Utility
        run: go install github.com/caddyserver/xcaddy/cmd/xcaddy@latest
      
      - name: Compile Caddy with Custom Plugins
        run: |
          xcaddy build latest \
            --with github.com/caddy-dns/cloudflare \
            --with github.com/mholt/caddy-ratelimit \
            --output ./caddy-production          
      
      - name: Verify Compiled Modules
        run: |
          chmod +x ./caddy-production
          ./caddy-production version
          ./caddy-production list-modules | grep -E "cloudflare|rate_limit"          
      
      - name: Upload Binary Artifact
        uses: actions/upload-artifact@v4
        with:
          name: caddy-custom-linux-amd64
          path: ./caddy-production
          retention-days: 7

Cross-Compilation and Private Repository Access #

Caddy and the Go compiler support extraordinary cross-compilation capabilities natively. You can compile custom binaries for different operating systems and hardware architectures directly from your local computer just by defining environment variables (GOOS and GOARCH).

# Create a binary for Linux ARM64 (e.g., for AWS Graviton or Raspberry Pi)
GOOS=linux GOARCH=arm64 xcaddy build --with github.com/caddy-dns/cloudflare

# Create a binary for Windows AMD64 (produces a caddy.exe file)
GOOS=windows GOARCH=amd64 xcaddy build --with github.com/caddy-dns/cloudflare

# Create a binary for macOS Apple Silicon (M1/M2/M3) from a Linux machine
GOOS=darwin GOARCH=arm64 xcaddy build --with github.com/caddy-dns/cloudflare

2. Compiling Using Private Plugin Repositories #

When your company’s internal team develops a special, confidential Caddy plugin stored in a private GitHub or GitLab repository, the xcaddy command fails by default because the Go compiler can’t download that private code without authentication.

To overcome this obstacle, you must configure your local Git credentials and define several Go-specific environment variables:

# 1. Configure Git to change the HTTPS URL scheme to SSH (or use an HTTPS token)
# This forces Git to use your local SSH keys for private download authentication
git config --global url."[email protected]:".insteadOf "https://github.com/"

# 2. Define the GOPRIVATE and GONOSUMCHECK environment variables.
# GOPRIVATE instructs Go not to download private modules through the public Go proxy server.
# GONOSUMCHECK instructs Go not to check private module checksum hashes against the public sum.golang.org database.
export GOPRIVATE="github.com/my-company/*"
export GONOSUMCHECK="github.com/my-company/*"

# 3. Run the compilation as usual
xcaddy build --with github.com/my-company/private-caddy-plugin

Troubleshooting Compilation Problems #

Although xcaddy is designed to simplify the binary building process, you can sometimes encounter technical obstacles due to code dependencies. Here are some common problems with their solutions:

1. Problem: “Module Not Found” or “Go Get Error” #

  • Cause: The plugin import path you entered is invalid, or the plugin repository has been moved/deleted by its creator.
  • Solution: Re-check the plugin repository URL on the creator’s GitHub page. Make sure the module supports the correct Go module version compatibility.

2. Problem: Dependency Resolution Failure (Dependency Conflict) #

  • Cause: A version conflict occurs in Go library packages used simultaneously by Caddy and the plugin. For example, Caddy uses package foo v2.0 while your custom plugin forces foo v1.0 usage.
  • Solution: Pin your custom plugin to an older, compatible tag version, or update the dependency code inside your custom plugin project to align with the Go module version used by the latest Caddy codebase.
  • You can trigger detailed debug mode to trace where the conflict is by defining a debug variable:
    XCADDY_DEBUG=1 xcaddy build --with github.com/example/plugin

xcaddy Compilation Processing Pipeline Diagram #

To make understanding the logical step order done by the xcaddy utility and Go compiler behind the scenes easier, let’s look at the flowchart below:

flowchart TD
    A["1. xcaddy Command Trigger Starts\n(e.g., xcaddy build --with plugin)"] --> B["2. Verify the Local Environment\n(Check the Go SDK installation in PATH)"]
    
    B --> C{"3. Is the Go SDK available?"}
    C -- "No" --> D["Exit with Error:\n'Go compiler not found'"]
    C -- "Yes" --> E["4. Create a Temporary Working Directory\n(Temporary directory in RAM/Disk)"]
    
    E --> F["5. Compose Custom Go Glue Code\n(Importing the Caddy base & registering plugin modules)"]
    F --> G["6. Run Go Modules Dependency Resolution\n(go get & download source code dependencies)"]
    
    G --> H{"7. Did the resolution succeed?"}
    H -- "No" --> I["Exit with Error:\n'Dependency Conflict'"]
    H -- "Yes" --> J["8. Trigger the Go Compiler build\n(Produce an optimized local machine binary)"]
    
    J --> K{"9. Did the compilation succeed?"}
    K -- "No" --> L["Exit with Error:\n'Compilation failed'"]
    K -- "Yes" --> M["10. Move the custom binary file to the active directory"]
    
    M --> N["11. Delete the Temporary Directory\n(Build cache residue cleanup)"]
    N --> O["12. Binary Building Process Complete"]

Summary #

  • Compile-Time Modules: Caddy uses compile-time modules to achieve maximum performance, standalone single-binary portability, and high data type security levels.
  • Main xcaddy Function: xcaddy is the automation bridge for downloading, configuring glue code, and compiling third-party plugins into the Caddy binary.
  • Go SDK Requirement: Development machines must have the Go compiler installed to successfully run local xcaddy build commands.
  • Credential Management: Always pin custom plugin versions (--with [email protected]) in production environments to guarantee long-term binary stability.
  • Zero-Downtime Upgrade: Use the reload signal (systemctl reload caddy) after moving the new binary for a server process transition without client request interruptions.
  • Cloud-Native Cleanliness: Apply Docker Multi-Stage Builds using the caddy:builder base image to assemble custom binaries without polluting the final production image.
  • Cross-Compilation: You can create custom Caddy binaries for Linux ARM64 or Windows platforms instantly by defining the GOOS and GOARCH variables.
  • Private Repository Access: Configure SSH Git URL replacement combined with the GOPRIVATE variable to allow xcaddy to load private plugin code.

← Previous: Plugins & Modules   Next: Popular Plugins →

About | Author | Content Scope | Editorial Policy | Privacy Policy | Disclaimer | Contact