Skip to main content

Caddy Systemd Service Restarts Forever: caddy run vs caddy start

Published: September 6, 2023 • Updated: May 25, 2026 Larry Qu 7 min read
Table of Contents

The Problem

If your Caddy service keeps restarting in a loop with no error logs, the cause is almost certainly using caddy start instead of caddy run in your systemd unit file.

Symptoms:

  • systemctl status caddy shows the service cycling through active → activating → active
  • journalctl -u caddy -f shows repeated start messages but no errors
  • The web server works briefly, then restarts again

Root Cause: start vs run

Caddy has two commands for starting the server:

caddy run    # Starts Caddy and BLOCKS (foreground process)
caddy start  # Starts Caddy in the BACKGROUND and returns immediately

Systemd expects ExecStart to run a foreground process that stays alive. When you use caddy start, the command launches Caddy in the background and then exits immediately with code 0. Systemd sees the process exit and — because Restart=always is set — restarts it. This creates an infinite loop.

systemd Service Anatomy

Understanding the three sections of a systemd service file helps prevent this and other configuration errors.

[Unit] Section

The [Unit] section describes the service and declares dependencies and ordering:

[Unit]
Description=Caddy Web Server
Documentation=https://caddyserver.com/docs/
After=network.target network-online.target
Requires=network-online.target
  • Description: Human-readable name shown in systemctl status.
  • Documentation: URL(s) for the service documentation.
  • After: Ordering directive — start this service after the listed units. Does not imply a dependency.
  • Requires: Strong dependency — if the listed unit fails, this unit fails too.
  • Wants: Weak dependency — the listed unit is started if possible, but its failure does not affect this unit.
  • BindsTo: Even stronger than Requires — if the bound unit stops, this unit stops.

[Service] Section

The [Service] section defines how the service is executed and managed:

[Service]
Type=notify
User=caddy
Group=caddy
ExecStart=/usr/bin/caddy run --environ --config /etc/caddy/Caddyfile
ExecReload=/usr/bin/caddy reload --config /etc/caddy/Caddyfile --force
ExecStop=/bin/kill -SIGQUIT $MAINPID
ExecStartPre=/bin/mkdir -p /var/log/caddy
ExecStopPost=/bin/rm -f /var/run/caddy.pid
TimeoutStopSec=5s
Restart=on-failure
RestartSec=5s
LimitNOFILE=1048576

Key directives:

Directive Purpose
Type= How systemd tracks service readiness (simple, notify, forking, oneshot, idle)
ExecStart= Main process command (required)
ExecReload= Command for systemctl reload (optional)
ExecStop= Command for systemctl stop (optional; default is SIGTERM)
ExecStartPre= Command(s) run before ExecStart; failure prevents start
ExecStopPost= Command(s) run after the service stops (even on crash)
Restart= Restart policy (always, on-failure, on-abnormal, no)
RestartSec= Wait time before restarting
TimeoutStartSec= Max time for ExecStart to complete (default 90s)
TimeoutStopSec= Max time for ExecStop to complete (default 90s)

ExecStart vs ExecStartPre/Post

ExecStart is the main command. ExecStartPre and ExecStartPost run before and after it:

[Service]
ExecStartPre=/bin/sh -c 'test -f /etc/caddy/Caddyfile || exit 1'
ExecStartPre=/bin/mkdir -p /var/log/caddy
ExecStart=/usr/bin/caddy run --config /etc/caddy/Caddyfile
ExecStartPost=/usr/bin/caddy version
ExecStopPost=/usr/local/bin/notify-slack.sh

If any ExecStartPre command exits non-zero, the service does not start.

[Install] Section

The [Install] section defines when the service is enabled:

[Install]
WantedBy=multi-user.target
  • WantedBy=multi-user.target: The service starts in the normal multi-user runlevel (standard for servers).
  • WantedBy=default.target: Used for user services (starts at user login).
  • RequiredBy=: Stronger than WantedBy — the target requires this service.

The Fix

Change caddy start to caddy run in your service file:

# /etc/systemd/system/caddy.service  (BROKEN)
[Unit]
Description=Caddy Web Server
After=network.target

[Service]
User=root
Group=root
Type=simple
Restart=always
RestartSec=15s
WorkingDirectory=/root/caddy
ExecStart=/root/caddy/caddy start   # <-- WRONG: exits immediately

[Install]
WantedBy=multi-user.target

# /etc/systemd/system/caddy.service  (FIXED)
[Unit]
Description=Caddy Web Server
After=network.target

[Service]
User=root
Group=root
Type=simple
Restart=always
RestartSec=15s
WorkingDirectory=/root/caddy
ExecStart=/root/caddy/caddy run     # <-- CORRECT: blocks indefinitely

[Install]
WantedBy=multi-user.target

After editing the file:

sudo systemctl daemon-reload
sudo systemctl restart caddy
sudo systemctl status caddy

Restart Policy Breakdown

Choose the right Restart= value:

Value Restarts On Use Case
no Never One-shot scripts, manual services
on-success Exit code 0 (success) Testing/debugging only
on-failure Non-zero exit, signal, timeout Production services (recommended)
on-abnormal Signal, timeout (not clean exit) Services that should survive crashes only
on-watchdog Watchdog timeout Services with WatchdogSec=
always Any exit, including clean Mission-critical services that must always run

For Caddy, Restart=on-failure is better than Restart=always. If Caddy exits cleanly (e.g., an admin runs caddy stop), on-failure respects the clean exit and does not restart. always would restart it immediately even after a deliberate stop.

A Better Production Caddy Service File

The official Caddy project provides a well-configured service file. Here’s a hardened version:

# /etc/systemd/system/caddy.service
[Unit]
Description=Caddy Web Server
Documentation=https://caddyserver.com/docs/
After=network.target network-online.target
Requires=network-online.target

[Service]
Type=notify
User=caddy
Group=caddy
ExecStart=/usr/bin/caddy run --environ --config /etc/caddy/Caddyfile
ExecReload=/usr/bin/caddy reload --config /etc/caddy/Caddyfile --force
TimeoutStopSec=5s
LimitNOFILE=1048576
PrivateTmp=true
ProtectSystem=full
AmbientCapabilities=CAP_NET_BIND_SERVICE

Restart=on-failure
RestartSec=5s

[Install]
WantedBy=multi-user.target

Key improvements over the minimal version:

Setting Purpose
Type=notify Caddy signals systemd when it’s ready (more reliable than simple)
ExecReload=caddy reload Graceful config reload without downtime
LimitNOFILE=1048576 Raise file descriptor limit for high-traffic sites
PrivateTmp=true Isolate /tmp for security
ProtectSystem=full Read-only access to /usr, /boot, /etc
AmbientCapabilities=CAP_NET_BIND_SERVICE Allow binding to ports 80/443 as non-root
Restart=on-failure Only restart on failure, not on clean exit

Using the Official Package

If you installed Caddy via the official package (apt/rpm), the service file is already correct:

# Install via apt (Debian/Ubuntu)
sudo apt install caddy

# The package includes a correct service file at:
# /lib/systemd/system/caddy.service

sudo systemctl enable caddy
sudo systemctl start caddy

Reloading Config Without Restart

One advantage of caddy run with systemd is zero-downtime config reloads:

# Reload Caddy config without dropping connections
sudo systemctl reload caddy

# Or directly
caddy reload --config /etc/caddy/Caddyfile

This is much better than restart, which briefly drops all connections.

Caddy API for Config Management

Caddy exposes an admin API (default on localhost:2019) for runtime configuration changes without restarting or reloading:

# Load a new configuration
curl -X POST http://localhost:2019/load \
  -H "Content-Type: application/json" \
  -d @new-config.json

# Get the current configuration
curl http://localhost:2019/config/

# Enable or disable a site dynamically
curl -X DELETE http://localhost:2019/config/apps/http/servers/srv0/routes/2

The API is useful for dynamic reconfiguration in containerized or orchestrated environments where you want to avoid systemctl reload entirely.

Caddyfile Validation

Validate the Caddyfile syntax before applying it:

caddy validate --config /etc/caddy/Caddyfile

# Or validate from stdin
caddy validate < Caddyfile

Always validate before reloading. A broken Caddyfile causes the reload to fail, leaving the old (working) configuration in place.

journalctl Log Analysis

Use journalctl to diagnose systemd services:

# Follow live logs
journalctl -u caddy -f

# View last 50 lines of logs
journalctl -u caddy -n 50

# View logs since last boot
journalctl -u caddy -b

# View logs from the previous boot (for crash analysis)
journalctl -u caddy -b -1

# View logs since a specific time
journalctl -u caddy --since "1 hour ago"

# Show only errors and warnings
journalctl -u caddy -p err -b

# Follow all failed services
journalctl -p err -f

# Export JSON for programmatic analysis
journalctl -u caddy -o json --since "2026-05-20"

Key log patterns to look for:

# Normal startup (expected)
May 25 10:00:01 host caddy[1234]: {"level":"info","msg":"serving","server":"srv0"}

# Restart loop (indicates problem)
May 25 10:00:01 host caddy[1234]: ...starting...
May 25 10:00:02 host systemd[1]: caddy.service: Main process exited, code=exited
May 25 10:00:02 host systemd[1]: caddy.service: Failed with result 'exit-code'.
May 25 10:00:07 host systemd[1]: caddy.service: Scheduled restart job, restart counter 42

The restart counter in the last line shows how many times the service has been restarted. A growing counter with no TLS certificates or “serving” messages indicates a configuration problem, not a start/run mismatch.

Common systemd Pitfalls

1. daemon-reload After Every Change

systemd caches unit files. Always run systemctl daemon-reload after editing any file in /etc/systemd/system/, or the old configuration remains in effect:

sudo nano /etc/systemd/system/caddy.service
sudo systemctl daemon-reload
sudo systemctl restart caddy

2. Missing Absolute Path

ExecStart= requires an absolute path. If Caddy is installed in a non-standard location, use the full path:

which caddy
# /usr/local/bin/caddy

ExecStart=/usr/local/bin/caddy run

3. Wrong Type for Backgrounding Programs

If you must run a program that backgrounds itself, use Type=forking with PIDFile=:

[Service]
Type=forking
PIDFile=/var/run/caddy.pid
ExecStart=/usr/local/bin/caddy start
ExecStop=/bin/kill -SIGQUIT $MAINPID

But for Caddy, Type=notify is always preferred.

4. Environment Variables Not Set

systemd does not inherit the user’s shell environment. If Caddy needs specific environment variables, use Environment= or EnvironmentFile=:

[Service]
Environment=CADDY_ADMIN=0.0.0.0:2019
EnvironmentFile=-/etc/caddy/env

The - prefix means “ignore if the file does not exist.”

5. Timeout During Shutdown

If Caddy takes longer than TimeoutStopSec to shut down, systemd kills it with SIGKILL, which may corrupt in-flight requests:

[Service]
TimeoutStopSec=10s

Increase this for servers doing long-running upstream requests.

Debugging Systemd Services

Useful commands when troubleshooting any systemd service:

# View service status
systemctl status caddy

# Follow live logs
journalctl -u caddy -f

# View last 50 lines of logs
journalctl -u caddy -n 50

# View logs since last boot
journalctl -u caddy -b

# Check if service is enabled on boot
systemctl is-enabled caddy

# List all failed services
systemctl --failed

# Show full service file (including overrides)
systemctl cat caddy

# Show service dependencies
systemctl list-dependencies caddy

# Verify service file syntax
systemd-analyze verify /etc/systemd/system/caddy.service

# Show why a service failed
systemctl status caddy --full

# Reset the restart counter (after fixing the issue)
systemctl reset-failed caddy

Resources

Comments

Share this article

Twitter Bluesky Mastodon LinkedIn Facebook Reddit Hacker News WhatsApp Telegram Email

Scan to read on mobile

👍 Was this article helpful?