Fix Certbot SSL Auto-Renewal Failure on Nginx: ACME Challenge Guide

Certbot auto-renewals usually fail silently until you open your site one morning and get hit with an ERR_CERT_DATE_INVALID error in production. Nine times out of ten, the automated renewal cron job choked on an HTTP-01 challenge because Let’s Encrypt couldn’t reach the temporary validation token inside /.well-known/acme-challenge/.

Here is why Nginx blocks or misroutes those ACME challenge files, how to debug the exact failure code from Certbot dry-runs, and how I set up an isolated webroot so renewals never break again during app updates.

Why HTTP-01 Validation Fails in Nginx

Let’s Encrypt uses the ACME protocol for automated verification. When Certbot triggers an HTTP-01 challenge renewal, Let’s Encrypt issues a token. Certbot writes that token to a local directory on your server, and Let’s Encrypt fires a plain HTTP GET request to http://yourdomain.com/.well-known/acme-challenge/{token} to verify you own the box.

If anything gets in the way of that HTTP request, the challenge fails and you don’t get a certificate. In Nginx, this usually happens for a few specific reasons:

• Reverse proxy passthrough: A catch-all location / block forwards everything to an upstream app (Node, Go, Python, PHP-FPM, Docker) that has no clue what the ACME challenge file is.
• Aggressive HTTPS redirects: Your port 80 block redirects requests in a way that strips URI paths or forces an early HTTPS redirect before the challenge can complete.
• Document root mismatches: The webroot path inside /etc/letsencrypt/renewal/yourdomain.conf points to a different folder than the root directive in your active Nginx server block.
• Filesystem permissions: Nginx runs under www-data or nginx and doesn’t have read access to the directory Certbot dropped the token into.

Before tweaking configs at random, run a dry run to see the exact HTTP status code Let’s Encrypt is getting.

Reproducing the Failure with Certbot Dry-Run

Don’t wait for your production cert to expire to test this. Certbot has a staging environment that lets you test renewals repeatedly without hitting Let’s Encrypt rate limits.

Run a dry run from your server terminal:

sudo certbot renew --dry-run

When the renewal fails, Certbot spits out an error block with the domain, challenge type, and the server’s response:

Certbot failed to authenticate some domains (authenticator: nginx). The Certificate Authority reported these problems:
Domain: api.example.com
Type: unauthorized
Detail: 203.0.113.10: Invalid response from http://api.example.com/.well-known/acme-challenge/8fJk2...: 404
Hint: The Certificate Authority failed to verify the temporary challenge files created by Certbot. Ensure the listed domains point to this server and that it can be accessed from the internet.

Pay close attention to the status code in the Detail line:

• 404 Not Found: Nginx answered the request, but looked in the wrong folder or passed it upstream.
• 403 Forbidden: File permission issue or Nginx access restriction.
• Connection Refused / Timeout: Firewall rules or DNS records pointing to the wrong IP. If you changed DNS recently, check our guide on how to point a subdomain to a separate server with Nginx to verify your records.

Fixing Catch-All Reverse Proxies and SPA Rewrites

This is where I see renewals break most often: servers acting as reverse proxies or running Single Page Applications. If your Nginx block forwards all traffic to a backend socket or rewrites every URI to index.html, Nginx sends the challenge request straight to your app, which throws a 404.

To fix this, add a high-priority location block in your port 80 (HTTP) server block that catches /.well-known/acme-challenge/ before any proxy or rewrite rule runs.

Open your site’s Nginx configuration:

sudo nano /etc/nginx/sites-available/example.com

Update your HTTP server block like this:

server { listen 80; listen [::]:80; server_name example.com www.example.com; # Dedicated rule for ACME challenges location ^~ /.well-known/acme-challenge/ { default_type "text/plain"; root /var/www/letsencrypt; allow all; } # Redirect all other HTTP traffic to HTTPS location / { return 301 https://$host$request_uri; }
}

The ^~ modifier is essential here. It tells Nginx: if this prefix matches, stop checking other regexes and location rules immediately. That prevents SPA rewrites and proxy passes from hijacking the ACME token request.

Resolving File Permissions and Document Root Mismatches

Once you’ve mapped a path like /var/www/letsencrypt, you need to create it and make sure Nginx can actually read it.

Create the directory and assign ownership to your web server user:

sudo mkdir -p /var/www/letsencrypt/.well-known/acme-challenge
sudo chown -R www-data:www-data /var/www/letsencrypt
sudo chmod -R 755 /var/www/letsencrypt

Before running Certbot again, manually test whether Nginx serves static files from this directory. Drop a dummy token into the folder:

echo "test-renewal-token-12345" | sudo tee /var/www/letsencrypt/.well-known/acme-challenge/test-check

Test your Nginx syntax and reload:

sudo nginx -t && sudo systemctl reload nginx

Send an HTTP GET request with curl from your local machine to confirm the file resolves without redirect loops or 404s:

curl -i http://example.com/.well-known/acme-challenge/test-check

You should see HTTP/1.1 200 OK with test-renewal-token-12345 in the body. Once that works, delete the test file:

sudo rm /var/www/letsencrypt/.well-known/acme-challenge/test-check

Handling Cloudflare and HTTPS Redirection Loops

If your domain sits behind Cloudflare or another CDN, edge proxy settings can mess with HTTP-01 validation. If your CDN forces SSL before reaching your origin server, Let’s Encrypt can get caught in redirect loops or 520 errors.

If you’re stuck in a loop behind a CDN, our guide on fixing SSL ERR_TOO_MANY_REDIRECTS on Nginx covers aligning your SSL handshake modes. Per the Let’s Encrypt Challenge Documentation, HTTP-01 challenges start on port 80 over plain HTTP and will follow 301/302 redirects to port 443, as long as the target returns a valid cert or serves the challenge over HTTP.

If you can’t keep port 80 open (e.g., internal staging servers), switch to DNS-01 validation. See our tutorial on setting up Let’s Encrypt wildcard SSL with Certbot DNS-01 on Nginx to validate via DNS TXT records instead.

Building a Universal ACME Snippet for Nginx

If you manage multiple virtual hosts, pasting the same location block into every config file gets messy fast. I use a shared snippet across all my servers instead.

Create a shared snippet file:

sudo nano /etc/nginx/snippets/letsencrypt-acme.conf

Add the ACME location block:

location ^~ /.well-known/acme-challenge/ { default_type "text/plain"; root /var/www/letsencrypt; allow all;
}

Now you can include this one-liner in any virtual host config under /etc/nginx/sites-available/:

server { listen 80; listen [::]:80; server_name myapp.example.com; include /etc/nginx/snippets/letsencrypt-acme.conf; location / { return 301 https://$host$request_uri; }
}

If you ever need custom or commercial certs alongside ACME, check out our guide on how to install custom SSL on Nginx and combine CRT bundles.

Updating Certbot Renewal Configurations

Certbot stores parameters for each cert in /etc/letsencrypt/renewal/. If you originally generated certs with the --nginx plugin and renewals keep breaking after config edits, switching to the webroot authenticator makes renewals much more resilient.

Open your domain’s renewal config file:

sudo nano /etc/letsencrypt/renewal/example.com.conf

Change the [renewalparams] section to use the webroot plugin and point it to the shared directory:

# Options used in the renewal process
[renewalparams]
authenticator = webroot
webroot_path = /var/www/letsencrypt,
server = https://acme-v02.api.letsencrypt.org/directory
key_type = ecdsa
[[webroot_map]]
example.com = /var/www/letsencrypt
www.example.com = /var/www/letsencrypt

Save it and run your dry run again:

sudo certbot renew --dry-run

You should get a clean bill of health:

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Processing /etc/letsencrypt/renewal/example.com.conf
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Simulating renewal of an existing certificate for example.com and www.example.com
The dry run was successful.

Automating Renewal Hooks and Systemd Timers

When Certbot renews a certificate in the background, Nginx keeps the old certificate loaded in memory until you reload the service. If you don’t configure an automatic reload hook, your cert will renew on disk while your site serves an expired cert to visitors.

Per the Certbot Documentation, you can drop a deploy hook script directly into Certbot’s renewal directory:

sudo nano /etc/letsencrypt/renewal-hooks/deploy/reload-nginx.sh

Add this script:

#!/bin/sh
nginx -t && systemctl reload nginx

Make it executable:

sudo chmod +x /etc/letsencrypt/renewal-hooks/deploy/reload-nginx.sh

Finally, double-check that your systemd timer is active and scheduled to run Certbot automatically:

sudo systemctl status certbot.timer

If you run into issues with worker process reloading, the Nginx Official Documentation details process signal handling.

Frequently Asked Questions

Why does Certbot fail with 404 even when port 80 is open?

An open port 80 just means Nginx answered. If your configuration passes traffic to a backend app via proxy_pass or rewrites requests for an SPA, the challenge request gets sent to your application rather than the directory where Certbot placed the temporary token.

Can I renew Let’s Encrypt certificates if port 80 is blocked by my ISP?

No, HTTP-01 challenges strictly require port 80. If port 80 is blocked, you’ll need to switch to DNS-01 challenges, which verify domain control using DNS TXT records instead of HTTP requests.

What is the difference between certbot –nginx and certbot –webroot?

The --nginx plugin tries to edit your live Nginx configs during validation and reverts them after. If your configs use custom includes, regex blocks, or non-standard structures, it can choke. The --webroot plugin simply drops files into a folder on disk and leaves your Nginx configuration alone.

How do I test my renewal hook without renewing the actual certificate?

Run sudo certbot renew --dry-run --run-deploy-hooks. This simulates renewal and executes all scripts in /etc/letsencrypt/renewal-hooks/deploy/ so you can confirm Nginx reloads cleanly without errors.