Guides / Django health check failing: the deploy keeps rolling back
Django health check failing: the deploy keeps rolling back
Your deploy hangs and then rolls back, or the load balancer shows the instance as "unhealthy" and keeps replacing it, although the site works when you open it. The health check is failing for a boring reason. It's usually one of five.
What it means
A load balancer or deploy tool requests a URL from every new instance and only sends traffic if it gets a success response (typically a 200) within a time limit. Anything else, including a 301 redirect, a 400, or no answer, counts as a failure. The health check doesn't behave like a browser: it calls the container directly, over http, by IP address, with no cookies.
Why it happens
- It gets a redirect.
SECURE_SSL_REDIRECTanswers the http check with a301to https (too many redirects), orAPPEND_SLASHredirects/healthto/health/. - It gets a 400. The check calls the instance by IP, which isn't in
ALLOWED_HOSTS(DisallowedHost). - Nothing is listening where it looks. The app binds to
127.0.0.1instead of0.0.0.0, or on a different port than the one the check uses. - It starts too slowly.
migrateandcollectstaticrun in the start command, so the app isn't up before the check gives up. - The path needs login or a database. The check hits
/, which redirects to a login page or fails because the database is slow or down.
The fix
Give the load balancer a dedicated endpoint that is cheap, public and can't redirect. The simplest reliable version is a middleware at the top of MIDDLEWARE, because it answers before Django checks the host or the SSL redirect:
# myproject/middleware.py
from django.http import HttpResponse
class HealthCheckMiddleware:
def __init__(self, get_response):
self.get_response = get_response
def __call__(self, request):
if request.path in ("/healthz", "/healthz/"): # both: no APPEND_SLASH redirect
return HttpResponse("ok", content_type="text/plain")
return self.get_response(request)
# settings.py
MIDDLEWARE = [
"myproject.middleware.HealthCheckMiddleware", # first
"django.middleware.security.SecurityMiddleware",
# ...
]
Point the health check at /healthz/ on the port the app listens on, and make sure the app binds to all interfaces:
gunicorn myproject.wsgi:application --bind 0.0.0.0:${PORT:-8000}
Test the same way the load balancer does, from outside the app:
curl -i http://<container-ip>:8000/healthz/ # expect: HTTP/1.1 200 OK
Making it stick in production
- Keep the health endpoint shallow. If it checks the database, a database blip will make the load balancer kill every healthy instance. Many teams use two endpoints: a shallow one for the load balancer and a deep one for monitoring.
- Run migrations as a separate release step, so the web process starts in seconds (unapplied migrations).
- Set the check's grace period (start period) longer than your slowest startup, and the failure threshold above one.
- Run
python manage.py check --deploybefore each release (deployment checklist).
Or skip it
Django Cloud answers the load balancer's health check for you, lets you set the health check path as a build setting, and only makes a release live once your app has passed it. If it fails, the previous release keeps serving.
pip install djangocloud-cli
djangocloud deploy
Free plans are available for students, AWS Activate startups, open-source, non-profit and research projects. See how Django Cloud compares with other ways to host Django.
Django is a registered trademark of the Django Software Foundation. Django Cloud is not affiliated with or endorsed by it. Found a mistake in this guide? Tell us and we will fix it.