Anything slow or unreliable does not belong on the request path. Sending an email, generating a report, calling a third-party application programming interface (API): each one can take seconds or fail outright. Celery moves that work to a background worker, and RabbitMQ is the broker that carries the jobs.
First principles
A web request has a budget. The user is waiting, the server in front of the app has a timeout, and the web worker handling the request is tied up until it responds. Slow or unreliable work spends that budget on something the user does not need to wait for.
A queue splits the work in two. The web process writes a message and returns. A worker process, possibly on another machine, reads the message later and does the work. Putting a network and a delay between the two has two consequences:
- The message is a copy, made earlier. Anything inside it may be out of date by the time a worker reads it.
- Delivery is at most once or at least once. If the broker forgets a message as soon as a worker receives it, a crash loses the work. If it waits for the worker to confirm, a crash can make the work run twice. You choose which failure you prefer; exactly once is not on offer.
The rules below (pass ids, make tasks idempotent, choose when to acknowledge) all come from those two facts.
What belongs in a task
If a piece of work can be slow, can fail in a way worth retrying, or does not need to finish before the user sees a response, it is a task. Everything else can stay inline.
Wiring Celery to RabbitMQ
# myproject/celery.py
import os
from celery import Celery
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "myproject.settings")
app = Celery("myproject")
app.config_from_object("django.conf:settings", namespace="CELERY")
app.autodiscover_tasks()In settings, CELERY_BROKER_URL points at RabbitMQ. The broker holds the queue of pending jobs and hands each one to whichever worker is free.
A task that can fail safely
from celery import shared_task
@shared_task(bind=True, max_retries=3, default_retry_delay=10)
def sync_order_to_warehouse(self, order_id):
try:
order = Order.objects.get(pk=order_id)
warehouse_client.push(order.as_payload())
except WarehouseTimeout as exc:
raise self.retry(exc=exc)Pass ids, not objects
A task argument is serialized and may sit in the queue for a while. Pass a primary key and load the row fresh inside the task. A serialized model instance is a snapshot that is stale before the worker ever picks it up.
Make every task idempotent
Retries mean a task can run more than once. Design each task so that running it twice does no harm: check whether the work is already done, or key the side effect on a unique id before committing it.
acks_late and the worker that crashes
By default, a job is acknowledged the moment a worker receives it, so a crash mid-task loses the work. Setting acks_late=True holds the acknowledgement until the task finishes, so a crashed task is redelivered to another worker.
Give tasks a time limit
A soft_time_limit raises an exception inside a task that runs too long, so one stuck job cannot hold a worker hostage forever. Pair it with monitoring on queue depth: a queue that only grows is telling you the workers cannot keep up.
The shape this gives you
The request handler does the minimum, fires a task, and returns. The worker does the slow part, retries on failure, and survives a crash. The user never waits for any of it.