Backpressure connects a consumer’s limited capacity to the behavior of its producers. A full channel may make a producer wait, reject new work, or apply a deliberately chosen shedding policy. The important property is that the limit changes what upstream components do. A growing queue or a dashboard showing lag is evidence of imbalance, but neither is automatically a mechanism that controls admission.

When it breaks

An unbounded queue can absorb a temporary burst while hiding sustained overload. If work arrives faster than it finishes, outstanding work and waiting time continue growing. Moving the same backlog into pending threads, retry tasks, or another unbounded buffer does not remove that pressure. The capacity boundary must cover the place where work is actually retained, not merely one convenient queue object.

A bounded queue also does not define a complete overload policy by itself. Blocking forever can consume all producer threads or leave shutdown stuck. Immediate rejection can lose work if callers treat it as success. Retrying every rejection immediately can increase load. Decide whether work should wait within a deadline, be rejected for later retry, or be discarded under an explicit application contract.

Queue capacity and processing completion are separate. Removing an item can free a queue slot while the consumer is still working on that item. A capacity of two waiting items may therefore coexist with additional in-progress work. Likewise, a producer’s successful enqueue says only that the queue accepted the item; it does not establish execution, durable storage, or completion of its downstream side effects.

Check it quickly

Save this as check-backpressure.py and run python3 check-backpressure.py. It uses only Python’s standard library and was executed with Python 3.14.4. Threads coordinate through explicit events and bounded waits, so the example does not depend on a remote broker, database, or an artificially slow external service. All queues and work items exist only inside the process:

from queue import Queue,Full
from threading import Event,Thread
import json

unbounded=Queue()
for item in range(1000):unbounded.put_nowait(item)
assert unbounded.qsize()==1000
channel=Queue(maxsize=2)
channel.put('a');channel.put('b')
try:
    channel.put('deadline-rejected',timeout=.02)
    raise AssertionError('a full bounded queue accepted another item')
except Full:rejected=True
assert channel.qsize()==2
entered,delivered=Event(),Event()
def produce():
    entered.set();channel.put('c',timeout=2);delivered.set()
producer=Thread(target=produce,daemon=True);producer.start();assert entered.wait(1)
assert not delivered.wait(.05) and channel.qsize()==2
first=channel.get(timeout=1);assert first=='a'
assert delivered.wait(1);producer.join(timeout=1);assert not producer.is_alive()
assert channel.qsize()==2
rest=[channel.get(timeout=1),channel.get(timeout=1)];assert rest==['b','c']
assert channel.empty()
completed=Event()
def join():channel.join();completed.set()
waiter=Thread(target=join,daemon=True);waiter.start();assert not completed.wait(.05)
for item in [first,*rest]:channel.task_done()
assert completed.wait(1);waiter.join(timeout=1);assert not waiter.is_alive()
print(json.dumps({'unbounded_backlog':unbounded.qsize(),'bounded_capacity':2,'deadline_rejected':rejected,
    'blocked_producer_resumed_after_get':delivered.is_set(),'delivered_items':[first,*rest],
    'empty_queue_did_not_mean_processing_complete':True,'join_finished_after_task_done':completed.is_set()}))

The first queue has the default unbounded capacity. With no consumer running, a thousand put_nowait calls all succeed and leave a backlog of a thousand items. This is a small behavioral demonstration, not a memory-exhaustion benchmark. It shows that successful admission alone cannot establish downstream progress or that the arrival rate is sustainable.

The second queue has capacity two and starts with items a and b. Another put uses a short timeout and raises Full, leaving the queued items unchanged. The failed item is not secretly enqueued for later execution. This gives the caller a definite admission outcome on which to base retry or rejection handling instead of presenting an unlimited wait as successful delivery.

A producer thread then tries to put c. The test confirms that it remains blocked while both slots are occupied. Removing a opens one slot, the producer completes its put, and b and c remain queued. The program verifies that all three accepted items are retrieved in order. This demonstrates pressure reaching the producer and being released by a concrete capacity change.

After retrieving the remaining items, the queue is empty, but a separate thread waiting in join still has not completed. Only after task_done is called for all three accepted items does join return. Dequeueing freed capacity earlier; task acknowledgment marks the tracked work complete later. The test separates these events so an empty queue cannot be mistaken for completed business processing.

Preserve the admission contract across the pipeline

Use the queue operation’s actual result rather than a check-then-act decision based on qsize or full. Those observations are approximate when other threads can change the queue. The fixture’s events and paused consumers make its assertions controlled observations; a production caller should handle Full or a successful put atomically through the provided operation.

Choose a bound appropriate to retained resources. An item-count limit does not cap memory when items can be arbitrarily large, and a waiting-queue limit does not include every active worker or external request. Include processing concurrency and upstream retry behavior when evaluating overload. A bounded component surrounded by unlimited pending work can still participate in an unbounded system.

Finally, define what completion means. Python task_done updates queue accounting; it is not a durable transaction or proof that an external side effect succeeded. A worker failure, process crash, or retry after an ambiguous response needs the application’s corresponding delivery and idempotency contract. Backpressure controls admission and pace; correctness still depends on handling accepted, rejected, in-progress, and completed work explicitly.

Sources