Backpressure: Bound Work Before It Becomes a Backlog

Backpressure: let the slow stage control admission

Backpressure is feedback that makes an upstream stage reduce or stop production when a downstream stage cannot keep up. Buffering delays the problem; throttling imposes a rate; load shedding rejects work. They can support backpressure, but a growing queue alone is not a control loop.

Calculate the failure window

Assume a producer sends 1,200 events per second and a worker pool completes 1,000. The backlog grows by 200 events per second. A 10,000-event queue fills in 50 seconds from empty if those rates remain constant.

If the producer later slows to 800 events per second, the same workers have 200 events per second of spare capacity and need another 50 seconds to drain that backlog. Adding a larger queue increases the time before overflow, while also increasing the oldest event's delay. It does not fix sustained overload.

At 1,000 completions per second, 10,000 queued events represent roughly ten seconds of waiting under stable FIFO service. If requests expire after two seconds, much of that queued work may already be useless. Queue age can reveal that problem more clearly than queue depth.

Bound both queued and running work

A queue of 100 items does not bound memory if each item contains an unrestricted upload. Define a byte limit or a maximum item size. Likewise, a bounded input queue does not help when each consumer immediately starts an unlimited number of asynchronous operations.

Keep a concurrency limit on active work. Return capacity only when the relevant operation finishes, not merely when it is removed from the queue. For a pipeline, every boundary needs a compatible limit; an unbounded buffer at one stage can hide downstream slowness from all previous stages.

A complete local demonstration

This Python example starts two consumers and a queue with four waiting slots. await queue.put(...) suspends the producer when the queue is full. It does not create a separate pending task for every item, so admission remains bounded.

import asyncio

async def consume(queue):
    while True:
        item = await queue.get()
        try:
            if item is None:
                return
            await asyncio.sleep(0.01)  # Simulated bounded work
            print(f"processed {item}")
        finally:
            queue.task_done()

async def main():
    queue = asyncio.Queue(maxsize=4)
    workers = [asyncio.create_task(consume(queue)) for _ in range(2)]
    for item in range(12):
        await queue.put(item)
    for _ in workers:
        await queue.put(None)
    await queue.join()
    await asyncio.gather(*workers)

asyncio.run(main())

Python documents that a positive maxsize causes put() to wait when full. This is a single-process demonstration, not a durable job queue. Real processing errors need a policy for cancellation, retry, dead-letter handling, and shutting down workers; calling task_done() is bookkeeping, not proof of business success. Python asyncio queues.

Across a network, demand must be visible

A sender needs a signal it can obey: credits, an acknowledgment window, a paused read, or an explicit overloaded response. Reactive Streams specifies non-blocking backpressure through bounded demand; a generic observable API does not automatically implement that contract. Reactive Streams specification overview.

In Node.js, a writable stream's write() returning false asks the producer to wait for drain. Ignoring that result can accumulate buffered data. For a supported stream chain, pipeline() handles propagation and errors more safely than a manually assembled sequence of unchecked writes. Node.js stream documentation.

TCP flow control bounds transport buffers, but an application can drain those bytes into an unbounded work queue and defeat the intended protection. Application acknowledgment should represent the capacity or durability promise the sender actually relies on.

Choose an overflow policy from the meaning of the data

A producer that cannot slow down forces a choice among buffering, shedding, and adding enough capacity. An infinite memory queue is not a fourth option.

Test a slow dependency, a stopped consumer, and a recovery burst. Observe queued bytes, oldest age, active work, rejections, and completion rate. Also check that retries respect the same admission limit; otherwise every rejected request can return with several friends and turn a controlled overload into a larger one.