Backpressure explained: why a bigger queue is not enough

The queue holds the difference

A producer creates work; a consumer processes it. They might be an uploader and a file writer, or an event source and a database worker. A buffer lets them run at different speeds for a while. Backpressure communicates when the receiving side cannot comfortably accept more.

This is why asynchronous does not mean unlimited. Reactive Streams explicitly connects demand signalling with bounded buffering. Without a boundary, work still waits somewhere: in memory, on disk, in a broker, or at the sender.

[1]

Worked example: 1,000 jobs arrive, 600 finish

Imagine an empty queue receiving 1,000 jobs per second. Its workers finish 600 per second. Assume constant rates, equally sized jobs, no failures and no discarded work. Every second, another 400 jobs remain unfinished. The arithmetic below is a simplified fluid approximation, not a benchmark.

After one minute, 24,000 jobs are waiting. A newly arriving job behind that backlog waits roughly 24,000 ÷ 600 = 40 seconds before processing, assuming first-in, first-out service and unchanged capacity. Its own processing time comes afterwards. Expanding the buffer postpones a storage limit; it does not reduce this delay.

Slowing to capacity stops growth, but does not clear it

Suppose backpressure reduces arrivals to 600 per second after that first minute. The backlog stops growing, but stays around 24,000: all processing capacity is still occupied by new arrivals. To drain it, arrivals must fall below departures, or processing capacity must rise.

At 400 arrivals and 600 departures per second, the spare capacity is 200. Clearing 24,000 queued jobs then takes about 120 seconds. If arrivals stop entirely, it takes about 40 seconds. This distinction matters when deciding whether a recovered service is actually caught up.

[3]

Buffering, backpressure and rejection solve different parts

Start with the constraint: can the sender pause, can work wait, and is dropping any of it acceptable?

• Buffer a temporary burst when later spare capacity can drain it within the deadline.
• Apply backpressure when the producer can respond to a pause, a demand signal or an explicit retry-later response.
• Bound admission and reject excess work when you cannot safely accept it. This is load shedding; it becomes useful feedback only if callers respond appropriately.
• Increase effective consumer capacity if the bottleneck can scale. Adding workers does not help if they all wait on the same saturated dependency.

[3][1]

Misconception: a buffer limit automatically slows the sender

A limit defines where the system must make a decision. The producer still has to honour the signal. In Node.js writable streams, a false return from write() tells the producer to wait for drain before continuing. Ignoring it defeats the intended flow control.

An external event source may be unable to pause. You then need a deliberate policy for storage, admission, loss or extra capacity. No finite buffer can preserve unlimited incoming work forever when processing stays slower. Repeated immediate retries can make the incoming load worse rather than relieve it.

Rate limiting is related but not identical: a configured request allowance can protect a service without tracking a particular consumer's current capacity. Backpressure responds to downstream constraints. A system can use both.

[2][3]