What is rate limiting? Controlling how often clients can call
By Unlearnt
Rate limiting caps how many requests a client can make to a service in a given time. It protects the service from overload and abuse, shares capacity fairly between clients, and controls cost. When a client exceeds its limit, a well-behaved API returns a 429 Too Many Requests response rather than failing silently.
Why cap requests
A service has finite capacity. Without a limit, one client, whether malicious, buggy or simply popular, can consume it all and degrade the service for everyone. Rate limiting puts a ceiling on each client so no single one can crowd out the rest.
It also protects downstream resources and cost: fewer wasted calls to databases or paid third-party APIs. The standard signal for a limited client is HTTP 429, often with a Retry-After header telling the client when to try again.
A worked example: the token bucket
A common method is the token bucket. Imagine a bucket that holds up to 10 tokens and refills one token per second. Each request spends one token; if the bucket is empty, the request is rejected with a 429 or must wait.
A client can send a burst of up to 10 requests at once, because the bucket starts full. After that it is limited to roughly one request per second, the refill rate. So the bucket allows short bursts while enforcing a steady long-run rate.
Allow short bursts, then a steady rate.
Burst of 10 requests
Bucket starts full, 10 tokens
All 10 allowed, bucket now empty
11th request, immediately
No tokens left
Rejected with 429, or waits
After 1 second
Refills 1 token per second
1 more request allowed
Algorithms differ
Different limiters make different trade-offs between simplicity, burst tolerance and smoothness.
| Method | How it works | Trade-off |
|---|---|---|
| Fixed window | Count requests per fixed interval, reset at the boundary. | Simple, but allows a spike across the boundary of two windows. |
| Sliding window | Count over a moving interval ending now. | Smoother than fixed, slightly more to track. |
| Token bucket | Spend a token per request; refill at a set rate. | Allows bursts up to the bucket size, with a steady long-run rate. |
Misconception: rate limiting is the same as blocking
A limit is not a ban. A rate-limited client is told to slow down, usually with a 429 and a time to retry, and can continue once its allowance refills. Blocking removes access entirely. Rate limiting shapes traffic rather than cutting it off.
It is related to backpressure, but not identical: backpressure signals a producer to slow when a system is overloaded, while rate limiting enforces a fixed per-client budget whether or not the system is currently busy. Many services use both.
Check your understanding
Optional: apply the idea to a new scenario.
An API uses a token bucket that holds 10 tokens and refills 1 token per second. A client sends 10 requests instantly, then keeps sending 2 requests every second. What happens to the client's requests over the first few seconds, and how does this behave differently from a fixed limit of 60 requests per minute?
Answer with the tutor →Free to start, no account needed.