What is rate limiting?
Rate limiting restricts requests or work over time according to a policy. An API might allow a signed-in workspace to start a bounded number of report jobs, while applying another limit to unauthenticated traffic. The useful policy names who is counted, what consumes capacity, and what happens when capacity is unavailable.
HTTP 429 means Too Many Requests. RFC 6585 permits a Retry-After header and leaves the identification and counting strategy to the server. A 429 response therefore tells a client that a limit was exceeded; it does not by itself reveal a fixed window, token bucket, or other algorithm.
A rate limit can reduce a particular source of load or abuse, but it does not establish permission to use an endpoint. Authenticate and authorize the action separately. Also consider expensive accepted work already running: limiting starts per second does not necessarily cap concurrent jobs.
Download a deterministic local lab
The JavaScript download demonstrates a small in-memory token bucket with a caller-supplied clock. Run node rate-limiting-lab.mjs with Node.js 22 or later to print a sequence of synthetic decisions. It sends no requests, writes no files, and changes no service configuration.
The companion worksheet describes the example policy, a blank policy form, client behavior, and the questions needed before using a limiter in a real service. The lab exports createTokenBucket so you can inspect the behavior in a separate local experiment.
This is teaching code for one process with a bounded number of synthetic keys. It is not a distributed limiter, a production gateway, or a defense against every source of resource exhaustion. The clock and identity are explicit inputs so you can see their effect without sending traffic to an API.
Choose the identity and counting rule
For an authenticated SaaS operation, a workspace or account ID derived from trusted authorization context can be more useful than a user-entered header. Allowing callers to choose arbitrary bucket identities lets them evade a per-account rule. Do not treat a submitted account ID as proof of membership.
An IP-based rule can be appropriate at another layer, but shared networks and changing addresses complicate its meaning. Several legitimate users may share an address. A single user may appear under several addresses. Decide what the rule is meant to protect before choosing its identity.
Specify whether all attempts, only matching requests, or a defined response condition count. Cloudflare's rate-limiting documentation describes how counting configuration affects request-rate measurements. Confirm the semantics of the platform actually enforcing your policy rather than assuming two tools interpret the same number identically.
The lab accepts trimmed synthetic keys up to 128 characters and has an explicit maximum number of keys. New keys beyond that bound throw an error. That is a visible classroom limitation, not a recommended production response to account growth.
Work through a token bucket
A token bucket has a capacity and a refill rate. An accepted operation consumes tokens. Unused refill cannot grow the bucket past its capacity, allowing a bounded burst after idle time. This differs from resetting a counter at the start of each fixed calendar window.
The original example starts with three tokens and replenishes one token per second. Each request costs one. Three requests at time zero succeed; the fourth fails. At 999 milliseconds the bucket has 0.999 tokens, so another request still fails. At 1,000 milliseconds one token is available and the request succeeds.
The lab calculates a retry delay from the missing tokens, rounds milliseconds up, and exposes Retry-After seconds rounded up from that delay. This is the example algorithm's suggested wait, not a reservation. Another request can consume the token before a retry arrives.
| Time | Action | Decision |
|---|---|---|
| 0 ms | First three requests | Allowed; three tokens consumed |
| 0 ms | Fourth request | Denied; wait 1,000 ms |
| 999 ms | Next request | Denied; wait 1 ms |
| 1,000 ms | Next request | Allowed |
| 10,000 ms | Inspect through another request | Refill is capped at capacity |
Keep rate, quota, and concurrency separate
A monthly quota limits total consumption over a billing period. A rate limit controls work over shorter time intervals. A concurrency limit caps simultaneous operations. A service can need all three: a workspace may have quota remaining while a short burst or a long-running job still exceeds safe capacity.
Variable-cost operations need an explicit unit. Counting an inexpensive status check as equivalent to a costly generation request can hide the resource you care about. The lab supports positive token costs up to capacity, but the service must decide whether cost can be determined reliably before execution.
Use AI cost controls when the real problem is spend rather than request count alone. A request limiter cannot infer model cost, guarantee a monthly budget, or reverse work that has already been accepted.
Handle 429 responses without amplifying load
A client should recognize the service's documented limit response and inspect a valid Retry-After value when provided. Use bounded retries, account for cancellation and deadlines, and avoid a tight loop. Where many clients may retry together, an appropriate backoff with jitter can reduce synchronization.
Check whether repeating the operation is safe. A timeout after submitting a job can mean the server accepted it even though the client did not receive the result. Use the API's documented idempotency mechanism or inspect status before creating another side effect. A limiter is not a duplicate-request contract.
Keep the failure understandable in the interface. Tell the user which action is waiting and whether it will retry automatically. Do not present a rate-limit rejection as a subscription upgrade requirement unless that is actually the product policy.
Know what must change for production
Separate server processes cannot enforce a global limit with unrelated local maps. A distributed design needs appropriate atomic updates, a consistent time model, key lifecycle management, failure behavior, and observability. Decide how the application behaves if the limiting dependency is unavailable.
The lab rejects invalid numbers, impossible costs, invalid keys, and a clock that moves backward. It tests independent accounts, burst exhaustion, the exact refill boundary, capped refill after idle time, and the configured key bound. Those checks establish the demonstration's behavior, not production scalability or distributed correctness.
Write the policy before selecting an implementation, then test it against realistic authorized traffic in your own environment. Connect the limit to API integration error handling and your operational response. Sources and local example checked October 27, 2026.