API Rate Limiting: Implementation Strategies and Best Practices

API Rate Limiting: Implementation Strategies and Best Practices

By TCAL — Tech, Coding & AI Learning

Disclosure: This article may contain affiliate links. We only recommend products we believe in. See how we make money.

Without a cap, one buggy loop or one unfriendly client can spend the CPU, the database, and the LLM bill for everyone else. Rate limiting is how you keep a public API boring: a known budget per key, a clear rejection, and a time when it is legal to try again.

This is an implementation guide, not a vendor bake-off. The same patterns apply to your own routes and to the upstream APIs you call. For typing the request/response at the boundary, see TypeScript beyond the basics. For where the API actually runs, see web hosting for developers.

What you are actually limiting

Name the dimension before you pick an algorithm:

  • Identity: API key, user id, IP. IP is a last resort (NAT lies). Prefer the authenticated key.
  • Unit: requests, tokens, bytes, expensive jobs. Token-based LLM APIs fail when you cap only RPM and ignore TPM — the Claude API notes call this out.
  • Window: per second for abuse, per minute for APIs, per day for free tiers.
  • Scope: one route vs the whole service. A /export job should not share a bucket with /health.

A limit you cannot explain to a customer (“why was I blocked?”) will be turned off the first time it pages someone.

Algorithms

Fixed window

Count requests in 12:00:00–12:00:59, reset at the wall-clock boundary.

Good: trivial. Bad: two bursts, one at :59 and one at :00, get you almost 2 × limit in two seconds. Fine for rough quotas. Poor as the only shield against a spike.

Sliding window log

Store a timestamp per request. Drop entries older than now - window. Reject when the remaining count is at the limit. Smooth. Memory grows with request rate — a 10k req/s key with a 60s window is a lot of sorted-set entries.

Sliding window counter (approximation)

Weight the previous window by how much of it still overlaps. Cheap, almost as smooth as a log, slightly sloppy at the edges. Many gateways ship this.

Token bucket

A bucket holds up to capacity tokens and refills at rate tokens per second. Each request spends one token. Empty bucket → reject. This is the algorithm that allows a burst (up to capacity) while enforcing a long-run average. Official write-ups of AWS API Gateway and many CDNs describe a variant of this.

Use a token bucket when a human clicking around should feel instant, but a script should not run at 200 rps forever.

Use a window when the product promise is “100 requests per minute, period.”

Redis: do not INCR then hope

The following pattern is a race: GET, then SET/INCR in the application. Two replicas can both see 99 and both allow. Also, INCR without an expire on first write can leave a key that never dies if the process crashes between INCR and EXPIRE.

Run the increment and the expire in Redis, as one script. This is the usual production shape for a fixed window:

import time

# EVAL this script; KEYS[1] = rate_limit:<id>:<window>
# ARGV[1] = max_requests, ARGV[2] = window_seconds
SCRIPT = """
local n = redis.call("INCR", KEYS[1])
if n == 1 then
  redis.call("EXPIRE", KEYS[1], ARGV[2])
end
return n
"""

def allow(r, client_id: str, max_requests: int = 100, window: int = 60) -> tuple[bool, int]:
    window_id = int(time.time()) // window
    key = f"rl:{client_id}:{window_id}"
    n = r.eval(SCRIPT, 1, key, max_requests, window)
    return int(n) <= max_requests, int(n)

Sliding-window log with a sorted set (official Redis pattern: ZREMRANGEBYSCORE, ZCARD, ZADD, EXPIRE in a pipeline or Lua):

LOG_SCRIPT = """
local key = KEYS[1]
local now = tonumber(ARGV[1])
local window = tonumber(ARGV[2])
local max = tonumber(ARGV[3])
redis.call("ZREMRANGEBYSCORE", key, 0, now - window)
local count = redis.call("ZCARD", key)
if count >= max then
  return {0, count}
end
redis.call("ZADD", key, now, now)
redis.call("EXPIRE", key, window)
return {1, count + 1}
"""

Token bucket is the same idea with a stored tokens and last_refill hash, updated in Lua so two replicas cannot both pour from an empty bucket.

Local counters are not “good enough for now” once you have two processes. Four workers each allowing 100/min is 400/min. If you are on a single VPS, a process-local bucket can be a temporary shield. The moment you add a second replica or a serverless isolate, move the count to Redis, Durable Objects, or the platform’s rate-limit product.

What the client should see

RFC 9110 registers 429 Too Many Requests via the IANA status-code registry; RFC 6585 defined it. Retry-After is the signal a well-behaved client needs. IETF’s RateLimit header fields are the current standardization effort; many APIs still send the older X-RateLimit-* names. Send both if you have clients of mixed age:

HTTP/1.1 429 Too Many Requests
Retry-After: 30
RateLimit: limit=100, remaining=0, reset=30
X-RateLimit-Limit: 100
X-RateLimit-Remaining: 0
X-RateLimit-Reset: 1710000030

Retry-After may be delta-seconds or an HTTP-date. Pick one and document it. Clients that ignore it will retry immediately; that is their bug, but your origin still feels it — which is why an edge limiter in front of Cloudflare Pages / Workers or a VPS is worth having.

Body content is optional. A small JSON { "error": "rate_limited", "retry_after": 30 } is enough. Do not return a stack trace.

Tiers, writes, and expensive routes

Separate buckets:

KeyExample
rl:{id}:readGET /v1/*
rl:{id}:writePOST/PATCH/DELETE
rl:{id}:expensiveexports, embeddings, agent runs

A “Pro” tier is just different max values on the same scripts, not a different algorithm. Free vs paid is a product decision; the code is a lookup from plan → {limit, window}.

For LLM features, cap tokens as well as requests. A single 200k-token prompt is one request and a large bill. The security notes on AI-generated endpoints pair well here: an unauthenticated demo route needs a tighter bucket than a billed key.

Fail-open vs fail-closed

When Redis times out:

  • Fail-closed: return 429 or 503. The origin stays up. Paying customers get blocked during an infra blip.
  • Fail-open: allow the request. Availability wins; a Redis outage becomes an unpaid stampede.

There is no universal right answer. A checkout API often fails closed. A marketing site API often fails open. Log the fallback either way. Do not crash the request on a ConnectionError and call that a limiter.

When not to build this yourself

  • You are already behind an API gateway or CDN that offers per-key limits. Use that, then add an app-level bucket only for authenticated tiers.
  • The surface is a static site. You do not need Redis to serve HTML.
  • The only client is your own CI. A concurrency: 1 job is cheaper than a limiter.

If you do build it, ship the headers and the Lua (or the gateway config) before you ship the blog post that announces a public API.