Rate-Limiting · XB Field Notes

A token bucket smooths the request rate

Wed, 01 Dec 2027 00:00:00 +0000

A token bucket bounds admitted work while allowing accumulated credit to be spent in a burst. It does not automatically space requests evenly, guarantee completed throughput, or cap the number of operations still running. Those are separate scheduling and capacity questions. The bucket’s two parameters describe a budget, not a promise that the service can perform it.

Refill credit, then charge work

Let capacity B be the maximum stored tokens and rate r be tokens added per second. At each arrival, update credit to min(B, previous_credit + r × elapsed). Admit work only if its cost fits, then subtract that cost. Under the strict rule used here, rejected work does not consume credit.

The bucket can refill lazily when requests arrive; a background timer is unnecessary for this calculation. Idle time cannot accumulate unlimited credit because capacity clips the result. Initial credit is a policy choice: the example starts full, allowing an immediate burst, while an empty initialization would behave differently.

A burst allowance is not a peak rate

For this ideal strict bucket, admitted cost over an interval of duration t is at most B + r × t. As the interval grows, the burst contribution becomes relatively smaller. That bound does not mean every one-second window admits at most r requests, nor does it establish a separate instantaneous peak-rate ceiling.

For capacity three and rate two, three unit-cost requests can pass at the same timestamp. A fourth is denied. Half a second later one token is available. A real scheduler, bounded queue, or additional pacing rule is needed if permitted work must depart at evenly spaced times.

Rate limiting admission also does not bound concurrent execution: slow requests can remain active while more tokens refill. Protect scarce worker or database capacity with an appropriate concurrency limit and timeout policy. Likewise, an accepted request may still fail downstream, so refill rate is not guaranteed successful throughput.

Match the budget to its scope

A token need not mean one request. Weighted costs can represent bytes or estimated work, provided refill rate and capacity use the same units. A request costing more than B can never fit this strict bucket, even after a long idle period. Handle that case explicitly instead of promising an eventual retry time.

Choose the identity and ownership of the budget carefully. Four independent full buckets each holding three tokens can admit twelve requests at once. They are not one global three-token bucket. A shared limiter must make refill, comparison, and deduction an atomic decision; merely sharing a counter without coordinating those steps is insufficient.

Use an elapsed-time clock for local refill calculations. Wall-clock adjustments must not mint credit or move the accounting backward. Python’s monotonic clock is designed for elapsed differences; its reference point is not a portable timestamp to exchange between unrelated machines. Distributed clock and state ownership need their own contract.

When it breaks

A full bucket can release a burst that overwhelms a fragile dependency even when the long-run allowance looks conservative. Unbounded waiting queues turn excess demand into memory growth and latency. Denial and delayed shaping are different outcomes, so document which one callers receive.

Observe admitted and denied cost, queueing, concurrency, and downstream latency together. Retries should respect the limiting policy instead of immediately recreating rejected load. Charging attempted work versus completed work is another explicit choice; refunding failures changes the model and requires careful accounting.

Managed implementations can provide weaker ceilings than the ideal algorithm. Amazon API Gateway documents throttling and quotas as best-effort targets that can be exceeded. Do not present the mathematical bound as a verified guarantee for that service, a billing cap, or any other untested deployment.

Check it quickly

Save the block as bucket_check.py and run python3 bucket_check.py with Python 3.10 or newer. It ran with Python 3.14.4. The standard-library Fraction type and injected nondecreasing timestamps make boundaries exact without sleeping. This is a single-threaded admission model, not a network limiter or distributed store.

from fractions import Fraction


class Bucket:
 def __init__(self, capacity, rate):
 assert capacity > 0 and rate > 0
 self.capacity, self.rate = capacity, rate
 self.tokens, self.updated = Fraction(capacity), Fraction(0)

 def take(self, now, cost=1):
 assert now >= self.updated and cost > 0
 self.tokens = min(self.capacity, self.tokens + (now - self.updated) * self.rate)
 self.updated = now
 if self.tokens < cost:
 return False
 self.tokens -= cost
 return True


bucket = Bucket(capacity=3, rate=2)
events = ([(Fraction(0), 1)] * 4 + [(Fraction(1, 4), 1), (Fraction(1, 2), 1),
 (Fraction(3, 2), 2)] + [(Fraction(100), 1)] * 4)
results = [(now, cost, bucket.take(now, cost)) for now, cost in events]
assert [allowed for _, _, allowed in results] == [
 True, True, True, False, False, True, True, True, True, True, False,
]
for now, cost, allowed in results:
 print(f"t={str(now):>3} cost={cost} {'allow' if allowed else 'deny'}")
assert bucket.tokens == 0
marks = sorted({now for now, _ in events})
for start in marks:
 for end in marks:
 if end >= start:
 consumed = sum(cost for now, cost, allowed in results if allowed and start <= now <= end)
 assert consumed <= bucket.capacity + bucket.rate * (end - start)
print("every event-bounded interval obeys admitted cost <= B + r * elapsed")

assert not bucket.take(Fraction(200), cost=4)
assert bucket.tokens == 3
assert bucket.take(Fraction(200), cost=3)
print("cost above capacity never fits; rejection does not consume tokens")

workers = [Bucket(capacity=3, rate=2) for _ in range(4)]
local_admitted = sum(worker.take(Fraction(0)) for worker in workers for _ in range(4))
shared = Bucket(capacity=3, rate=2)
shared_admitted = sum(shared.take(Fraction(0)) for _ in range(16))
assert local_admitted == 12 and shared_admitted == 3
print("four independent full buckets admit", local_admitted, "at once; one shared bucket admits", shared_admitted)

The trace allows three simultaneous requests, rejects the fourth and the quarter-second request, then allows one at half a second and cost two at one-and-a-half seconds. After long idle time it permits only another three-token burst. Assertions check every interval bounded by event timestamps against the cost envelope.

An oversized request is denied without spending credit. Finally, four independent buckets admit twelve simultaneous requests while one shared sequential model admits three. No concurrency races, real clock adjustments, queue scheduling, cloud throttling, network partitions, or production performance are tested.

Sources

RFC 3290, Appendix A’s token-bucket envelope and strict conformance calculation, checked 2026-09-08.
Amazon API Gateway throttling, request budgets, bursts, and best-effort limits, checked 2026-09-08.
Python time, monotonic elapsed-time clocks and their reference-point limits, checked 2026-09-08.

Retry-After tells when to try again

Tue, 04 May 2027 00:00:00 +0000

Retry-After tells a client how long it ought to wait before a follow-up request. It is a scheduling signal, not proof that the next attempt will succeed or that replaying the original operation is safe. A client still needs a deadline, an attempt policy, and a decision about which operations it can retry.

Interpret the value and its context

RFC 9110 permits either a nonnegative decimal integer number of seconds or an HTTP date. Delay-seconds are measured after receiving the response, not after sending the original request. An HTTP date names an absolute UTC time, so interpreting it depends on the client’s wall clock being reasonably accurate.

Senders generate the preferred IMF-fixdate form. Recipients parsing HTTP dates must accept the two obsolete forms as well; the two-digit year in the old RFC 850 form has a fifty-year interpretation rule. Negative numbers, fractional seconds, and arbitrary words are not valid delay-seconds. Zero and a date already in the past do not require additional server-directed waiting.

With 503, the field estimates how long the service is expected to be unavailable. With a 3xx response, it asks for a minimum wait before the redirected request. RFC 6585 separately permits Retry-After on 429 responses. It is optional there, not a header every throttling response is required to contain.

Respect the wait without waiting forever

A valid wait can exceed the caller’s remaining deadline. The client can stop retrying and report failure or arrange a separate authorized workflow. It should not shorten a server-requested wait and retry early merely to fit a convenient local cap. A long value is not automatically malformed.

After converting an absolute date into a delay, schedule and account for elapsed time with a monotonic clock. Recheck the deadline before the actual attempt, reserving time for that attempt to finish. Clock skew, response transit time, and scheduling delays are reasons to distinguish a plan from observed retry timing.

When the field is absent or unusable, a bounded client policy can use its own backoff. That is a client decision, not a server-supplied recovery estimate. Keep maximum attempts and total elapsed time explicit so that an optional field does not turn transient failure into unlimited retries.

The same delay can synchronize callers

If thousands of clients receive the same delay at about the same time, they can return together. Retry-After alone does not spread them across a recovery window. Add nonnegative jitter after respecting the requested delay, or otherwise schedule no earlier than that boundary.

Combining local backoff and server guidance requires preserving both constraints. The example takes the larger of its local backoff and parsed server delay, then adds positive jitter. Its injected offsets demonstrate the scheduling relationship; they are not measurements of real distributed traffic or a recommendation for a particular production jitter range.

When it breaks

Blindly replaying a write can duplicate side effects when the original outcome was uncertain. A 429 or 503 response and a Retry-After field do not grant an idempotency guarantee. Use the application’s retry contract, including any operation identifiers or deduplication rules, before repeating nonidempotent work.

Libraries also differ in automatic retries and redirect handling. Verify which statuses, methods, and fields the actual client honors. The fixture’s policy retries only GET and HEAD after 429 or 503; it deliberately does not follow redirects or replay POST. That is a bounded example policy, not the full set of methods HTTP considers idempotent.

Malformed values, huge waits, stale dates, exhausted deadlines, and attempt limits belong in tests. Observe actual attempts and delay rather than assuming that configuring a header caused clients to obey it. The service can remain unavailable after its estimate, so retries still need a terminal outcome.

Check it quickly

Save the block as retry_check.py and run python3 retry_check.py with Python 3.10 or newer. It ran with Python 3.14.4. Only the standard library is required. The script makes real requests to its own loopback HTTP server, uses a fixed fixture date and injected clocks, then stops that server. It does not sleep until a retry or contact an external service.

from datetime import datetime, timedelta, timezone
from email.utils import format_datetime, parsedate_tz
from http.client import HTTPConnection
from http.server import BaseHTTPRequestHandler, HTTPServer
import re
from threading import Thread


UTC = timezone.utc
REFERENCE = datetime(2026, 9, 8, 12, 0, 0, tzinfo=UTC)


def delay_seconds(value, received_wall):
 if value is None:
 return None
 value = value.strip(" \t")
 if re.fullmatch(r"[0-9]+", value):
 # Huge decimal values become infinity and fail the bounded retry budget.
 return float(value)
 parts = parsedate_tz(value)
 if parts is None:
 return None
 year, month, day, hour, minute, second = parts[:6]
 old_date = re.fullmatch(r"[A-Za-z]+, [0-9]{2}-[A-Za-z]{3}-([0-9]{2}) [0-9:]{8} GMT", value)
 if old_date:
 now = datetime.fromtimestamp(received_wall, UTC)
 year = now.year // 100 * 100 + int(old_date[1])
 if (year, month, day, hour, minute, second) > (
 now.year + 50, now.month, now.day, now.hour, now.minute, now.second
 ):
 year -= 100
 if not 0 <= second <= 60:
 return None
 try:
 stamp = datetime(year, month, day, hour, minute, min(second, 59), tzinfo=UTC)
 stamp += timedelta(seconds=(second == 60) - parts[9])
 except (ValueError, OverflowError):
 return None
 return max(0.0, stamp.timestamp() - received_wall)


def retry_at(status, method, value, received_wall, monotonic_now, remaining, attempt, jitter=0.25):
 assert attempt >= 1 and jitter >= 0
 if status not in (429, 503) or method not in ("GET", "HEAD") or attempt >= 3:
 return None
 server_delay = delay_seconds(value, received_wall)
 wait = max(server_delay or 0.0, 2 ** (attempt - 1)) + jitter
 if wait >= remaining:
 return None
 return monotonic_now + wait


wall = REFERENCE.timestamp()
for value in ("Tue, 08 Sep 2026 12:00:07 GMT", "Tuesday, 08-Sep-26 12:00:07 GMT",
 "Tue Sep 8 12:00:07 2026"):
 assert delay_seconds(value, wall) == 7
assert delay_seconds("Sunday, 06-Nov-94 08:49:37 GMT", wall) == 0
assert delay_seconds("Monday, 08-Sep-70 12:00:00 GMT", wall) == (
 datetime(2070, 9, 8, 12, tzinfo=UTC).timestamp() - wall
)
assert delay_seconds("Sat, 31 Dec 2016 23:59:60 GMT",
 datetime(2016, 12, 31, 23, 59, 59, tzinfo=UTC).timestamp()) == 1
for invalid in ("-1", "1.5", "later", "", "٣", "Tue, 31 Feb 2026 12:00:00 GMT"):
 assert delay_seconds(invalid, wall) is None
assert delay_seconds(" 0003\t", wall) == 3
assert retry_at(503, "GET", "9" * 5000, wall, 100, 10, 1) is None
assert retry_at(503, "GET", "3", wall, 100, 2, 1) is None
assert retry_at(503, "POST", "3", wall, 100, 10, 1) is None
assert retry_at(503, "GET", "3", wall, 100, 10, 3) is None
print("both delay forms, obsolete dates, year interpretation, and invalid values are checked")
print("an oversized wait, exhausted deadline, POST, or attempt cap stops this retry policy")

cases = {
 "/seconds": (429, "3"),
 "/date": (503, format_datetime(REFERENCE + timedelta(seconds=7), usegmt=True)),
 "/invalid": (503, "later"),
 "/missing": (503, None),
 "/redirect": (307, "3"),
 "/ready": (200, None),
}
visited = []


class Handler(BaseHTTPRequestHandler):
 def do_GET(self):
 visited.append(self.path)
 status, value = cases.get(self.path, (404, None))
 self.send_response_only(status)
 self.send_header("Date", format_datetime(REFERENCE, usegmt=True))
 if value is not None:
 self.send_header("Retry-After", value)
 if status == 307:
 self.send_header("Location", "/ready")
 self.send_header("Content-Length", "0")
 self.end_headers()

 def log_message(self, *_):
 pass


with HTTPServer(("127.0.0.1", 0), Handler) as server:
 thread = Thread(target=server.serve_forever, kwargs={"poll_interval": 0.05})
 thread.start()
 try:
 for route, expected in (("/seconds", 103.25), ("/date", 107.25),
 ("/invalid", 101.25), ("/missing", 101.25), ("/redirect", None)):
 client = HTTPConnection(*server.server_address, timeout=2)
 try:
 client.request("GET", route)
 response = client.getresponse()
 response.read()
 planned = retry_at(response.status, "GET", response.getheader("Retry-After"),
 wall, 100, 10, 1)
 assert planned == expected
 print(route, "status", response.status, "planned monotonic time", planned)
 finally:
 client.close()
 assert "/ready" not in visited
 finally:
 server.shutdown()
 thread.join(timeout=2)
 assert not thread.is_alive()

same_delay = [retry_at(429, "GET", "3", wall, 100, 10, 1, jitter=0) for _ in range(4)]
spread = [retry_at(429, "GET", "3", wall, 100, 10, 1, jitter=j) for j in (0.1, 0.2, 0.3, 0.4)]
assert same_delay == [103] * 4
assert len(set(spread)) == 4 and min(spread) > 103
print("identical server delays synchronize plans; injected positive jitter spreads them:", spread)

Parser checks include both value forms, obsolete dates, year interpretation, a leap-second boundary, invalid inputs, and an extremely long numeric value. The bounded policy abandons a wait exceeding its deadline, POST replay, and attempts beyond its cap.

The local 429 with three seconds plans time 103.25 from injected monotonic time 100. The dated 503 plans 107.25. Missing or invalid fields use the stated local backoff, while the redirect is not followed. Identical delays produce four plans at 103; supplied positive jitter produces 103.1 through 103.4.

These are response and scheduling-decision tests. Actual timed retries, clock corrections, network partitions, proxy behavior, production jitter distributions, and business-operation deduplication remain untested.

Sources

RFC 9110, sections 5.6.7 and 10.2.3 define HTTP dates and Retry-After, checked 2026-09-08.
RFC 6585, optional Retry-After with 429 responses, checked 2026-09-08.
Python email utilities, date parsing, UTC offsets, and formatted timestamps, checked 2026-09-08.

Rate limiting needs a stable client identifier

Thu, 29 Oct 2026 00:00:00 +0000

A rate-limit key defines which requests share a budget. An account, API credential, address, tenant, and authenticated session are different identities, not interchangeable spellings of “client.” Choosing the wrong grouping can punish unrelated users or let one workload obtain fresh allowance simply by changing a field.

Use the identity the policy actually promises

For a per-account rule, derive the key from the account resolved by authentication, not from a caller-supplied account header or query parameter. A credential’s presence is not proof that it is valid. Authentication must verify it and determine the principal before the account-level decision uses that result.

A per-credential budget is legitimate when that is the intended product contract, but it behaves differently from a per-account budget. Rotating a credential or creating another credential can produce a new credential key while retaining the same account. Decide whether that should renew allowance rather than discovering the distinction through abuse or support tickets.

One account is not necessarily one human or one application, and one person can hold several accounts. A stable account key improves consistency of grouping; it does not prove fairness, prevent account farming, or bound the service’s aggregate demand. Additional scopes need their own justified policies.

An address describes a network path

Several accounts behind a corporate NAT, mobile carrier, or VPN can share a public address. An IP bucket combines their traffic. Conversely, a caller that changes addresses can reach a fresh IP bucket. Neither result means the network address was parsed incorrectly; it means IP and principal identity represent different things.

Address-based limits can still protect public or preauthentication endpoints where no verified account exists. Size and evaluate that policy as an address-level control, including legitimate shared-network traffic. Do not describe it as exact per-person enforcement, or assume a single address can catch a scraper distributed across many networks.

Normalize address syntax before using it as a key. Equivalent IPv6 spellings should not create multiple buckets. Whether to aggregate prefixes or equate particular address families is a separate policy choice with collateral-impact tradeoffs; textual normalization does not answer it.

Forwarded identity needs a trust boundary

An arbitrary Forwarded, X-Forwarded-For, or similar header is not proof of client origin. RFC 7239 explicitly warns that the client and intervening nodes can alter forwarding information. Trust depends on the directly connected peer, the configured proxy chain, and whether that chain removes or safely handles user-supplied values.

NGINX’s real-IP module makes trusted source addresses and recursive processing explicit configuration. “Take the first address in the header” is not an adequate general rule. A proxy accepting and retransmitting an untrusted claim can turn a stable-looking key into caller-controlled budget creation.

The fixture below models exactly one trusted proxy that overwrites a field with one IP literal. It ignores replacement claims from other peers and rejects a multi-address value from that proxy. It is not a general forwarding-chain parser or a deployed NGINX configuration.

When it breaks

Omitting tenant scope can merge unrelated accounts that share an identifier. Keying on raw URLs can split the same operation by query strings; use the stable operation identity resolved by routing when the policy is per operation. Ambiguous string concatenation can also collide, so preserve component boundaries in the key encoding.

Avoid using raw bearer secrets as budget keys or diagnostic labels. Stable internal credential or principal identifiers are more appropriate, and logs still need privacy controls. Unbounded identity creation can consume limiter memory; expiration or state eviction must be evaluated alongside the extra allowance that resetting a bucket can create.

Test realistic identities before selecting thresholds: shared addresses, roaming clients, credential rotation, tenants, and forged claims. Also inspect the source of the supposedly verified principal. No choice of dictionary key can repair authentication data that was actually copied from an untrusted request.

Check it quickly

Save the block as identity_check.py and run python3 identity_check.py with Python 3.10 or newer. It ran with Python 3.14.4 and uses only the standard library. The records represent already authenticated and routed fixture requests. They are not an authentication implementation, and the counters model one test window without servers, network traffic, or file writes.

from collections import Counter
from ipaddress import ip_address


def account_key(request):
 # These two fields come from authentication and routing, not request headers.
 tenant, account = request["verified_actor"]
 return ("account", tenant, account, request["operation"])


def admit(counts, key, limit=2):
 if counts[key] >= limit:
 return False
 counts[key] += 1
 return True


alice = {"verified_actor": ("north", "alice"), "operation": "export",
 "peer": "198.51.100.8", "credential_id": "key-a", "claimed_account": "alice"}
bob = {**alice, "verified_actor": ("north", "bob"), "credential_id": "key-b"}
by_ip, by_account = Counter(), Counter()
assert [admit(by_ip, str(ip_address(request["peer"]))) for request in (alice, alice, bob)] == [
 True, True, False,
]
assert [admit(by_account, account_key(request)) for request in (alice, alice, bob)] == [
 True, True, True,
]
print("two accounts behind one address collide in the IP budget, but not the account budget")

roaming = {**alice, "peer": "203.0.113.9", "credential_id": "key-a-rotated"}
assert admit(by_ip, str(ip_address(roaming["peer"])))
assert not admit(by_account, account_key(roaming))
for claim in ("bob", "new-account", "admin"):
 assert account_key({**roaming, "claimed_account": claim}) == account_key(alice)
print("changing address, credential ID, or an untrusted account claim does not reset this account budget")

other_tenant = {**alice, "verified_actor": ("south", "alice")}
other_operation = {**alice, "operation": "thumbnail"}
assert admit(by_account, account_key(other_tenant))
assert admit(by_account, account_key(other_operation))
assert account_key({**alice, "verified_actor": ("a:b", "c")}) != account_key(
 {**alice, "verified_actor": ("a", "b:c")}
)
print("tenant and operation are explicit dimensions; tuple encoding avoids delimiter collisions")

# One configured proxy that overwrites the header with exactly one IP literal.
trusted_proxy = ip_address("192.0.2.10")


def network_key(peer, replacement):
 peer = ip_address(peer)
 return str(ip_address(replacement)) if peer == trusted_proxy else str(peer)


assert network_key("198.51.100.8", "203.0.113.77") == "198.51.100.8"
assert network_key("198.51.100.8", "not-an-address") == "198.51.100.8"
assert network_key("192.0.2.10", "198.51.100.8") == "198.51.100.8"
assert network_key("2001:0db8:0:0::1", "ignored") == network_key("2001:db8::1", "ignored")
try:
 network_key("192.0.2.10", "198.51.100.8, 203.0.113.9")
except ValueError:
 pass
else:
 raise AssertionError("the single-address proxy contract must reject a chain")
print("untrusted peers cannot choose a forwarded identity; equivalent IPv6 text shares one key")
print("the configured one-address proxy contract rejects a multi-address value")

Alice exhausts an address budget and thereby blocks Bob at the same address, while account grouping leaves Bob’s allowance separate. Alice’s changed address creates new IP allowance, but neither that move, a credential-ID rotation, nor forged account claims reset the account key.

Separate tenants and operations receive separate keys; tuple components avoid delimiter collisions. Untrusted peers cannot choose a replacement address, equivalent IPv6 text normalizes identically, and the single-proxy contract rejects a list of addresses. Authentication, real proxy forwarding, concurrent counting, distributed quotas, and production cardinality are not exercised.

Sources

RFC 6585, section 4 leaves identity and counting scope to the limiter’s policy, checked 2026-09-08.
RFC 7239, forwarding-header integrity, trusted proxies, and information exposure, checked 2026-09-08.
NGINX real-IP module, trusted replacement addresses and recursive processing, checked 2026-09-08.

Rate limiting is a shared budget

Sat, 22 Aug 2026 00:00:00 +0000

A rate limit is a budget for admitted work, defined by its unit, identity, scope, and time rule. “Five requests” is incomplete without saying whose requests, across which endpoints and workers, during which interval, and what happens after exhaustion. A shared counter helps only when every relevant admission uses the same authoritative decision.

State the policy before counting

A request can cost one unit, several estimated work units, or a number of bytes. Count attempted admission, successful execution, and refunded work differently only when the contract says so. Accepting a request into the budget does not guarantee that its downstream operation succeeds.

Window algorithms also differ. A fixed-window counter can allow bursts near a boundary; a rolling-window rule measures a different interval. A token bucket permits stored credit to be spent quickly and does not automatically pace departures. None of these alone limits the number of slow operations still running.

Per-account, per-route, regional, and service-wide limits answer different questions. A remaining-budget value is a snapshot, not a reservation: another worker can spend credit before the caller uses it. Independent worker-local counters multiply the effective allowance unless a documented partitioning rule divides the intended budget between them.

Make the decision atomic

A shared Redis key does not make a sequence of client commands atomic. Two callers can both read remaining capacity, both decide to proceed, and then both increment. Each INCR is atomic, but the earlier comparison was outside that operation. The total can exceed the promised allowance.

The example places comparison, first-counter creation with expiration, and increment inside one short Lua execution. Its window starts at the first accepted request; rejection neither increments the count nor extends the deadline. All callers supply the same key and fixed positive parameters. The fixture owns that numeric counter exclusively.

This is a first-arrival-anchored window, not a continuously sliding quota. The script’s noninterleaving property is also not rollback, durable reservation, or a transaction with the application’s business operation. Keep scripts short because Redis blocks other server activity while executing them. Treat connection loss as an uncertain result, not proof that credit was never spent.

Explain the refusal accurately

RFC 6585 defines HTTP 429 for excessive request rates, but leaves identity and counting policy to the server. The response should explain the condition and may include Retry-After. A 429 response must not be stored by a cache. Generic service unavailability is a different condition from a particular caller exceeding its allowance.

Retry-After can communicate a wait without guaranteeing future capacity. If exposing remaining capacity or reset metadata, define their units and scope in the API contract. Historical X-RateLimit field conventions are not a universal interpretation supplied by RFC 6585. Do not invent a standard merely because several APIs use similar names.

When it breaks

Split check/update operations overspend under contention. Separately creating a counter and attaching expiration can leave an immortal counter if the client stops between commands. Eviction, restart, or asynchronous failover can also lose budget state; atomic execution on one live server does not prove a hard ceiling across those failures.

Document what an unavailable budget store means for admission. Silently treating an error as an empty counter creates extra allowance. Protect the limiter’s own latency and capacity, observe allowed and denied work together, and distinguish resource protection from arbitrary early rejection of requests still entitled to proceed.

Check it quickly

Use a POSIX environment with a C compiler, make, curl, tar, and Python 3.10 or newer. This ran on macOS with Redis 8.10.1, redis-py 8.1.0, and Python 3.14.4. The setup builds Redis core and installs the client in a temporary environment.

redis_lab=$(mktemp -d)
curl --fail --location --output "$redis_lab/redis.tar.gz" https://download.redis.io/releases/redis-8.10.1.tar.gz
tar -xzf "$redis_lab/redis.tar.gz" -C "$redis_lab"
make -C "$redis_lab/redis-8.10.1" build redis -j2 MALLOC=libc
export REDIS_SERVER="$redis_lab/redis-8.10.1/src/redis-server"
python3 -m venv "$redis_lab/venv"
"$redis_lab/venv/bin/python" -m pip install redis==8.1.0

Run the Python block with "$redis_lab/venv/bin/python". It starts only its own Redis process on a private Unix socket, verifies its process ID, disables TCP and persistence, and cleans up afterward. Sixteen independent client connections contend for five admissions. The first path deliberately demonstrates a broken check/update sequence.

from concurrent.futures import ThreadPoolExecutor
from contextlib import contextmanager
import os
from pathlib import Path
import subprocess
from tempfile import TemporaryDirectory
from threading import Barrier
import time

import redis
from redis.backoff import NoBackoff
from redis.retry import Retry


def connection(endpoint):
 return redis.Redis(unix_socket_path=endpoint, decode_responses=False,
 socket_timeout=2, retry=Retry(NoBackoff(), 0))


@contextmanager
def instance():
 with TemporaryDirectory(prefix="redis-budget-", dir="/tmp") as directory:
 endpoint = str(Path(directory) / "redis.sock")
 process = subprocess.Popen([
 os.environ["REDIS_SERVER"], "--port", "0", "--unixsocket", endpoint,
 "--unixsocketperm", "700", "--dir", directory,
 "--save", "", "--appendonly", "no",
 ], stdout=subprocess.DEVNULL, stderr=subprocess.PIPE)
 client = connection(endpoint)
 try:
 deadline = time.monotonic() + 5
 while True:
 assert process.poll() is None, "Redis exited during startup"
 try:
 if client.ping():
 break
 except redis.ConnectionError:
 assert time.monotonic() < deadline, "Redis did not start"
 time.sleep(0.02)
 assert client.info("server")["process_id"] == process.pid
 yield client, endpoint
 finally:
 client.close()
 if process.poll() is None:
 process.terminate()
 try:
 _, errors = process.communicate(timeout=5)
 except subprocess.TimeoutExpired:
 process.kill()
 process.communicate()
 raise
 assert process.returncode == 0, errors.decode()


acquire = """
local current = tonumber(redis.call('GET', KEYS[1]) or '0')
if current >= tonumber(ARGV[1]) then
 return {0, current, redis.call('PTTL', KEYS[1])}
end
if current == 0 then
 redis.call('SET', KEYS[1], '1', 'PX', ARGV[2])
else
 redis.call('INCR', KEYS[1])
end
return {1, current + 1, redis.call('PTTL', KEYS[1])}
"""

LIMIT, WINDOW_MS, WORKERS = 5, 60000, 16

with instance() as (control, endpoint):
 def concurrent(operation):
 barrier = Barrier(WORKERS)

 def run(_):
 with connection(endpoint) as client:
 return operation(client, barrier)

 with ThreadPoolExecutor(max_workers=WORKERS) as pool:
 return list(pool.map(run, range(WORKERS)))

 control.set("budget:unsafe", 0)

 def split_check(client, barrier):
 observed = int(client.get("budget:unsafe"))
 barrier.wait(timeout=5) # All callers read before any caller increments.
 if observed >= LIMIT:
 return False
 client.incr("budget:unsafe")
 return True

 unsafe = concurrent(split_check)
 assert sum(unsafe) == WORKERS
 assert int(control.get("budget:unsafe")) == WORKERS
 print("split GET/INCR admits 16 against a budget of 5")

 def atomic_check(client, barrier):
 barrier.wait(timeout=5)
 return client.eval(acquire, 1, "budget:shared", LIMIT, WINDOW_MS)

 results = concurrent(atomic_check)
 assert sum(reply[0] for reply in results) == LIMIT
 assert sorted(reply[1] for reply in results if reply[0]) == list(range(1, LIMIT + 1))
 assert all(reply[1] == LIMIT for reply in results if not reply[0])
 assert all(0 < reply[2] <= WINDOW_MS for reply in results)
 assert control.get("budget:shared") == b"5"
 print("one atomic script admits 5 and rejects 11 across independent client connections")

 ttl_before = control.pttl("budget:shared")
 assert control.eval(acquire, 1, "budget:shared", LIMIT, WINDOW_MS)[:2] == [0, LIMIT]
 assert 0 < control.pttl("budget:shared") <= ttl_before
 assert control.get("budget:shared") == b"5"
 print("a rejection neither spends more budget nor refreshes the window")

 assert control.pexpire("budget:shared", 50)
 deadline = time.monotonic() + 3
 while control.exists("budget:shared"):
 assert time.monotonic() < deadline, "test deadline did not expire"
 time.sleep(0.01)
 first = control.eval(acquire, 1, "budget:shared", LIMIT, WINDOW_MS)
 assert first[:2] == [1, 1] and 0 < first[2] <= WINDOW_MS
 print("after deliberately shortening and expiring the test window, the next admission starts at 1")

A barrier makes every unsafe caller read before any increments, reproducibly admitting sixteen. The atomic script then admits exactly five, rejects eleven, and stores five; successful replies contain each count from one through five once. Rejection leaves both the count and original expiration intact.

The test deliberately shortens its own deadline to fifty milliseconds, waits for real expiration, and verifies that the next admission starts at one with a new window. This avoids waiting a full minute without claiming to have measured a minute-long workload. Cluster operation, failover, persistence, HTTP middleware, production latency, and coupling admission to business side effects are not tested.

Sources

RFC 6585, section 4’s 429 semantics, optional Retry-After, and caching prohibition, checked 2026-09-08.
RFC 9110, Retry-After and service-unavailability semantics, checked 2026-09-08.
Redis scripting introduction, atomic execution and blocking behavior, checked 2026-09-08.
Redis EVAL, explicit key and argument passing, checked 2026-09-08.
Redis INCR, counter operations and rate-limiter race conditions, checked 2026-09-08.
Redis PEXPIRE, millisecond expiration used in the isolated test, checked 2026-09-08.
Redis 8.10.1 build instructions, core build and allocator selection, checked 2026-09-08.

Rate-Limiting · XB Field Notes

A token bucket smooths the request rate

Refill credit, then charge work

A burst allowance is not a peak rate

Match the budget to its scope

When it breaks

Check it quickly

Further reading

Sources

Retry-After tells when to try again

Interpret the value and its context

Respect the wait without waiting forever

The same delay can synchronize callers

When it breaks

Check it quickly

Further reading

Sources

Rate limiting needs a stable client identifier

Use the identity the policy actually promises

An address describes a network path

Forwarded identity needs a trust boundary

When it breaks

Check it quickly

Further reading

Sources

Rate limiting is a shared budget

State the policy before counting

Make the decision atomic

Explain the refusal accurately

When it breaks

Check it quickly

Further reading

Sources