A token bucket bounds admitted work while allowing accumulated credit to be spent in a burst. It does not automatically space requests evenly, guarantee completed throughput, or cap the number of operations still running. Those are separate scheduling and capacity questions. The bucket’s two parameters describe a budget, not a promise that the service can perform it.

Refill credit, then charge work

Let capacity B be the maximum stored tokens and rate r be tokens added per second. At each arrival, update credit to min(B, previous_credit + r × elapsed). Admit work only if its cost fits, then subtract that cost. Under the strict rule used here, rejected work does not consume credit.

The bucket can refill lazily when requests arrive; a background timer is unnecessary for this calculation. Idle time cannot accumulate unlimited credit because capacity clips the result. Initial credit is a policy choice: the example starts full, allowing an immediate burst, while an empty initialization would behave differently.

A burst allowance is not a peak rate

For this ideal strict bucket, admitted cost over an interval of duration t is at most B + r × t. As the interval grows, the burst contribution becomes relatively smaller. That bound does not mean every one-second window admits at most r requests, nor does it establish a separate instantaneous peak-rate ceiling.

For capacity three and rate two, three unit-cost requests can pass at the same timestamp. A fourth is denied. Half a second later one token is available. A real scheduler, bounded queue, or additional pacing rule is needed if permitted work must depart at evenly spaced times.

Rate limiting admission also does not bound concurrent execution: slow requests can remain active while more tokens refill. Protect scarce worker or database capacity with an appropriate concurrency limit and timeout policy. Likewise, an accepted request may still fail downstream, so refill rate is not guaranteed successful throughput.

Match the budget to its scope

A token need not mean one request. Weighted costs can represent bytes or estimated work, provided refill rate and capacity use the same units. A request costing more than B can never fit this strict bucket, even after a long idle period. Handle that case explicitly instead of promising an eventual retry time.

Choose the identity and ownership of the budget carefully. Four independent full buckets each holding three tokens can admit twelve requests at once. They are not one global three-token bucket. A shared limiter must make refill, comparison, and deduction an atomic decision; merely sharing a counter without coordinating those steps is insufficient.

Use an elapsed-time clock for local refill calculations. Wall-clock adjustments must not mint credit or move the accounting backward. Python’s monotonic clock is designed for elapsed differences; its reference point is not a portable timestamp to exchange between unrelated machines. Distributed clock and state ownership need their own contract.

When it breaks

A full bucket can release a burst that overwhelms a fragile dependency even when the long-run allowance looks conservative. Unbounded waiting queues turn excess demand into memory growth and latency. Denial and delayed shaping are different outcomes, so document which one callers receive.

Observe admitted and denied cost, queueing, concurrency, and downstream latency together. Retries should respect the limiting policy instead of immediately recreating rejected load. Charging attempted work versus completed work is another explicit choice; refunding failures changes the model and requires careful accounting.

Managed implementations can provide weaker ceilings than the ideal algorithm. Amazon API Gateway documents throttling and quotas as best-effort targets that can be exceeded. Do not present the mathematical bound as a verified guarantee for that service, a billing cap, or any other untested deployment.

Check it quickly

Save the block as bucket_check.py and run python3 bucket_check.py with Python 3.10 or newer. It ran with Python 3.14.4. The standard-library Fraction type and injected nondecreasing timestamps make boundaries exact without sleeping. This is a single-threaded admission model, not a network limiter or distributed store.

from fractions import Fraction


class Bucket:
    def __init__(self, capacity, rate):
        assert capacity > 0 and rate > 0
        self.capacity, self.rate = capacity, rate
        self.tokens, self.updated = Fraction(capacity), Fraction(0)

    def take(self, now, cost=1):
        assert now >= self.updated and cost > 0
        self.tokens = min(self.capacity, self.tokens + (now - self.updated) * self.rate)
        self.updated = now
        if self.tokens < cost:
            return False
        self.tokens -= cost
        return True


bucket = Bucket(capacity=3, rate=2)
events = ([(Fraction(0), 1)] * 4 + [(Fraction(1, 4), 1), (Fraction(1, 2), 1),
                                  (Fraction(3, 2), 2)] + [(Fraction(100), 1)] * 4)
results = [(now, cost, bucket.take(now, cost)) for now, cost in events]
assert [allowed for _, _, allowed in results] == [
    True, True, True, False, False, True, True, True, True, True, False,
]
for now, cost, allowed in results:
    print(f"t={str(now):>3} cost={cost} {'allow' if allowed else 'deny'}")
assert bucket.tokens == 0
marks = sorted({now for now, _ in events})
for start in marks:
    for end in marks:
        if end >= start:
            consumed = sum(cost for now, cost, allowed in results if allowed and start <= now <= end)
            assert consumed <= bucket.capacity + bucket.rate * (end - start)
print("every event-bounded interval obeys admitted cost <= B + r * elapsed")

assert not bucket.take(Fraction(200), cost=4)
assert bucket.tokens == 3
assert bucket.take(Fraction(200), cost=3)
print("cost above capacity never fits; rejection does not consume tokens")

workers = [Bucket(capacity=3, rate=2) for _ in range(4)]
local_admitted = sum(worker.take(Fraction(0)) for worker in workers for _ in range(4))
shared = Bucket(capacity=3, rate=2)
shared_admitted = sum(shared.take(Fraction(0)) for _ in range(16))
assert local_admitted == 12 and shared_admitted == 3
print("four independent full buckets admit", local_admitted, "at once; one shared bucket admits", shared_admitted)

The trace allows three simultaneous requests, rejects the fourth and the quarter-second request, then allows one at half a second and cost two at one-and-a-half seconds. After long idle time it permits only another three-token burst. Assertions check every interval bounded by event timestamps against the cost envelope.

An oversized request is denied without spending credit. Finally, four independent buckets admit twelve simultaneous requests while one shared sequential model admits three. No concurrency races, real clock adjustments, queue scheduling, cloud throttling, network partitions, or production performance are tested.

Further reading

Sources

  • RFC 3290, Appendix A’s token-bucket envelope and strict conformance calculation, checked 2026-09-08.
  • Amazon API Gateway throttling, request budgets, bursts, and best-effort limits, checked 2026-09-08.
  • Python time, monotonic elapsed-time clocks and their reference-point limits, checked 2026-09-08.