nginx request limiting uses a leaky-bucket method to account for excess requests against a configured rate. The burst allowance determines how much excess can be accepted, while nodelay and delay affect when accepted excess proceeds. These are admission and scheduling decisions. They do not make every request above the nominal rate fail, nor do they measure concurrent application work.

When it breaks

Describing nodelay as “drop the excess immediately” reverses its useful effect. Within the burst allowance, it lets excess requests proceed without the normal delay. Requests beyond that allowance are rejected. The accepted burst still consumes capacity that must recover over time; repeatedly arriving bursts do not receive a fresh allowance merely because earlier requests completed quickly.

Delayed requests also do not automatically become a sequence of 503 responses. Requests admitted within the burst can wait and then receive successful application responses. Rejection occurs when the configured allowance is exceeded, with the selected rejection status. A client’s own deadline may expire while waiting, so an acceptable server-side queue can still be unsuitable for that client’s latency contract.

Configuration syntax needs verification too. In nginx 1.28.0, writing burst=0 is rejected even though omitting burst selects a default allowance of zero. An invalid configuration is not an especially strict working limiter. Use syntax validation and actual requests, and ensure the protected location reaches request limiting rather than ending in an earlier rewrite-phase return.

Check it quickly

Save this as check-burst.py and run it with Python, Docker, and the pinned images. The recorded setup uses nginx 1.28.0 and a real HTTP backend. Each case gets a separate public fixture key, a one-request-per-second rate, and six clients released together. nginx exposes its limit decision in a response header:

from contextlib import contextmanager
from pathlib import Path
from tempfile import TemporaryDirectory
import http.client,json,socket,subprocess,time,uuid

NGINX='nginx@sha256:552e7481ca93ffccd046aa658dbbed22caefbc09c66fa7cd247cbb90b8a5c609'
PYTHON='python@sha256:528257d48c1da0dcecc2e725d1ae34498d60c965f1241e39cd6a85a8859bdf84'
def docker(*args,check=True):
    result=subprocess.run(['docker',*args],capture_output=True,text=True,timeout=120)
    if check:assert result.returncode==0,result.stderr
    return result
def wait(predicate):
    deadline=time.monotonic()+15
    while time.monotonic()<deadline:
        try:
            if predicate():return
        except OSError:pass
        time.sleep(.1)
    raise AssertionError('fixture did not become ready')

@contextmanager
def lab(configuration,files=None,backend=None,extra_ports=()):
    name='xbcatm-nginx-'+uuid.uuid4().hex[:10];tags=[name+':proxy',name+':backend'];containers=[name+'-proxy',name+'-backend']
    try:
        with TemporaryDirectory(prefix=name+'-') as directory:
            root=Path(directory);(root/'files').mkdir();(root/'nginx.conf').write_text(configuration)
            for path,content in (files or {}).items():
                target=root/'files'/path;target.parent.mkdir(parents=True,exist_ok=True)
                target.write_bytes(content if isinstance(content,bytes) else content.encode())
            docker('network','create',name)
            if backend is not None:
                (root/'backend.py').write_text(backend)
                (root/'backend.Dockerfile').write_text(f'FROM {PYTHON}\nCOPY backend.py /backend.py\nCMD ["python", "/backend.py"]\n')
                docker('build','--network','none','-f',str(root/'backend.Dockerfile'),'-t',tags[1],str(root))
                docker('run','-d','--name',containers[1],'--network',name,'--network-alias','backend',tags[1])
                wait(lambda:docker('exec',containers[1],'python','-c','import socket;s=socket.create_connection(("127.0.0.1",8081),timeout=.5);s.close()',check=False).returncode==0)
            (root/'Dockerfile').write_text(f'FROM {NGINX}\nCOPY nginx.conf /etc/nginx/nginx.conf\nCOPY files /srv\n')
            docker('build','--network','none','-t',tags[0],str(root))
            docker('run','--rm','--network',name,tags[0],'nginx','-t')
            published=[arg for value in (8080,*extra_ports) for arg in ('-p','127.0.0.1::'+str(value))]
            docker('run','-d','--name',containers[0],'--network',name,*published,tags[0])
            details=json.loads(docker('inspect',containers[0]).stdout)[0]
            binding=details['NetworkSettings']['Ports']['8080/tcp'][0];assert binding['HostIp']=='127.0.0.1';port=int(binding['HostPort'])
            def ready():
                connection=socket.create_connection(('127.0.0.1',port),timeout=.5);connection.close();return True
            wait(ready)
            def request(path,method='GET',headers=None,body=None):
                connection=http.client.HTTPConnection('127.0.0.1',port,timeout=5)
                try:
                    connection.request(method,path,body=body,headers=headers or {})
                    response=connection.getresponse();return response.status,{k.lower():v for k,v in response.getheaders()},response.read()
                finally:connection.close()
            request.port=port;request.network=name
            request.ports={int(key.split('/')[0]):int(value[0]['HostPort']) for key,value in details['NetworkSettings']['Ports'].items() if value}
            yield containers[0],request
    finally:
        docker('rm','-f',*containers,check=False);docker('image','rm',*tags,check=False);docker('network','rm',name,check=False)

def config(locations,extra=''):
    return 'worker_processes 1;\nerror_log /tmp/error.log notice;\nevents {}\nhttp {\naccess_log off;\n'+extra+'\nserver { listen 8080; root /srv;\n'+locations+'\n}\n}\n'
backend=r'''
from http.server import BaseHTTPRequestHandler,ThreadingHTTPServer
import json
class Handler(BaseHTTPRequestHandler):
    protocol_version='HTTP/1.1'
    def log_message(self,*args):pass
    def do_GET(self):
        body=json.dumps({'path':self.path,'method':self.command}).encode()
        self.send_response(200);self.send_header('Content-Length',str(len(body)));self.end_headers();self.wfile.write(body)
ThreadingHTTPServer(('0.0.0.0',8081),Handler).serve_forever()
'''
from concurrent.futures import ThreadPoolExecutor
import threading
extra='limit_req_zone $arg_k zone=budget:1m rate=1r/s; add_header X-Limit $limit_req_status always;'
locations='location / { return 404; }\n'
for path,policy in [('delay','burst=2'),('nowait','burst=2 nodelay'),('partial','burst=2 delay=1'),('strict','')]:
    locations+=f'location /{path} {{ limit_req zone=budget {policy}; limit_req_status 429; proxy_pass http://backend:8081; }}\n'
locations+='location /default { limit_req zone=budget; proxy_pass http://backend:8081; }'
invalid=config('location / { limit_req zone=b burst=0 nodelay; proxy_pass http://backend:8081; }',
    'limit_req_zone $binary_remote_addr zone=b:1m rate=1r/s;')
with lab(config(locations,extra),{'invalid.conf':invalid},backend) as (proxy,request):
    results={}
    for mode in ['delay','nowait','partial','strict']:
        barrier=threading.Barrier(7)
        def client():
            barrier.wait();start=time.monotonic();status,headers,_=request('/'+mode+'?k='+mode)
            return {'status':status,'limit':headers.get('x-limit'),'elapsed':round(time.monotonic()-start,3)}
        with ThreadPoolExecutor(max_workers=6) as pool:
            futures=[pool.submit(client) for _ in range(6)];barrier.wait();observed=[f.result() for f in futures]
        good=[r for r in observed if r['status']==200];bad=[r for r in observed if r['status']==429]
        assert len(good)==(1 if mode=='strict' else 3) and len(good)+len(bad)==6,observed
        assert all(r['limit']=='REJECTED' for r in bad)
        assert all(r['limit'] in {'PASSED','DELAYED'} for r in good)
        delayed=sum(r['limit']=='DELAYED' for r in good)
        assert delayed=={'delay':2,'nowait':0,'partial':1,'strict':0}[mode],observed
        if mode=='delay':assert sorted(r['elapsed'] for r in good)[-1]>=1.7
        if mode=='partial':assert max(r['elapsed'] for r in good)>=.7
        results[mode]=observed
        if mode=='nowait':
            assert request('/nowait?k=nowait')[0]==429
            time.sleep(1.1);assert request('/nowait?k=nowait')[0]==200
            results['nodelay_debt_then_recovery']=[429,200]
    first=request('/default?k=default');second=request('/default?k=default')
    assert first[0]==200 and second[0]==503 and second[1]['x-limit']=='REJECTED'
    results['default_rejection']=503
    rejected=docker('exec',proxy,'nginx','-t','-c','/srv/invalid.conf',check=False)
    assert rejected.returncode!=0 and 'invalid burst value' in rejected.stderr
    results['literal_burst_zero_valid']=False
    print(json.dumps(results,sort_keys=True))

With burst=2, one request proceeds immediately and two accepted excess requests are delayed. The recorded successful responses complete near zero, one, and two seconds; three other requests return 429. Their headers distinguish PASSED, DELAYED, and REJECTED. These observations show accepted waiting work completing successfully, rather than treating every delayed request as an eventual error.

With burst=2 nodelay, three requests succeed without the limiter marking them DELAYED, and three return 429. An immediate additional request using the same key is also rejected. After waiting 1.1 seconds, another request succeeds. The experiment therefore checks both immediate burst admission and the continuing debt that prevents the next burst from bypassing the configured rate accounting.

With burst=2 delay=1, two accepted requests proceed without delay and the third is delayed by about one second. Three are rejected. Omitting burst admits only one of the six closely arriving requests, with five rejections. The tests assert decision classes and broad delay bounds rather than presenting millisecond measurements as a portable performance benchmark.

The protected comparison locations explicitly choose 429. A separate location leaves the rejection status at its default and returns 503 on the second immediate request. Finally, nginx syntax-checks a disposable configuration containing literal burst=0 and rejects it. The running valid configuration remains separate from that failed candidate, so the negative syntax test does not interrupt the other cases.

Fit the waiting policy to the endpoint

Use delayed excess when smoothing accepted arrivals is useful and waiting fits the request deadline. Use nodelay when a bounded immediate burst is acceptable to the backend. Partial delay provides a middle region before pacing starts. None of these settings alone caps the number of long-running requests already executing, so do not confuse arrival rate with concurrent resource occupancy.

Choose the key as carefully as the rate. The query key here makes independent cases reproducible; it is intentionally client-controlled and is not a production identity scheme. A client that can change its key can change which budget it consumes. An IP key has different boundaries, including multiple users sharing an address and trusted proxy handling.

Expose enough information to separate application errors from limiter rejections and delayed requests. A 429 response can describe the rate policy and may include Retry-After when a meaningful retry time is available, but this example does not generate that header automatically. Verify the effective status, scheduling, and backend behavior instead of assuming a directive name describes the complete client experience.

Sources