Routing · XB Field Notes

An nginx prefix is a candidate until matching finishes

Mon, 17 Jan 2028 00:00:00 +0000

An nginx location selects configuration for a request within the chosen server. For the flat prefix and regex locations tested here, an exact match finishes the search; otherwise nginx remembers the longest matching prefix and can then choose the first matching regex. If no applicable regex wins, the remembered prefix supplies the configuration. This is a selection process, not simply the first block encountered in the file.

When it breaks

A prefix is a string prefix, not automatically a complete path segment. location /api also matches /apix. If the intended family is the endpoint /api plus descendants under /api/, express those boundaries deliberately. Depending on an unstated segment boundary can apply the wrong handler, cache policy, or access rule to paths that merely share the same initial letters.

The longest ordinary prefix is not always the final choice. A regex matching .php can take a request that also matched a more detailed API prefix. Moving that prefix above the regex does not change the basic priority rule. Configuration order matters for competing regexes, while ordinary prefixes are compared by matching length rather than declaration position.

The ^~ modifier suppresses regex searching when that prefix is the selected longest prefix at the relevant level. It is not a regex operator, and it is not an unconditional protection for every path that begins with a shorter marked prefix. Nested location structures and internal redirects require additional reasoning; the following fixture intentionally uses a flat server to make each selection unambiguous.

Check it quickly

Run the script with host Python, Docker, and the pinned nginx image available. The recorded environment used nginx 1.28.0 on Linux arm64 with Docker 29.3.1. It builds a temporary configuration image, validates it with nginx -t, sends real HTTP requests through a port published only on host loopback, and removes its own containers, image tags, and network:

from contextlib import contextmanager
from pathlib import Path
from tempfile import TemporaryDirectory
import http.client,json,socket,subprocess,time,uuid

NGINX='nginx@sha256:552e7481ca93ffccd046aa658dbbed22caefbc09c66fa7cd247cbb90b8a5c609'
PYTHON='python@sha256:528257d48c1da0dcecc2e725d1ae34498d60c965f1241e39cd6a85a8859bdf84'
def docker(*args,check=True):
 result=subprocess.run(['docker',*args],capture_output=True,text=True,timeout=120)
 if check:assert result.returncode==0,result.stderr
 return result
def wait(predicate):
 deadline=time.monotonic()+15
 while time.monotonic()<deadline:
 try:
 if predicate():return
 except OSError:pass
 time.sleep(.1)
 raise AssertionError('fixture did not become ready')

@contextmanager
def lab(configuration,files=None,backend=None):
 name='xbcatm-nginx-'+uuid.uuid4().hex[:10];tags=[name+':proxy',name+':backend'];containers=[name+'-proxy',name+'-backend']
 try:
 with TemporaryDirectory(prefix=name+'-') as directory:
 root=Path(directory);(root/'files').mkdir();(root/'nginx.conf').write_text(configuration)
 for path,content in (files or {}).items():
 target=root/'files'/path;target.parent.mkdir(parents=True,exist_ok=True);target.write_text(content)
 docker('network','create',name)
 if backend is not None:
 (root/'backend.py').write_text(backend)
 (root/'backend.Dockerfile').write_text(f'FROM {PYTHON}\nCOPY backend.py /backend.py\nCMD ["python", "/backend.py"]\n')
 docker('build','--network','none','-f',str(root/'backend.Dockerfile'),'-t',tags[1],str(root))
 docker('run','-d','--name',containers[1],'--network',name,'--network-alias','backend',tags[1])
 wait(lambda:docker('exec',containers[1],'python','-c','import socket;s=socket.create_connection(("127.0.0.1",8081),timeout=.5);s.close()',check=False).returncode==0)
 (root/'Dockerfile').write_text(f'FROM {NGINX}\nCOPY nginx.conf /etc/nginx/nginx.conf\nCOPY files /srv\n')
 docker('build','--network','none','-t',tags[0],str(root))
 docker('run','--rm','--network',name,tags[0],'nginx','-t')
 docker('run','-d','--name',containers[0],'--network',name,'-p','127.0.0.1::8080',tags[0])
 details=json.loads(docker('inspect',containers[0]).stdout)[0]
 binding=details['NetworkSettings']['Ports']['8080/tcp'][0];assert binding['HostIp']=='127.0.0.1';port=int(binding['HostPort'])
 def ready():
 connection=socket.create_connection(('127.0.0.1',port),timeout=.5);connection.close();return True
 wait(ready)
 def request(path,method='GET',headers=None,body=None):
 connection=http.client.HTTPConnection('127.0.0.1',port,timeout=5)
 try:
 connection.request(method,path,body=body,headers=headers or {})
 response=connection.getresponse();return response.status,{k.lower():v for k,v in response.getheaders()},response.read()
 finally:connection.close()
 yield containers[0],request
 finally:
 docker('rm','-f',*containers,check=False);docker('image','rm',*tags,check=False);docker('network','rm',name,check=False)

def config(locations,extra=''):
 return 'worker_processes 1;\nerror_log /tmp/error.log notice;\nevents {}\nhttp {\naccess_log off;\n'+extra+'\nserver { listen 8080; root /srv;\n'+locations+'\n}\n}\n'

locations=r'''
location / { return 200 "fallback"; }
location /api/ { return 200 "api-slash"; }
location /api/admin/ { return 200 "admin"; }
location = /api { return 200 "exact"; }
location /api { return 200 "api-string"; }
location ^~ /static/ { return 200 "static"; }
location ~ \.php$ { return 200 "php-regex"; }
'''
with lab(config(locations)) as (container,request):
 cases={'/api':'exact','/api?x=1':'exact','/apix':'api-string','/api/users':'api-slash','/api/admin/log':'admin','/api/admin/test.php':'php-regex','/static/test.php':'static','/unmatched':'fallback'}
 for path,expected in cases.items():
 status,headers,body=request(path);assert status==200 and body.decode()==expected,(path,status,body)
 print('exact matching ignores the query; ordinary prefixes use longest string match, including /api matching /apix')
 print('a regex overrides the ordinary longest prefix, while the selected ^~ prefix skips that regex')

Both /api and /api?x=1 return exact. The query string does not participate in location matching, so adding an argument does not make the exact path stop matching. The selected block explicitly returns that marker. Exact matching selects configuration; it does not inherently mean a filesystem root is read or a particular application is invoked.

The request /apix returns api-string, demonstrating that the plain /api prefix reaches beyond a complete API segment. /api/users instead returns api-slash, because /api/ is the longer matching prefix. /api/admin/log returns admin, showing the next level of ordinary longest-prefix selection without relying on the order those blocks appear.

Requesting /api/admin/test.php returns php-regex. The long /api/admin/ prefix was a candidate, but the regex phase selected another block. This is the important failure shape for a route whose author assumed a longer prefix automatically defeated a general script handler. A successful status alone would conceal the mistake, so each handler returns a distinguishable body.

Requesting /static/test.php returns static, because the selected /static/ prefix has ^~. The same suffix regex therefore does not take that request. An unmatched path returns the broad fallback marker. Together, these cases demonstrate the actual selection boundaries rather than merely proving that nginx accepts the configuration syntax.

The client sends the paths explicitly and checks response bodies as well as statuses. The fixture contains no real authentication policy or script execution, so its markers validate routing decisions only. When applying the pattern to a protected endpoint, test the actual handler and access behavior that will run after selection.

Review the selected handler and its later transitions

Start with the intended URL families, then list exact endpoints, descendant prefixes, and any overlapping regexes. Include near-miss paths such as /apix, case variants where relevant, and encoded or normalized forms. nginx matches a normalized URI, so a security review based only on the raw request text can miss which configuration ultimately applies.

Inspect the effective configuration, including included files and the selected virtual server. A correct location in a server block that the request never enters cannot fix routing in another block. Use controlled response markers or logs during verification rather than inferring selection from a familiar page returned by several handlers.

Finally, account for rewrites, index handling, and other internal redirects that can trigger a new location search. The first selected block may not be the final handler. Test the full request path and preserve enough evidence to explain both the initial match and the response actually produced.

Sources

nginx core location directive, normalized URI matching, exact matches, prefixes, and regex selection, checked 2026-09-12.
How nginx processes a request, server selection and location-based request handling, checked 2026-09-12.

Anycast serves the nearest replica

Mon, 01 Nov 2027 00:00:00 +0000

Anycast gives an address several homes. Multiple sites announce the same prefix, and each network selects one usable path according to its BGP policy. A client usually lands at a topologically close replica without first contacting a global load balancer. When one site withdraws its announcement, the control plane can converge on another home.

One address, many announcements

An anycast service assigns the same unicast address or prefix to independent instances. Each edge router advertises reachability with BGP. Neighbors compare paths and install a best route in their local routing information base; packets then follow ordinary forwarding toward that selected next hop. The application sees one destination even though several sites can answer it.

“Nearest” means nearest according to routing policy and topology, not a guaranteed geographic distance. A higher local preference can beat a physically shorter path. If local preference ties, a shorter AS path is generally preferred, followed by more vendor- and policy-specific tie breakers. Operators can steer traffic by changing these attributes, prepending an AS path, or withdrawing a prefix. The important contract is reachability, not a promise that every client chooses the same site.

Routing performs the failover

If a site loses its service or upstream, its route should be withdrawn, often by a health-controlled session. Upstream routers propagate the withdrawal and select a surviving announcement. The forwarding path does not need a DNS answer to change, so a whole edge location can disappear while the address remains valid. BGP convergence is not instantaneous: packets can be lost during the transition, and different networks may converge at different times. Clients still need retries and sensible connection timeouts.

Anycast is especially useful for stateless or easily re-established protocols such as DNS, UDP-based discovery, and CDN edges. It also spreads an attack surface across sites, although capacity and filtering must be coordinated. A route that remains advertised while the process is dead is worse than no route: traffic is sent to a black hole. Tie route announcements to a real readiness signal and monitor both control-plane reachability and application responses.

Stateful sessions mind the move

An established TCP connection can reach a different site after a route change. Without the old socket state, that instance may reset or drop it. A shared backend preserves application state, not the TCP connection; clients still need reconnect-and-rebuild behavior. Idempotency keys can prevent repeated side effects after a reconnect. Do not infer stickiness from one successful trace: topology changes can select another replica even during a flow.

Observability must include the serving site, route age, and withdrawal reason. A single global success rate can hide one region black-holing traffic while other regions look healthy. Sample from several networks, record the anycast address separately from the site identity, and alert when a site is still advertised but its service checks fail.

When it breaks

A route leak or an accidental high local preference can attract traffic far from the intended site. A partial withdrawal can leave some peers on a stale path, producing asymmetric loss rather than a clean failover. MTU, firewall, and ACL differences between replicas create failures that look like random routing. If the service has local state, users may see resets only after the control plane has already converged.

Avoid advertising a prefix from a site until the process, dependencies, and return path are ready. Test withdrawal from a staging address, measure convergence from more than one vantage point, and keep a non-anycast management path for recovery. Routing is the selection mechanism; it is not an application health check unless the announcement is explicitly coupled to one.

Check it quickly

This local Go model captures only local preference and AS-path length for one prefix. It tests path tie-breaking, input-order independence, policy override, withdrawal, an empty candidate set, and backup recovery. No packets or BGP session leave the process.

package main

import "fmt"

type route struct {
 name string
 localPref int
 asPath int
 advertised bool
}

// This fixture models only local preference and AS-path length for one prefix.
func best(routes []route) (route, bool) {
 var chosen route
 for _, candidate := range routes {
 if !candidate.advertised {
 continue
 }
 if chosen.name == "" || candidate.localPref > chosen.localPref ||
 (candidate.localPref == chosen.localPref && candidate.asPath < chosen.asPath) {
 chosen = candidate
 }
 }
 return chosen, chosen.name != ""
}

func want(ok bool, message string) {
 if !ok {
 panic(message)
 }
}

func main() {
 routes := []route{
 {name: "tokyo", localPref: 100, asPath: 2, advertised: true},
 {name: "singapore", localPref: 100, asPath: 3, advertised: true},
 }
 chosen, ok := best(routes)
 want(ok && chosen.name == "tokyo", "shorter AS path should win the tie")
 chosen, ok = best([]route{routes[1], routes[0]})
 want(ok && chosen.name == "tokyo", "input order must not beat path length")
 routes[1].localPref = 200
 chosen, ok = best(routes)
 want(ok && chosen.name == "singapore", "local policy should override path length")
 routes[1].advertised = false
 chosen, ok = best(routes)
 want(ok && chosen.name == "tokyo", "withdrawal should select the surviving site")
 routes[0].advertised = false
 _, ok = best(routes)
 want(!ok, "with every announcement withdrawn there is no route")
 _, ok = best(nil)
 want(!ok, "an empty candidate set must not invent a route")
 fallback := route{name: "sydney", localPref: 50, asPath: 4, advertised: true}
 chosen, ok = best(append(routes, fallback))
 want(ok && chosen.name == "sydney", "a backup announcement should recover reachability")
 fmt.Println("PASS: path length, policy priority, withdrawal, empty set, and backup")
}

Save this as anycast_model.go and run go run anycast_model.go. The expected output is PASS: path length, policy priority, withdrawal, empty set, and backup. Check formatting with gofmt -d, static diagnostics with go vet, and execution with go run -race. The model omits MED, eBGP/iBGP rules, communities, topology, and convergence timing. It proves selection among given candidates, not convergence on the Internet.

Sources

RFC 4271: A Border Gateway Protocol 4, route selection and path attributes, checked 2026-09-09.
RFC 4786: Operation of Anycast Services, deployment, routing, and failure considerations, checked 2026-09-09.

BGP aggregation shrinks the routing table

Fri, 08 Oct 2027 00:00:00 +0000

Aggregation replaces a set of adjacent prefixes with one covering announcement, so every peer downstream stores one entry instead of dozens and stops processing updates for each component. The table gets smaller, and the churn of a flapping edge stops propagating outward.

The tension is that a summary is a claim about reachability. Once the covering prefix is in the table, the forwarding engine acts on it for addresses whose component routes are gone, so a smaller table is bought with a blurrier failure signal.

What aggregation actually buys

A peer receiving 10.1.4.0/24, 10.1.7.0/24, 10.1.9.0/24 and a covering 10.1.0.0/16 stores four entries. Configure aggregate-address 10.1.0.0/16 summary-only and the peer stores one. Suppression happens on egress: the more-specifics stay in the aggregator’s own table and keep forwarding its own traffic, they are simply not advertised. FRR is strict about this — a more-specific must exist in the BGP table, not merely in the RIB, before the aggregate is originated at all.

The second prize is churn. Every flap of a component is an update the peer must parse, select and reinstall. Behind a summary those updates stop at the aggregator, so a customer network oscillating twice a minute costs the rest of the internet nothing. On a flapping edge that is the property wanted.

summary-only, suppress-map and as-set

summary-only suppresses every longer prefix to every neighbour. suppress-map suppresses only the more-specifics a route map matches, for “summarise this block but keep that one exception”. as-set changes what the path looks like: instead of collapsing components into the aggregator’s own AS, it emits an AS_SET of the real origin ASes.

as-set preserves loop detection but damages everything else. An AS_SET lengthens the path for AS-path comparison, gives origin validation nothing to check, and is being deprecated — FRR ships bgp reject-as-sets to drop such routes both ways. The genuinely dangerous configuration is aggregating across different origins at all. Folding 10.1.4.0/24 from AS64501 and 10.1.7.0/24 from AS64502 into one 10.1.0.0/16 originated by AS64500 tells every peer that AS64500 originates the whole block. A neighbour filtering on origin now accepts a prefix it should not, and one filtering on AS path no longer sees the customers. Detail is destroyed, not compressed.

Longest prefix match is the only forwarding rule

BGP decides reachability; the FIB decides forwarding, and the FIB has one rule — longest mask wins. The aggregate is never preferred or depreferenced, and the more-specific wins whenever it exists. Replaying this over a built table: with both present, a packet to 10.1.4.9 matches 10.1.4.0/24 via AS_PATH 64500 64501. Withdraw that /24 while the aggregate survives and the same packet matches 10.1.0.0/16, with no change whatsoever to the aggregate. Withdraw the aggregate as well and the packet finally becomes “no route, ICMP unreachable”.

That is why announcing an aggregate without its more-specifics changes the forwarding path, not merely the table size. The summary is a standing promise and the FIB enforces it. It cuts the other way too: a more-specific is a traffic magnet, so whatever you announce more specifically receives that traffic no matter what the covering prefix claims.

When it breaks

The classic failure is a summary advertised while a more-specific has been withdrawn somewhere else. The aggregate outlives its component, so traffic for the dead block is drawn to the aggregator and dropped inside it — a blackhole with no ICMP, no withdrawal to notice, no counter at the edge. In the replay, 10.1.4.9 moves from the /24 to the /16 and is silently discarded. It presents as “some destinations in the /16 are unreachable and everything else is fine”, which sends the investigation to the wrong layer.

The second failure is the flap that opens a temporary blackhole. Withdraw a component, re-announce it two seconds later, and for those two seconds every packet in that range matches the aggregate and dies. Blast radius scales with how much address space sits behind the summary: summarise fifty thousand customer addresses behind one /16 and one customer’s flap blacks out all fifty thousand. Mitigate by not summarising across origins, by keeping the more-specifics advertised and paying for the entries, or by using an unreachable-prefix announcement so the withdrawal propagates a discard instead of silence.

Check it quickly

Confirm the aggregate and its components still agree:

show bgp ipv4 unicast 10.1.0.0/16 longer-prefixes
show bgp ipv4 unicast 10.1.4.0/24
show bgp ipv4 unicast 10.1.0.0/16
show ip route 10.1.4.9

The first lists every component under the summary; the second proves the suspect one is still there. The third is where a lost origin shows up as AS_SET or a bare aggregator AS, and the fourth shows what the FIB will really match — if it prints the /16 for an address whose /24 you expected, the component is gone and the aggregate is lying.

Sources

RFC 4271 BGP-4, defines ATOMIC_AGGREGATE, AGGREGATOR and AS_SET, checked 2026-09-16.
FRR BGP route aggregation, documents aggregate-address, summary-only, suppress-map, as-set and unreachable-prefix announcement, checked 2026-09-16.
RFC 4632 CIDR, the classless prefix and longest-match forwarding model, checked 2026-09-16.

A default route matches broadly; it does not rescue every failed path

Tue, 14 Sep 2027 00:00:00 +0000

An IPv4 default route is the prefix 0.0.0.0/0: it matches every IPv4 destination, with the least possible specificity. In a selected routing table, a longer matching prefix normally takes precedence. The default is not a procedure that retries whenever a more specific route fails. Routing policy selects relevant tables, and the chosen route describes a forwarding decision rather than promising that the entire round trip will succeed.

When it breaks

A machine without a default can still reach destinations covered by connected or explicit routes. Conversely, adding a default does not prove internet access. The next hop needs to be reachable, forwarding and filtering must permit the traffic, and replies need a return path. A displayed gateway is only one part of that contract; application communication supplies the evidence that the configured path works.

A more specific rejection route wins over a working default. Linux route types such as unreachable intentionally stop forwarding for matching destinations. A failed neighbor lookup through a selected gateway also does not mean the kernel will automatically search for a healthier, less specific path. Static route preference and metrics are not a service-health monitoring or failover system.

Return-path errors are particularly misleading. The client can select the expected gateway and send traffic toward a listening server, yet fail because the server cannot route replies to the client. Repeatedly changing the client’s default may damage working destinations without repairing that missing reverse path. Inspect both directions and the actual source address before attributing every connection timeout to the sender’s gateway.

Check it quickly

Save this as check-default-route.py and run python3 check-default-route.py with Docker. Python 3.11.16 and iproute2 6.1.0 build a client–router–target topology inside one disposable Linux container. NET_ADMIN and SYS_ADMIN support its nested namespaces and temporary proc mount. All interfaces are newly created veth devices, and no port or route is published on the host:

from pathlib import Path
from tempfile import TemporaryDirectory
import json,subprocess,uuid
recipe='FROM python@sha256:528257d48c1da0dcecc2e725d1ae34498d60c965f1241e39cd6a85a8859bdf84\nRUN apt-get update && apt-get install -y --no-install-recommends iproute2=6.1.0-3 iputils-ping=3:20221126-1+deb12u1 nftables=1.0.6-2+deb12u2 tcpdump=4.99.3-1 traceroute=1:2.1.2-1 procps=2:4.0.2-3 && rm -rf /var/lib/apt/lists/*\nCMD ["python3"]\n'
program=r"""
from contextlib import ExitStack,contextmanager
from pathlib import Path
from tempfile import TemporaryDirectory
import json,os,select,signal,subprocess,time

def run(*args,check=True,input=None,timeout=8):
 result=subprocess.run(args,input=input,capture_output=True,text=True,timeout=timeout)
 if check:assert result.returncode==0,(args,result.returncode,result.stdout,result.stderr)
 return result
def ns(name,*args,**kwargs):return run('ip','netns','exec',name,*args,**kwargs)
def ip(name,*args,**kwargs):return ns(name,'ip',*args,**kwargs)
def stop(process):
 if process.poll() is None:
 process.terminate()
 try:process.wait(timeout=2)
 except subprocess.TimeoutExpired:process.kill();process.wait(timeout=2)
def line(process,timeout=3):
 assert select.select([process.stdout],[],[],timeout)[0],'child response deadline exceeded'
 value=process.stdout.readline();assert value,'child exited without a response'
 return value
def spawn(stack,command):
 process=subprocess.Popen(command,stdin=subprocess.PIPE,stdout=subprocess.PIPE,stderr=subprocess.PIPE,text=True)
 stack.callback(stop,process);return process
@contextmanager
def lab(names):
 with TemporaryDirectory(prefix='net-path-') as directory,ExitStack() as stack:
 root=Path(directory);proc=root/'proc';proc.mkdir();run('mount','-t','proc','proc',str(proc))
 stack.callback(run,'umount',str(proc))
 for name in names:
 run('ip','netns','add',name);stack.callback(run,'ip','netns','delete',name,check=False)
 ip(name,'link','set','lo','up')
 def setting(name,key,value):
 assert name in names
 path=proc/'sys'/key.replace('.','/')
 return ns(name,'python3','-c',f'from pathlib import Path; p=Path({str(path)!r}); p.write_text({str(value)!r}); print(p.read_text().strip())').stdout.strip()
 yield root,stack,setting
def wire(left,left_if,right,right_if,left_addresses,right_addresses):
 run('ip','link','add',left_if,'type','veth','peer','name',right_if)
 for name,interface,addresses in [(left,left_if,left_addresses),(right,right_if,right_addresses)]:
 run('ip','link','set',interface,'netns',name)
 for address in addresses:ip(name,'address','add',address,'dev',interface,*(['nodad'] if ':' in address else []))
 ip(name,'link','set',interface,'up')
def nft(name,script):return ns(name,'nft','-f','-',input=script)
def counters(name):
 data=json.loads(ns(name,'nft','-j','list','ruleset').stdout);found={}
 for entry in data['nftables']:
 rule=entry.get('rule',{})
 if 'comment' in rule:
 for expression in rule['expr']:
 if 'counter' in expression:found[rule['comment']]=expression['counter']['packets']
 return found

def server(stack,name,address,port=18080,label='server'):
 code=r'''
from http.server import BaseHTTPRequestHandler,ThreadingHTTPServer
import json,socket,sys
address,port,label=sys.argv[1],int(sys.argv[2]),sys.argv[3]
class Handler(BaseHTTPRequestHandler):
 def do_GET(self):
 body=json.dumps({'server':label,'peer':self.client_address,'path':self.path}).encode()
 self.send_response(200);self.send_header('Content-Length',str(len(body)));self.end_headers();self.wfile.write(body)
 def log_message(self,*args):pass
class Server(ThreadingHTTPServer):
 address_family=socket.AF_INET6 if ':' in address else socket.AF_INET
 def server_bind(self):
 if self.address_family==socket.AF_INET6:self.socket.setsockopt(socket.IPPROTO_IPV6,socket.IPV6_V6ONLY,1)
 super().server_bind()
service=Server((address,port),Handler);print('ready',flush=True);service.serve_forever()
'''
 process=spawn(stack,['ip','netns','exec',name,'python3','-u','-c',code,address,str(port),label])
 assert line(process).strip()=='ready';return process
def request(name,address,port=18080):
 code=r'''
import http.client,json,sys
client=http.client.HTTPConnection(sys.argv[1],int(sys.argv[2]),timeout=.7)
try:
 client.request('GET','/lab-visible-message');response=client.getresponse();body=json.loads(response.read())
 assert response.status==200;print(json.dumps({'ok':True,'status':response.status,'body':body}))
except OSError as error:print(json.dumps({'ok':False,'type':type(error).__name__,'errno':error.errno}))
finally:client.close()
'''
 return json.loads(ns(name,'python3','-c',code,address,str(port)).stdout)

def routed(setting):
 wire('client','c0','router','r0',['10.201.1.2/30'],['10.201.1.1/30'])
 wire('router','r1','target','t0',['10.201.2.1/30'],['10.201.2.2/30'])
 setting('router','net.ipv4.ip_forward',1)
 ip('client','route','add','default','via','10.201.1.1')
 ip('target','route','add','default','via','10.201.2.1')
with lab(['client','router','target']) as (root,stack,setting):
 routed(setting);ip('client','route','delete','default')
 ip('target','addr','add','198.51.100.9/32','dev','lo');ip('router','route','add','198.51.100.9/32','via','10.201.2.2')
 daemon=server(stack,'target','198.51.100.9')
 connected=ns('client','ping','-n','-c','1','-W','1','10.201.1.1');assert connected.returncode==0
 missing=request('client','198.51.100.9');assert not missing['ok'] and missing['errno']==101
 ip('client','route','add','default','via','10.201.1.1')
 chosen=json.loads(ip('client','-j','route','get','198.51.100.9').stdout)[0];assert chosen['gateway']=='10.201.1.1'
 healthy=request('client','198.51.100.9');assert healthy['ok']
 ip('client','route','add','unreachable','198.51.100.9/32')
 specific=request('client','198.51.100.9');assert not specific['ok'] and specific['errno']==113
 ip('client','route','delete','unreachable','198.51.100.9/32')
 recovered=request('client','198.51.100.9');assert recovered['ok']
 ip('target','route','delete','default')
 bad_return=request('client','198.51.100.9');assert not bad_return['ok'] and bad_return['type']=='TimeoutError'
 assert json.loads(ip('client','-j','route','get','198.51.100.9').stdout)[0]['gateway']=='10.201.1.1'
 ip('target','route','add','default','via','10.201.2.1')
 final=request('client','198.51.100.9');assert final['ok']
 print(json.dumps({'connected_ping_without_default':0,'no_default':missing,'chosen_route':chosen,'with_default':healthy,
 'specific_unreachable_route':specific,'specific_removed':recovered,'missing_return_route':bad_return,'return_route_restored':final}))
"""
name='xbcatm-netpath-'+uuid.uuid4().hex[:10]
def docker(*args,**kwargs):return subprocess.run(['docker',*args],capture_output=True,text=True,timeout=300,**kwargs)
try:
 with TemporaryDirectory(prefix=name+'-') as directory:
 root=Path(directory);(root/'Dockerfile').write_text(recipe)
 build=docker('build','-t',name,str(root));assert build.returncode==0,build.stderr
 result=docker('run','--rm','-i','--name',name,'--network','none','--cap-add','NET_ADMIN','--cap-add','SYS_ADMIN',
 '--memory','256m','--pids-limit','128',name,'python3','-',input=program)
 assert result.returncode==0,result.stderr
 print(json.dumps(json.loads(result.stdout),sort_keys=True))
finally:
 docker('rm','-f',name);docker('image','rm',name)

The client uses 10.201.1.2, with router neighbor 10.201.1.1. The target sits behind the router on another link and owns service address 198.51.100.9 on loopback. The router has a specific route to that service, and the target initially has a return default. The documentation address is used only within this isolated topology; no public server is contacted.

With the client’s default removed, pinging its directly connected router succeeds. An HTTP connection to the remote service fails with Linux ENETUNREACH, errno 101. Adding a default through 10.201.1.1 changes the route lookup to that gateway, and an actual HTTP request returns status 200. This demonstrates both the narrower connectivity that already existed and the additional path enabled by the default.

An unreachable /32 route is then installed for 198.51.100.9. The same connection now fails with EHOSTUNREACH, errno 113, despite the default still being present. Removing that specific route restores HTTP success. The kernel honored the more specific rejection; it did not treat the error as an invitation to fall back to the default gateway.

Next, the script removes only the target’s default route. The client’s lookup still names 10.201.1.1, but a new HTTP attempt reaches its timeout. Restoring the target’s default restores communication. This controlled comparison isolates the return-path dependency while leaving the client’s choice unchanged, providing stronger evidence than a one-sided route listing or an unexplained failed ping.

Inspect the exact flow before editing defaults

Record source address, destination, relevant policy rules, chosen interface, and gateway. A machine with several interfaces or routing tables can make different decisions for different sources, marks, or routing domains. Run diagnostic lookups with the application’s relevant inputs. The fixture uses ordinary destination-based routing and does not claim to reproduce every policy-routing or multipath configuration.

Check the next hop’s link reachability, the router’s forwarding decision, and the target’s route back to the actual source. NAT or a proxy may change which source the target sees. Error classes can guide this work: immediate local network-unreachable errors differ from an explicitly unreachable host route and from a deadline expiring while a reply cannot return. Keep those distinctions in incident evidence.

Defaults are useful because they summarize destinations whose individual routes the host need not know. Configure them deliberately, verify exceptions, and test representative round trips after changes. If automatic failover is required, use a mechanism that observes the relevant path and updates routing appropriately. A second configured gateway or a preferred metric alone does not establish that behavior.

Sources

Linux ip-route(8), default routes, route lookup and unreachable route types, checked 2026-09-12.
RFC 4632 CIDR, longest-prefix matching and route specificity, checked 2026-09-12.
RFC 1812 IPv4 router requirements, IPv4 forwarding and destination reachability, checked 2026-09-12.

BGP is the Internet routing table

Sun, 12 Sep 2027 00:00:00 +0000

BGP is the only protocol that stitches tens of thousands of independently operated networks into one forwarding table. It carries reachability, but what it really carries is policy: each network says which prefixes it originates, which neighbours it learned them from, and — through filters — which ones it is willing to pass on.

The tension is that this table is built on trust between parties who have no contract with each other. Nothing validates an announcement before it propagates, so a typo or a misconfigured filter is global within minutes and stays that way until a human notices.

The AS path is a loop check and a policy signal at once

On egress an eBGP speaker prepends its own AS number, and a receiver drops any update whose AS_PATH already contains its own AS. That is the entire loop-prevention mechanism, and it is why a route cannot be re-imported around a circle. iBGP does not prepend, which is why an iBGP-learned route carries the same AS_PATH as the eBGP route that brought it into the AS, and why an AS needs a full iBGP mesh or route reflectors to distribute it.

The same field does two more jobs. Operators filter on it — reject anything containing a peer’s AS, accept only routes whose path ends in a customer’s AS — and path selection compares its length. One field, three responsibilities, and they conflict: prepending to make a path less attractive also makes it look like a route your filters should reject.

Why the table is about a million prefixes

The real numbers, from the CIDR Report snapshot taken for this article: 1,080,189 IPv4 prefixes, or 597,258 if you collapse everything that can be aggregated. Amazon alone originates 18,025 prefixes, of which 15,759 are more-specifics of larger blocks. Roughly half the table is more-specifics of address space that is already announced.

That is a policy artifact, not a topology artifact. Multihoming creates a more-specific per provider, traffic engineering creates them per path, and a customer who wants their own routing decisions creates one more. Aggregation would halve the table, and nobody does it, because every one of those more-specifics is doing a job.

Path selection is a fixed order, not a distance

BGP picks a path by walking a list and stopping at the first step that breaks the tie: highest weight, then highest local preference, then locally originated, then shortest AS path, then lowest origin type, then lowest MED, then eBGP over iBGP, then lowest IGP cost to the next hop, then lowest router ID. Weight and local preference are local to one router or one AS and never travel in an update.

Running that over four candidate paths for one prefix: the decision lands on weight, because a weight of 100 beats 0 before anything else is read. Remove the weights and the decision moves to local preference. Remove local preference as well and it moves to AS-path length, where a two-AS path beats a three-AS path. The shortest path is step four of nine, which is why “BGP picks the shortest route” is not a useful statement.

When it breaks

The classic incident shape is a leaked more-specific that hijacks traffic. An unrelated AS originates 198.51.100.128/25 inside someone else’s 198.51.100.0/24, and longest-prefix-match hands it exactly half the victim’s address space — 128 of 256 addresses — with no withdrawal, no error, and the legitimate /24 still sitting in every table. It presents as partial unreachability and traffic appearing somewhere unexpected, and it is the shape documented in RFC 7908. RPKI origin validation is the mitigation, but it only answers whether an AS is allowed to originate a prefix; a ROA’s maxLength must cover the more-specific, and a forged AS path with a valid origin still validates.

The second failure is the table outgrowing the hardware. A device with 1,000,000 IPv4 FIB entries is already 80,189 prefixes short of today’s table; at 1,200,000 it fits. When a FIB fills, the router keeps the sessions up and stops installing routes — some prefixes move to software forwarding, others are dropped, and the blast radius is that unrelated destinations stop working while BGP looks perfectly healthy.

Check it quickly

Confirm what your router actually believes about a prefix:

show bgp ipv4 unicast summary
show bgp ipv4 unicast 198.51.100.0/24
show bgp ipv4 unicast 198.51.100.0/24 longer-prefixes
show ip route 198.51.100.200

The first prints the total prefix count, which is how you notice a peer dumping routes at you. The second and third together show whether a more-specific exists and who originates it — a /25 you do not recognise is the whole incident. The fourth is the verdict the forwarding engine reached, and if it disagrees with what you expected from the first three, the hijack is already live.

Sources

RFC 4271 BGP-4, the decision process and AS path loop detection, checked 2026-09-16.
RFC 6811 prefix origin validation, what RPKI validation does and does not prove, checked 2026-09-16.
RFC 7908 route leaks, the problem statement for leaked announcements, checked 2026-09-16.
CIDR Report, current IPv4 table size and per-AS more-specific counts, checked 2026-09-16.

proxy_pass maps the URI as well as selecting an upstream

Fri, 02 Apr 2027 00:00:00 +0000

nginx proxy_pass chooses an upstream destination and also affects the request URI sent there. The important distinction is whether the directive includes a URI component, not simply whether its text ends with a slash. In an ordinary prefix location, a literal target URI replaces the matching normalized prefix. An authority-only target can pass the original request URI when the request has not been rewritten.

When it breaks

An application serving /users may return 404 when the proxy forwards /api/users, even though the network connection succeeds. The opposite mismatch occurs when an application expects the public prefix and the proxy removes it. Comparing only client status and upstream reachability misses this class of error; inspect the exact method, path, query, and host the backend receives.

Slash boundaries are part of literal replacement. A target ending in /v1 still contains a URI, so replacing /join/ in /join/item yields /v1item. It does not preserve the original path merely because the target lacks a trailing slash. Thinking in terms of the replaced prefix and remaining suffix explains this result more accurately than a universal trailing-slash slogan.

Variables introduce another documented branch. When a variable-supplied proxy URL contains a URI, that URI is used as specified instead of applying the ordinary literal-prefix replacement rule. Regex and named locations also lack the same unambiguous prefix boundary. Configuration that works in a plain prefix location cannot be copied into those contexts without checking the corresponding contract.

Check it quickly

Run the script with host Python, Docker, and the pinned nginx and Python images available. The recorded environment used nginx 1.28.0 and Python 3.11.16 on Linux arm64 with Docker 29.3.1. It starts an actual HTTP backend that echoes received request fields, validates nginx configuration, publishes only the proxy on host loopback, and removes its own images, containers, and network:

from contextlib import contextmanager
from pathlib import Path
from tempfile import TemporaryDirectory
import http.client,json,socket,subprocess,time,uuid

NGINX='nginx@sha256:552e7481ca93ffccd046aa658dbbed22caefbc09c66fa7cd247cbb90b8a5c609'
PYTHON='python@sha256:528257d48c1da0dcecc2e725d1ae34498d60c965f1241e39cd6a85a8859bdf84'
def docker(*args,check=True):
 result=subprocess.run(['docker',*args],capture_output=True,text=True,timeout=120)
 if check:assert result.returncode==0,result.stderr
 return result
def wait(predicate):
 deadline=time.monotonic()+15
 while time.monotonic()<deadline:
 try:
 if predicate():return
 except OSError:pass
 time.sleep(.1)
 raise AssertionError('fixture did not become ready')

@contextmanager
def lab(configuration,files=None,backend=None):
 name='xbcatm-nginx-'+uuid.uuid4().hex[:10];tags=[name+':proxy',name+':backend'];containers=[name+'-proxy',name+'-backend']
 try:
 with TemporaryDirectory(prefix=name+'-') as directory:
 root=Path(directory);(root/'files').mkdir();(root/'nginx.conf').write_text(configuration)
 for path,content in (files or {}).items():
 target=root/'files'/path;target.parent.mkdir(parents=True,exist_ok=True);target.write_text(content)
 docker('network','create',name)
 if backend is not None:
 (root/'backend.py').write_text(backend)
 (root/'backend.Dockerfile').write_text(f'FROM {PYTHON}\nCOPY backend.py /backend.py\nCMD ["python", "/backend.py"]\n')
 docker('build','--network','none','-f',str(root/'backend.Dockerfile'),'-t',tags[1],str(root))
 docker('run','-d','--name',containers[1],'--network',name,'--network-alias','backend',tags[1])
 wait(lambda:docker('exec',containers[1],'python','-c','import socket;s=socket.create_connection(("127.0.0.1",8081),timeout=.5);s.close()',check=False).returncode==0)
 (root/'Dockerfile').write_text(f'FROM {NGINX}\nCOPY nginx.conf /etc/nginx/nginx.conf\nCOPY files /srv\n')
 docker('build','--network','none','-t',tags[0],str(root))
 docker('run','--rm','--network',name,tags[0],'nginx','-t')
 docker('run','-d','--name',containers[0],'--network',name,'-p','127.0.0.1::8080',tags[0])
 details=json.loads(docker('inspect',containers[0]).stdout)[0]
 binding=details['NetworkSettings']['Ports']['8080/tcp'][0];assert binding['HostIp']=='127.0.0.1';port=int(binding['HostPort'])
 def ready():
 connection=socket.create_connection(('127.0.0.1',port),timeout=.5);connection.close();return True
 wait(ready)
 def request(path,method='GET',headers=None,body=None):
 connection=http.client.HTTPConnection('127.0.0.1',port,timeout=5)
 try:
 connection.request(method,path,body=body,headers=headers or {})
 response=connection.getresponse();return response.status,{k.lower():v for k,v in response.getheaders()},response.read()
 finally:connection.close()
 yield containers[0],request
 finally:
 docker('rm','-f',*containers,check=False);docker('image','rm',*tags,check=False);docker('network','rm',name,check=False)

def config(locations,extra=''):
 return 'worker_processes 1;\nerror_log /tmp/error.log notice;\nevents {}\nhttp {\naccess_log off;\n'+extra+'\nserver { listen 8080; root /srv;\n'+locations+'\n}\n}\n'

backend=r'''from http.server import BaseHTTPRequestHandler,HTTPServer
import json
class Handler(BaseHTTPRequestHandler):
 def do_GET(self):
 body=self.rfile.read(int(self.headers.get('Content-Length','0')))
 document={'path':self.path,'method':self.command,'body':body.decode(),'headers':{key:self.headers.get(key) for key in ['Host','X-Forwarded-For','X-Real-IP','X-Forwarded-Proto','X-Forwarded-Host']}}
 data=json.dumps(document).encode()
 self.send_response(200);self.send_header('Content-Type','application/json');self.send_header('Content-Length',str(len(data)));self.send_header('X-Backend-Info','fixture-node');self.end_headers();self.wfile.write(data)
 do_POST=do_GET
 def log_message(self,*args):pass
HTTPServer(('0.0.0.0',8081),Handler).serve_forever()
'''

locations=r'''
location /keep/ { proxy_pass http://backend:8081; }
location /strip/ { proxy_pass http://backend:8081/; }
location /replace/ { proxy_pass http://backend:8081/v1/; }
location /join/ { proxy_pass http://backend:8081/v1; }
location /variable/ { set $destination "http://fixture_pool/fixed?chosen=1"; proxy_pass $destination; }
'''
invalid=config(r'location ~ ^/bad/ { proxy_pass http://backend:8081/replacement/; }')
with lab(config(locations,'upstream fixture_pool { server backend:8081; }'),{'invalid.conf':invalid},backend) as (container,request):
 cases={'/keep/item?x=1':'/keep/item?x=1','/strip/item?x=1':'/item?x=1','/replace/item?x=1':'/v1/item?x=1','/join/item?x=1':'/v1item?x=1','/keep/a%2Fb?x=1':'/keep/a%2Fb?x=1','/strip/a%2Fb?x=1':'/a/b?x=1','/variable/item?original=1':'/fixed?chosen=1'}
 for path,expected in cases.items():
 status,headers,body=request(path);assert status==200 and json.loads(body)['path']==expected,(path,status,body)
 invalid_result=docker('exec',container,'nginx','-t','-c','/srv/invalid.conf',check=False)
 assert invalid_result.returncode!=0 and 'cannot have URI part in location given by regular expression' in invalid_result.stderr
 print('no URI preserves the original path; a literal URI replaces the normalized matching prefix, including its slash boundaries')
 print('a variable-supplied URI is used as specified; regex-location URI replacement is rejected by nginx -t')

An authority-only target in /keep/ forwards /keep/item?x=1 unchanged. A target ending in / inside /strip/ instead forwards /item?x=1. The query survives both of these literal examples. The backend’s JSON response supplies the evidence, rather than an assumption based on what the public URL looked like before nginx handled it.

The /replace/ location maps its prefix to /v1/, so /replace/item?x=1 becomes /v1/item?x=1. The deliberately missing separator in the /join/ target produces /v1item?x=1. Both requests reach the backend successfully. Their distinct paths demonstrate why a successful TCP connection or HTTP response does not establish the intended route mapping.

Encoded separators expose the normalization boundary. /keep/a%2Fb?x=1 remains encoded in the authority-only case, while /strip/a%2Fb?x=1 reaches the backend as /a/b?x=1 after normalized prefix replacement. This can change how an application divides path segments. Align the proxy and application’s path interpretation when encoded identifiers or access-control boundaries depend on those separators.

The variable example names a predefined upstream group and supplies /fixed?chosen=1 as its target URI. Requesting /variable/item?original=1 reaches exactly that fixed URI; the original suffix and query are not automatically appended. Using a known upstream group also keeps this example separate from arbitrary per-request hostname resolution and its resolver requirements.

A second configuration deliberately puts a proxy target with a URI part inside a regex location. nginx -t rejects it with the documented URI-part restriction. The active configuration is not replaced by this invalid file. This verifies a real configuration failure in addition to the successful routing cases.

Treat each public-to-backend mapping as a contract

For every proxy location, write down the public prefix, replacement URI if any, and expected backend path and query. Include boundary slashes and encoded forms in tests. If a rewrite changes the URI before proxying, revisit authority-only behavior too: forwarding the original request is conditional on processing the original URI, not an unconditional promise for every rewritten request.

Keep destination selection constrained to intended backends. A variable derived from untrusted request data should not become unrestricted authority to contact arbitrary hosts. Named upstream groups, explicit mappings, and appropriate rejection behavior should follow the application’s actual routing requirements rather than silently accepting an unknown destination.

Configure forwarded headers, connection reuse, timeouts, and upstream TLS according to their own contracts. A resolver directive is not a general switch that makes variable targets or keepalive universally correct. Test those behaviors separately and retain the backend observations that prove the selected URI mapping.

Sources

nginx proxy_pass reference, URI replacement, unchanged request forwarding, variable targets, and regex restrictions, checked 2026-09-12.
nginx core location matching, normalized prefix selection before proxy processing, checked 2026-09-12.
nginx upstream module, named upstream groups and backend selection, checked 2026-09-12.

Routing · XB Field Notes

An nginx prefix is a candidate until matching finishes

When it breaks

Check it quickly

Review the selected handler and its later transitions

Sources

Anycast serves the nearest replica

One address, many announcements

Routing performs the failover

Stateful sessions mind the move

When it breaks

Check it quickly

Further reading

Sources

BGP aggregation shrinks the routing table

What aggregation actually buys

summary-only, suppress-map and as-set

Longest prefix match is the only forwarding rule

When it breaks

Check it quickly

Sources

A default route matches broadly; it does not rescue every failed path

When it breaks

Check it quickly

Inspect the exact flow before editing defaults

Sources

BGP is the Internet routing table

The AS path is a loop check and a policy signal at once

Why the table is about a million prefixes

Path selection is a fixed order, not a distance

When it breaks

Check it quickly

Sources

proxy_pass maps the URI as well as selecting an upstream

When it breaks

Check it quickly

Treat each public-to-backend mapping as a contract

Sources