Anycast serves the nearest replica

Mon, 01 Nov 2027 00:00:00 +0000

Anycast gives an address several homes. Multiple sites announce the same prefix, and each network selects one usable path according to its BGP policy. A client usually lands at a topologically close replica without first contacting a global load balancer. When one site withdraws its announcement, the control plane can converge on another home.

One address, many announcements

An anycast service assigns the same unicast address or prefix to independent instances. Each edge router advertises reachability with BGP. Neighbors compare paths and install a best route in their local routing information base; packets then follow ordinary forwarding toward that selected next hop. The application sees one destination even though several sites can answer it.

“Nearest” means nearest according to routing policy and topology, not a guaranteed geographic distance. A higher local preference can beat a physically shorter path. If local preference ties, a shorter AS path is generally preferred, followed by more vendor- and policy-specific tie breakers. Operators can steer traffic by changing these attributes, prepending an AS path, or withdrawing a prefix. The important contract is reachability, not a promise that every client chooses the same site.

Routing performs the failover

If a site loses its service or upstream, its route should be withdrawn, often by a health-controlled session. Upstream routers propagate the withdrawal and select a surviving announcement. The forwarding path does not need a DNS answer to change, so a whole edge location can disappear while the address remains valid. BGP convergence is not instantaneous: packets can be lost during the transition, and different networks may converge at different times. Clients still need retries and sensible connection timeouts.

Anycast is especially useful for stateless or easily re-established protocols such as DNS, UDP-based discovery, and CDN edges. It also spreads an attack surface across sites, although capacity and filtering must be coordinated. A route that remains advertised while the process is dead is worse than no route: traffic is sent to a black hole. Tie route announcements to a real readiness signal and monitor both control-plane reachability and application responses.

Stateful sessions mind the move

An established TCP connection can reach a different site after a route change. Without the old socket state, that instance may reset or drop it. A shared backend preserves application state, not the TCP connection; clients still need reconnect-and-rebuild behavior. Idempotency keys can prevent repeated side effects after a reconnect. Do not infer stickiness from one successful trace: topology changes can select another replica even during a flow.

Observability must include the serving site, route age, and withdrawal reason. A single global success rate can hide one region black-holing traffic while other regions look healthy. Sample from several networks, record the anycast address separately from the site identity, and alert when a site is still advertised but its service checks fail.

When it breaks

A route leak or an accidental high local preference can attract traffic far from the intended site. A partial withdrawal can leave some peers on a stale path, producing asymmetric loss rather than a clean failover. MTU, firewall, and ACL differences between replicas create failures that look like random routing. If the service has local state, users may see resets only after the control plane has already converged.

Avoid advertising a prefix from a site until the process, dependencies, and return path are ready. Test withdrawal from a staging address, measure convergence from more than one vantage point, and keep a non-anycast management path for recovery. Routing is the selection mechanism; it is not an application health check unless the announcement is explicitly coupled to one.

Check it quickly

This local Go model captures only local preference and AS-path length for one prefix. It tests path tie-breaking, input-order independence, policy override, withdrawal, an empty candidate set, and backup recovery. No packets or BGP session leave the process.

package main

import "fmt"

type route struct {
 name string
 localPref int
 asPath int
 advertised bool
}

// This fixture models only local preference and AS-path length for one prefix.
func best(routes []route) (route, bool) {
 var chosen route
 for _, candidate := range routes {
 if !candidate.advertised {
 continue
 }
 if chosen.name == "" || candidate.localPref > chosen.localPref ||
 (candidate.localPref == chosen.localPref && candidate.asPath < chosen.asPath) {
 chosen = candidate
 }
 }
 return chosen, chosen.name != ""
}

func want(ok bool, message string) {
 if !ok {
 panic(message)
 }
}

func main() {
 routes := []route{
 {name: "tokyo", localPref: 100, asPath: 2, advertised: true},
 {name: "singapore", localPref: 100, asPath: 3, advertised: true},
 }
 chosen, ok := best(routes)
 want(ok && chosen.name == "tokyo", "shorter AS path should win the tie")
 chosen, ok = best([]route{routes[1], routes[0]})
 want(ok && chosen.name == "tokyo", "input order must not beat path length")
 routes[1].localPref = 200
 chosen, ok = best(routes)
 want(ok && chosen.name == "singapore", "local policy should override path length")
 routes[1].advertised = false
 chosen, ok = best(routes)
 want(ok && chosen.name == "tokyo", "withdrawal should select the surviving site")
 routes[0].advertised = false
 _, ok = best(routes)
 want(!ok, "with every announcement withdrawn there is no route")
 _, ok = best(nil)
 want(!ok, "an empty candidate set must not invent a route")
 fallback := route{name: "sydney", localPref: 50, asPath: 4, advertised: true}
 chosen, ok = best(append(routes, fallback))
 want(ok && chosen.name == "sydney", "a backup announcement should recover reachability")
 fmt.Println("PASS: path length, policy priority, withdrawal, empty set, and backup")
}

Save this as anycast_model.go and run go run anycast_model.go. The expected output is PASS: path length, policy priority, withdrawal, empty set, and backup. Check formatting with gofmt -d, static diagnostics with go vet, and execution with go run -race. The model omits MED, eBGP/iBGP rules, communities, topology, and convergence timing. It proves selection among given candidates, not convergence on the Internet.

Sources

RFC 4271: A Border Gateway Protocol 4, route selection and path attributes, checked 2026-09-09.
RFC 4786: Operation of Anycast Services, deployment, routing, and failure considerations, checked 2026-09-09.

Anycast · XB Field Notes