Cover for alert routing, with observe, decide and act steps.

Alertmanager routes alerts through a label-matching tree

Alertmanager selects notification receivers by walking a routing tree against an alert’s labels. The tree expresses ownership and escalation policy, while receivers describe notification integrations. A configuration can parse correctly and still send the wrong class of alert to the wrong destination. Verify the label contract and tree behavior before treating a successful reload as evidence that routing works. ...

December 8, 2027 · 4 min · XB Field Notes
A controller that resizes a workload's replica count from a target metric.

The horizontal autoscaler scales on demand

The autoscaler derives desired replicas from current usage and a target, so the metric you choose matters more than the replica count you set. Metrics drive the count The HPA does not watch load directly; it polls a configured metric on a schedule, reads a current value against the target you set, and derives a desired replica count from the ratio. If the observed value runs twice the target it scales up, if half it scales down, and the result is recomputed on each reconciliation. That is why the metric choice, its aggregation (per pod or cluster-level), and the window matter, they define what the load even is. The desired count is then clamped by maxReplicas and minReplicas, and reconciled on the default --horizontal-pod-autoscaler-sync-period. To make scaling predictable you reason about the input metric, not about a single constant. ...

November 4, 2027 · 5 min · XB Field Notes
A rolling request budget alongside an explicit release decision.

Error budgets are a real budget

An error budget makes a reliability agreement usable during planning. Product and operations choose an objective, measure failures against it, and agree what changes when the allowance is exhausted. The number supports that agreement; it does not predict whether tomorrow’s deployment will cause an incident. Shipping a release creates risk, while observed bad events consume the measured budget. ...

March 25, 2027 · 4 min · XB Field Notes
Eligible requests split into good events and a union of failure conditions.

An error budget names the failures you allow

An objective such as “99.9% reliable” is incomplete until someone can classify an observation. Reliable for which users, at which boundary, over what window, and according to which outcome? An error budget gives the permitted failures a quantity only after those questions have answers. Its first engineering task is defining the population and the good-event rule, not choosing a dashboard color. ...

September 17, 2026 · 4 min · XB Field Notes