The HPA grows and shrinks a Deployment by load

Wed, 11 Aug 2027 00:00:00 +0000

A HorizontalPodAutoscaler is a feedback controller, not a capacity planner. Once every sync period — 15 seconds by default — it asks the resource metrics API for per-pod CPU, divides the observed value by the target to get a ratio, and writes the result into the target workload’s scale subresource. The formula is one line: desiredReplicas = ceil(currentReplicas * currentMetricValue / desiredMetricValue).

Everything interesting is in the qualifiers. The controller deliberately ignores small ratios, deliberately delays shrinking, and refuses to act at all when it cannot read a metric or cannot compute a denominator. Those behaviours make it stable, and they are also why a correctly configured-looking HPA can sit at a fixed replica count while the service burns.

The ratio and the dead band

For a utilization target the denominator is not a percentage but a number of millicores: a container with a 200m CPU request and a 60% target produces a desired metric of 120m. The ratio is then compared against 1.0 with a configurable tolerance, 0.1 by default, and the controller skips scaling entirely when the ratio is inside that band. The band is not a rounding detail; it is the anti-thrash mechanism.

The band means the observable behaviour is coarse. With a 120m desired metric, average usage of 110m, 120m and 132m all leave the replica count exactly where it was; 133m is the first value that moves it. The step itself is a ceiling, so 160m against 120m rounds three replicas up to four rather than to 3.99 — a fractional pod cannot exist, so the controller always errs on the side of capacity.

Missing metrics and the veto

Pods whose metrics have not arrived are set aside rather than counted as zero, and the controller then re-adds them conservatively: a missing pod is assumed to consume the full target when the decision is to scale down, and nothing when the decision is to scale up. That asymmetry shrinks the magnitude of any move in either direction. A pod with no CPU request has no denominator, so its utilization is undefined and the HPA takes no action for that metric — the commonest cause of an HPA that appears to do nothing.

Multiple metrics make this stricter rather than looser. The controller computes a count per metric and takes the largest, so a scale-up on any one metric still goes through. But if a metric cannot be converted into a count at all while the metrics that do resolve suggest scaling down, the whole scale-down is skipped. A broken metrics source therefore freezes shrinkage while leaving growth possible.

Stabilisation and rate limits

Scale-down is filtered through a 300-second stabilisation window. Every recommendation is recorded and the controller applies the highest one inside the window, approximating a rolling maximum. A brief spike that produced a recommendation of ten replicas keeps the applied count at ten until that recommendation ages out, so the first real shrink happens a full five minutes after load dropped. Scale-up has no stabilisation window at all.

Rate limiting is asymmetric too. The default scale-up policy allows the larger of 100% of current replicas or four pods to be added every 15 seconds, and scale-down allows 100% of replicas to be removed. A cold HPA starting from one replica can therefore go to five, then ten, then twenty, doubling before the metrics have had time to react — a feature during a spike and a hazard when the metric is wrong.

When it breaks

The classic failure is a target with no denominator. The HPA reports unknown utilization, logs a condition such as FailedGetResourceMetric or ScalingActive=False, and never changes the replica count — so the symptom is not an error page, it is a service that keeps timing out at a fixed size. The cause is usually a container without a CPU request, or a missing metrics-server; kubectl create deployment emits resources: {} and has no flag to set one, so this is the default state of a hand-made manifest.

The second failure is a target that cannot be reached. Because maxReplicas is applied after the ratio, a workload needing more than the cap keeps computing a larger number every tick and keeps being truncated: utilization stays pegged, the status reports that it cannot scale further, and no event says the target itself is impossible. Set the cap from measured capacity, not from hope.

Check it quickly

Confirm that a metric exists, that a denominator exists, and what the controller decided:

kubectl get hpa web -o wide
kubectl top pods -l app=web
kubectl get deploy web -o jsonpath='{.spec.template.spec.containers[0].resources.requests.cpu}{"\n"}'
kubectl get hpa web -o jsonpath='{range .status.conditions[*]}{.type}={.status} {.reason}{"\n"}{end}'
kubectl describe hpa web | sed -n '/Events/,$p'

The first line shows current against target utilization; <unknown> means the metric is missing. The third prints nothing when no CPU request is set, which is the usual reason the second command works while the HPA still refuses to scale. The last two name the condition reason and any FailedGetResourceMetric event, which is where the real explanation lives.

Sources

Horizontal Pod Autoscaling, the algorithm, tolerance, missing-metric handling, stabilisation window and default policies, checked 2026-09-16.
HPA walkthrough, the metrics-server dependency and the requests prerequisite, checked 2026-09-16.
metrics-server, the resource metrics API implementation the HPA reads, checked 2026-09-16.