The autoscaler derives desired replicas from current usage and a target, so the metric you choose matters more than the replica count you set.

Metrics drive the count

The HPA does not watch load directly; it polls a configured metric on a schedule, reads a current value against the target you set, and derives a desired replica count from the ratio. If the observed value runs twice the target it scales up, if half it scales down, and the result is recomputed on each reconciliation. That is why the metric choice, its aggregation (per pod or cluster-level), and the window matter, they define what the load even is. The desired count is then clamped by maxReplicas and minReplicas, and reconciled on the default --horizontal-pod-autoscaler-sync-period. To make scaling predictable you reason about the input metric, not about a single constant.

Scale up eagerly, down gently

Scale-down is deliberately asymmetric. The controller does not remove a pod the instant utilization dips, because a transient spike would otherwise ping-pong the replica count and thrash the cluster. It waits out a stabilization window set by --horizontal-pod-autoscaler-downscale-stabilization, keeps a floor via minReplicas, and only then shrinks. Scaling up is fast, because queued latency hurts right now; scaling down is slow, because idle capacity is cheaper than a flapping fleet. A dependable setup states a floor generous enough for the spikes you actually see, a target it can digest, and treats the stabilization window as the amount of load noise you are willing to ride out, so the group does not bounce every time the graph blips.

The whole controller inherits the quality of your metric. If the metric lags real demand, the controller is always chasing a stale number; if it watches a proxy that correlates poorly with the actual bottleneck, the controller scales the wrong thing. So the hard part of an HPA is rarely the YAML, it is the metric and the target. Putting CPU on a latency-bound service, or memory on an I-O-bound worker, often points scaling in the wrong direction. Begin from what actually saturates, requests in flight, queue depth, or connections, choose an average over a window you can act on, and prove that a rise in that metric really predicts worse latency. Then the HPA becomes a tool that scales your workload, not a timer that wastes a guess.

When it breaks

Stale or missing metrics make the controller flap. When the metrics pipeline lags, HPA scales against last minute’s load: it adds replicas after the spike already passed, then holds them through the downscale stabilization window, so dashboards show oscillation rather than tracking. If metrics go missing entirely, the controller refuses to scale and surfaces ScalingActive=False; treating “no data” as “no load” and deleting capacity is the classic outage version of this bug.

Bounds freeze the loop in the other direction. A maxReplicas set from cost anxiety caps the fleet during a real surge and reports ScalingLimited=True, while a generous minReplicas hides an overestimated target because the floor never lets the controller prove demand fell. Finally, do not let two controllers steer the same knob: HPA fighting VPA (one changes count, the other changes requests, which moves utilization) or an aggressive rolling update (new pods report zero usage while old pods still serve) produces sawtooth replicas that no target tuning will fix.

Check it quickly

Save the following as hpa_loop.py in a scratch directory and run python3 hpa_loop.py. It models one HPA control loop with pure standard library: desired replicas are the ceiling of current average utilization over target, clamped to min and max, with missing metrics holding the last decision and a downscale stabilization window blocking fast shrink. The script asserts scale-up on a hot sample, cooldown-blocked scale-down on the next cool sample, eventual shrink after the window, and clamping at both bounds.

import math

TARGET = 50.0
MIN_REPLICAS = 2
MAX_REPLICAS = 10
STABILIZATION_S = 300


def desired(current_replicas, current_util):
    raw = math.ceil(current_replicas * current_util / TARGET)
    return max(MIN_REPLICAS, min(MAX_REPLICAS, raw))


def step(state, now, current_util):
    replicas, last_scale = state
    if current_util is None:
        print(f"t={now}s metric=missing hold={replicas}")
        return state
    want = desired(replicas, current_util)
    if want < replicas and now - last_scale < STABILIZATION_S:
        print(f"t={now}s metric={current_util} want={want} hold={replicas} (cooldown)")
        return state
    print(f"t={now}s metric={current_util} {replicas}->{want}")
    if want != replicas:
        return (want, now)
    return state


def main():
    state = (3, -10**9)
    state = step(state, 0, 100)
    assert state[0] == 6, state
    state = step(state, 60, 15)
    assert state[0] == 6, state
    state = step(state, 400, 15)
    assert state[0] == 2, state
    state = step(state, 460, None)
    assert state[0] == 2, state
    assert desired(2, 500) == 10
    assert desired(2, 5) == 2
    print("PASS: scale-up 3->6, cooldown holds 6, then 6->2; clamps at 10 and 2")


if __name__ == "__main__":
    main()

The run prints each decision and ends with PASS. The hot sample grows 3 replicas to 6, the immediate cool sample stays at 6 while the stabilization window is fresh, the same cool value shrinks to 2 after the window expires, and out-of-range calculations stick at max 10 and min 2. The fixture checks control-loop math, not metrics-server latency or startup time. Validate cluster tuning against kubectl describe hpa conditions.

Further reading

Sources