Because it can act on the mean of a series instead of a single spike, you need to pair it with a max-rc that keeps the maths from running forever.

A smoothed average decides

The HPA does not react to a single instantaneous read; it averages the observed value over a window and smooths it before comparing against the target. That is deliberate: a one-second spike in a metric should not repaint the replica count, so the controller aggregates first, then only changes the count when the smoothed value moves past the target. To make that work you choose the metric source (CPU, custom metrics API, or external) and the target, because the smoothing window and the aggregation method, per-pod average versus cluster level, together define what ’the load’ actually is. A metric read over too short a window reacts to noise; too long and it lags demand. Reading the document’s sync-period and aggregation paragraph is the first step in tuning, not the last.

Pick the metric you can act on

The real design choice is the metric, because scaling is only as useful as the target you give it. CPU is easy on any cluster and works well for CPU-bound workloads, but it is a proxy: a service waiting on queues or a database is often standing idle on CPU while its latency explodes. Demand-driven metrics such as request rate or a custom kafka_consumer_lag are harder to wire but report the thing users feel. Whatever you pick, choose one you can act on, a target that, when crossed, really changes the experience, and use the same definition the app uses, so the HPA is scaling the number that matters rather than a number that happens to be cheap to read.

Automation is orthogonal to defaults

Set the initial replicas in the Deployment because that is the starting point, not because the HPA reads it, once the HPA exists it owns the count and the initial value is only a floor or a first estimate. The two are orthogonal in the good way: you can disable the controller and still have a sane static stated count, and you can run a steady HPA without a fancy initial guess. Keep the minimum realistic (that is the real floor), pick a maximum that bounds spend, and remember that the initial value is what makes the very first seconds sane before the controller’s first reconcile lands.

When it breaks

A missing custom metric does not fall back to CPU. When the custom-metrics API returns an error or no series, the HPA marks ScalingActive False and holds replicas instead of guessing, so a workload with only a queue metric freezes exactly when it needed scaling. List CPU alongside the custom metric and let the maximum win, keeping a degraded but live input.

A Prometheus Adapter pipeline also adds delay. Scrape intervals plus adapter caching can serve queue depth from minutes ago, causing scale-up against drained work and overshoot. Check the series timestamp and adapter latency before retuning the target, and expose one pre-aggregated series per workload.

Cardinality breaks the same path differently: per-user or per-request labels multiply one signal into thousands of series, slow the API, and flip readings to Unknown. Aggregate in Prometheus first. A target-type mismatch does the same: custom metrics accept only Value or AverageValue, and CPU utilization without pod requests stays Unknown. Describe the HPA and align the type with what the pipeline serves.

Check it quickly

Save the following as hpa_metric_sim.py and run python3 hpa_metric_sim.py. It uses only the standard library, touches no cluster, and models the rules above: multiple metrics combine by maximum, missing metrics are skipped, and an all-missing poll holds replicas.

import math
import sys


def desired_for(current, value, target):
    """Replicas wanted by one metric; None means unavailable."""
    if value is None or target is None or target <= 0:
        return None
    return math.ceil(current * value / target)


def decide(current, metrics, min_replicas, max_replicas):
    """Combine metrics by maximum; hold when every metric is missing."""
    wants = []
    for _, value, target in metrics:
        want = desired_for(current, value, target)
        if want is not None:
            wants.append(want)
    if not wants:
        return current
    return max(min_replicas, min(max_replicas, max(wants)))


def check(actual, expected, message):
    if actual != expected:
        raise AssertionError(f"{message}: got {actual}, want {expected}")
    print(f"OK: {message} -> {actual}")


def main():
    current = 4
    cpu = ("cpu", 80, 50)  # 4 * 80/50 = 6.4 -> 7
    queue = ("queue", 900, 300)  # 4 * 900/300 = 12
    check(decide(current, [cpu, queue], 1, 20), 12, "max wins across metrics")
    check(decide(current, [cpu, ("queue", None, 300)], 1, 20), 7, "one missing metric still scales on CPU")
    check(decide(current, [("cpu", None, 50), ("queue", None, 300)], 1, 20), 4, "all missing holds replicas")
    check(decide(current, [cpu, queue], 1, 10), 10, "maxReplicas still clamps metric math")
    print("PASS: 4 scenarios; max-wins and hold-on-missing verified")
    print(sys.version.split()[0])


if __name__ == "__main__":
    main()

The four scenarios prove the maximum wins when CPU asks for seven and queue depth asks for twelve, a single missing series still leaves CPU in charge, all-missing holds instead of scaling to zero, and the maximum bound still clamps the result. Expected output ends with PASS: 4 scenarios; max-wins and hold-on-missing verified.

The assertions passed with Python 3.12.6 on macOS. The script checks replica math only, not scrape latency, adapter caching, or cardinality. Before changing production, verify timestamps and HPA events against the real custom-metrics API.

Further reading

Sources