A DaemonSet puts one Pod on every eligible node

Mon, 29 Mar 2027 00:00:00 +0000

Where a Deployment cares about total count, a DaemonSet cares about coverage: a new node immediately gains the same service. Node-level concerns — shipping logs, exposing hardware metrics, running the network fabric, enforcing policy — must exist everywhere or they exist nowhere useful. A per-node script or a sidecar in every pod approximates this and rots immediately: new nodes miss it, version skew creeps in, and nobody can answer “which nodes run the agent” without an audit.

The DaemonSet controller closes that gap by making coverage declarative. You describe the daemon once, the controller reconciles reality toward one pod per eligible node forever, and scaling the cluster automatically scales the daemon fleet. Coverage stops being a runbook item and becomes an invariant the system maintains.

One per node, by design

You describe the daemon once and the controller attaches a copy to every node, which is the natural home for node-level logging, metrics agents, and kube-proxy-style traffic. There is no scaling knob to size by hand. That absence is a feature: replica counts drift from intent the moment someone edits them under pressure, while “every eligible node” cannot drift — it is recomputed continuously from the node list. Joining nodes gain the pod during registration; leaving nodes release it; cordoned nodes keep theirs unless drained.

Because placement is per-node rather than per-capacity, DaemonSet pods typically need priority and preemption headroom to survive on full nodes. Give them a high PriorityClass and modest, honest resource requests: they must schedule even when the node is otherwise saturated, because a monitoring agent missing from an overloaded node blinds you exactly when visibility matters most.

Coverage over count

Because the desired number of replicas is implicit, a rollout touches each member automatically: add a node to the pool and the daemon appears there without extra work. That is the guarantee no per-node script can match. Updates roll across the fleet with maxUnavailable bounding disruption — set it to 1 and the daemon fleet refreshes node by node, never leaving the fleet uncovered. Version skew, the chronic disease of node scripts, becomes structurally impossible: the controller converges every node toward the same template, and kubectl rollout status ds/agent tells you when convergence completes.

This guarantee extends to heterogeneous fleets through node selectors and affinity: one DaemonSet per node class, each converging its own subset. GPU nodes get the device plugin daemon, edge nodes get the lightweight forwarder, and each controller minds only its own coverage.

Rides the node lifecycle

Tolerations let a DaemonSet reach specialized nodes where normal taints keep generic workloads off, preserving fleet-wide consistency. The canonical examples are the masters’ control-plane taints and GPU or spot-instance taints: without tolerations, precisely the most interesting nodes lack the agent, and your fleet view has holes shaped exactly like your special hardware. Use nodeSelectors to pick which subsets get the daemon while keeping every eligible node in sync — selection by label keeps the subsets declarative, so relabeling a node moves it between daemon fleets without touching any controller.

Respect eviction and drain semantics in return. DaemonSet pods ignore ordinary scheduling pressure by design, but they must still terminate gracefully on drain: configure terminationGracePeriodSeconds honestly and handle SIGTERM, or node maintenance turns into log gaps and metric holes.

When it breaks

The classic failure is the silently uncovered node: a new node pool with a taint nobody told the DaemonSet about, so agents run everywhere except the forty new machines — which are, of course, the ones under investigation. Diagnose by comparing node count against daemon pod count per pool: desiredNumberScheduled versus numberReady per DaemonSet, grouped by node labels. Any pool where the numbers diverge names the missing toleration or selector immediately.

The second failure is the bad daemon update rolling fleet-wide: a broken agent image with maxUnavailable too high takes down logging on many nodes at once, blinding the very investigation the logs would serve. Keep maxUnavailable at 1 for observability daemons, pause and verify on a canary pool first (a nodeSelector-scoped DaemonSet makes a natural canary), and remember that rolling back a DaemonSet is one command — use it fast, because every minute without agents is a minute of missing evidence.

Check it quickly

Compare desired coverage against reality:

kubectl get ds -A -o custom-columns=NAME:.metadata.name,DESIRED:.status.desiredNumberScheduled,READY:.status.numberReady
kubectl get pods -o wide -l app=log-agent | awk '{print $7}' | sort | uniq -c | wc -l
kubectl get nodes --no-headers | wc -l

The first shows per-daemon coverage gaps, and the pod-node count versus node count proves every machine actually runs its copy.

Sources

Kubernetes DaemonSet, documents daemonset behavior, checked 2026-06-01.
Kubernetes taints and tolerations, documents reaching tainted nodes, checked 2026-09-05.