Labels are key-value metadata used to select Kubernetes objects. A Service selector, an operational query, and a workload controller can all rely on them, but each consumer has its own API and behavior. A label change can therefore alter group membership while leaving the underlying process healthy. Review labels as inputs to actual consumers rather than as purely descriptive stickers. ...
When several Prometheus servers send data to a common system, downstream readers need to know which collector or environment supplied it. Global external_labels can add that context during communication with external systems. They do not rewrite every locally stored series, and they do not override labels already present on the outgoing series. Understanding both boundaries prevents misleading configuration checks. ...
Service discovery produces candidate targets and metadata. Target relabeling transforms those candidates before Prometheus scrapes them. It can keep or drop targets, change how they are addressed, and create labels carried into collected series. This is a different stage from rewriting individual scraped samples. Understanding the stage prevents a configuration intended to remove one metric from accidentally removing an entire target. ...