Aggregate 2 PromQL gauge metrics without including extra labels from right operand Aggregate 2 PromQL gauge metrics without including extra labels from right operand kubernetes kubernetes

Aggregate 2 PromQL gauge metrics without including extra labels from right operand


This can be achieved with a combination of the following:

  • label_replace query function: For each timeseries in v, label_replace(v instant-vector, dst_label string, replacement string, src_label string, regex string) matches the regular expression regex against the value of the label src_label. If it matches, the value of the label dst_label in the returned timeseries will be the expansion of replacement, together with the original labels in the input. Capturing groups in the regular expression can be referenced with $1, $2, etc. If the regular expression doesn't match then the timeseries is returned unchanged. https://prometheus.io/docs/prometheus/latest/querying/functions/#label_replace

  • multiplication * operator and group_left() modifier: Many-to-one and one-to-many matchings refer to the case where each vector element on the "one"-side can match with multiple elements on the "many"-side. This has to be explicitly requested using the group_left or group_right modifier, where left/right determines which vector has the higher cardinality. https://prometheus.io/docs/prometheus/latest/querying/operators/

Example query:

label_replace(kube_pod_labels{},"label","$1","label_", "(.+)") * on (cluster,namespace, pod) group_left() (sum by (cluster,namespace, pod) (kube_pod_container_resource_requests_cpu_cores{}))

Note that: If the regular expression doesn't match then the timeseries is returned unchanged. In this case the regular expression does not match - hence the full set of labels is return unchanged.

Example result:

{cluster="my-cluster",label_foo="bar", label_app="my-app-label",namespace="my-ns",pod="my-pod",service="my-svc"} 0.05

Felipe provided a valuable hint on how to achieve this result in a comment for the original question.