Prometheus only scrapes one pod

Instead of using a static_config that scrapes just one host, try using kubernetes_sd_configs Kubernetes Service Discovery as provided by Prometheus.Your config file would look something like this:

- job_name: 'kubernetes-pods'  kubernetes_sd_configs:  - role: pod  relabel_configs:  # only scrape when annotation prometheus.io/scrape: 'true' is set  - source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_scrape]    action: keep    regex: true  - source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_path]    action: replace    target_label: __metrics_path__    regex: (.+)  - source_labels: [__address__, __meta_kubernetes_pod_annotation_prometheus_io_port]    action: replace    regex: (.+):(?:\d+);(\d+)    replacement: ${1}:${2}    target_label: __address__  - action: labelmap    regex: __meta_kubernetes_pod_label_(.+)  - source_labels: [__meta_kubernetes_namespace]    action: replace    target_label: kubernetes_namespace  - source_labels: [__meta_kubernetes_pod_name]    action: replace    target_label: kubernetes_pod_name

and then add the annotation to your Kubernetes Deployment yaml config like this:

kind: Deployment...spec:  template:    metadata:      annotations:        prometheus.io/scrape: "true"        prometheus.io/port: "<< PORT OF YOUR CONTAINER >>"

You can see a full working example here.

add prometheus annotations to your service, since prom will only scrape a service that:

• Exposes the exporter port
• Has a prometheus.io/scrape: "true" annotation
• Has a prometheus.io/port: "<exporter_port_here>" annotation

here is an official example

the scraped pod is probably prometheus itself