How to create a ServiceMonitor for prometheus-operator? How to create a ServiceMonitor for prometheus-operator? kubernetes kubernetes

How to create a ServiceMonitor for prometheus-operator?


Thanks to Peter who showed me that it idea in principle wasn't entirely incorrect I've found the missing link. As a servicemonitor does monitor services (haha), I missed the part of creating a service which isn't part of the gitlab helm chart. Finally this yaml did the trick for me and the metrics appear in Prometheus:

# Service targeting gitlab instancesapiVersion: v1kind: Servicemetadata:  name: gitlab-metrics  labels:    app: gitlab-runner-gitlab-runnerspec:  ports:  - name: metrics # expose metrics port    port: 9252 # defined in gitlab chart    targetPort: metrics    protocol: TCP  selector:    app: gitlab-runner-gitlab-runner # target gitlab pods---apiVersion: monitoring.coreos.com/v1kind: ServiceMonitormetadata:  name: gitlab-metrics-servicemonitor  # Change this to the namespace the Prometheus instance is running in  # namespace: default  labels:    app: gitlab-runner-gitlab-runner    release: prometheusspec:  selector:    matchLabels:      app: gitlab-runner-gitlab-runner # target gitlab service  endpoints:  - port: metrics    interval: 15s

Nice to know: the metrics targetPort is defined in the gitlab runner chart.


I know this question is already answered. But I had a similar problem when Prometheus deployed in Kubernetes with Helm's stable/prometheus-operator chart couldn't find any active targets for my ServiceMonitor. It turned out that my Service exposed a port that I didn't explicitly named:

  - protocol: TCP    port: 8080    targetPort: uwsgi

I could use it in Ingress by targeting uwsgi port. But it seems that ServiceMonitor needs an explicitly named port in Service even if it has the same name as its own tagetPort:

  - name: uwsgi    protocol: TCP    port: 8080    targetPort: uwsgi

I have written a blog post about this problem here