Filebeat with ELK stack running in Kubernetes does not capture pod name in logs

disclaimer: I'm a beats developer

What you want to do is not yet supported by filebeat, but definitely, it's something we want to put some effort on, so you can expect future releases supporting this kind of mapping.

In the meantime, I think your approach is correct. You can append the info you need to your logs so you have it in elasticsearch

I have achieved what you looking for, by assigning a group of specific pods to a namespace, and now can query the log I look for using a combination of namespace, pod name and container name which is also included in generated log which is piped by file beat without any extra effort as you can see here

For future people coming here, it is now already in place in a filebeat processor :

filebeat.prospectors:  - type: log    enabled: true    paths:      - /var/log/*.log      - /var/log/messages      - /var/log/syslog  - type: docker    containers.ids:    - "*"    processors:      - add_kubernetes_metadata:          in_cluster: true      - drop_event:          when:            equals:              kubernetes.container.name: "filebeat"

helm chart default values : https://github.com/helm/charts/blob/master/stable/filebeat/values.yaml

doc : https://www.elastic.co/guide/en/beats/filebeat/current/add-kubernetes-metadata.html