Dynamic tagging for Fluentd td-agent source plugin

You can use the combo fluent-plugin-kubernetes_metadata_filter and fluent-plugin-rewrite-tag-filter to set container name or something to the tag.

As an answer to your first question (How can I dynamically set the td-agent's tag value?), this seems the best way that you are doing which is defining tag "#{ENV['TAG_VALUE']}" inside fluentd config file.

For your second question, environment variable is assigned before a container's initialization.

So it means it should work and I tested with below sample yaml, and it just worked fine.

apiVersion: v1kind: ConfigMapmetadata:  name: fluentd-confdata:  fluentd.conf.template: |    <source>      @type tail      tag "#{ENV['TAG_VALUE']}"      path /var/log/nginx/access.log      format nginx    </source>    <match *.*>      @type stdout    </match>---apiVersion: v1kind: Podmetadata:  name: log-forwarder  labels:    purpose: test-fluentdspec:  containers:    - name: nginx      image: nginx:latest      volumeMounts:        - name: logvolume          mountPath: /var/log/nginx    - name: fluentd      image: fluent/fluentd      env:        - name: "TAG_VALUE"          value: "test.nginx"        - name: "FLUENTD_CONF"          value: "fluentd.conf"      volumeMounts:        - name: fluentd-conf          mountPath: /fluentd/etc        - name: logvolume          mountPath: /var/log/nginx  volumes:    - name: fluentd-conf      configMap:        name: fluentd-conf        items:          - key: fluentd.conf.template            path: fluentd.conf    - name: logvolume      emptyDir: {}  restartPolicy: Never

And when I curl nginx pod, I see this output on fluentd containers stdout.

kubectl logs -f log-forwarder fluentd2019-03-20 09:50:54.000000000 +0000 test.nginx: {"remote":"10.20.14.1","host":"-","user":"-","method":"GET","path":"/","code":"200","size":"612","referer":"-","agent":"curl/7.60.0","http_x_forwarded_for":"-"}2019-03-20 09:50:55.000000000 +0000 test.nginx: {"remote":"10.20.14.1","host":"-","user":"-","method":"GET","path":"/","code":"200","size":"612","referer":"-","agent":"curl/7.60.0","http_x_forwarded_for":"-"}2019-03-20 09:50:56.000000000 +0000 test.nginx: {"remote":"10.128.0.26","host":"-","user":"-","method":"GET","path":"/","code":"200","size":"612","referer":"-","agent":"curl/7.60.0","http_x_forwarded_for":"-"}

As you can see, my environment variable TAG_VALUE=test.nginx has applied to log entries.

I hope it will be useful.