How to configure Elasticsearch Index Lifecycle Management (ILM) durring installation in YAML file
I'll try to answer both of your questions.
index template
You can pass the index template with this configuration in your elasticsearch yaml. For instance:
setup.template: name: "<chosen template name>-%{[agent.version]}" pattern: "<chosen pattern name>-%{[agent.version]}-*"
Checkout the ES documentation to see where exactly this setup.template belongs and you're good to go.
ilm policy
The way to make this work is to get the ilm-policy.json file that has your ilm configuration to the pod's /usr/share/filebeat/ directory. in your YAML installation file, you can then use this line in your config to get it to work (I've added my whole ilm config):
setup.ilm: enabled: true policy_name: "<policy name>" rollover_alias: "<rollover alias name policy_file: "ilm-policy.json" pattern: "{now/d}-000001"
So, how to get the file there? The ingredients are 1 configmap containing your ilm-policy.json, and a volume and volumeMount in your daemonset configuration to mount the configmap's contents to the pod's directories.
Note: I used helm for deploying filebeat to an AKS cluster (v 1.15), which connects to Elastic cloud. In your case, the application folder to store your json will probably be /usr/share/elasticsearch/ilm-policy.json
.
Below, you'll see a line like {{ .Files.Get <...> }}
, which is a templating function for helm getting the contents of the files. Alternatively, you can copy the file contents directly into the configmap yaml, but to have the file separate makes it better managable in my opinion.
The configMap
Make sure your ilm-policy.json is somewhere reachable by your deployments. This is how the configmap can look:
apiVersion: v1kind: ConfigMapmetadata: name: ilmpolicy-config namespace: logging labels: k8s-app: filebeatdata: ilm-policy.json: |-{{ .Files.Get "ilm-policy.json" | indent 4 }}
The Daemonset
at the deamonSet's volumeMounts section, append this:
- name: ilm-configmap-volume mountPath: /usr/share/filebeat/ilm-policy.json subPath: ilm-policy.json readOnly: true
and at the volume section append this:
- name: ilm-configmap-volume configMap: name: ilmpolicy-config
I'm not exactly sure the spacing is correct in the browser, but this should give a pretty good idea.I hope this works for your setup! good luck.