ElasticSearch ILM not deleting indices

With these settings, I expected ES to delete indices older than 5-6 days, but there are still indices from 3 weeks ago in ES. Currently, it says there are 108 linked indices to this ILM policy.

With your settings, the delete phase starts at 4 day from rollover. If you want to start the delete phase at 4 day from "index creation" you need to remove the rollover action from the hot phase:

{  "policy": {    "phases": {      "hot": {        "min_age": "0ms",        "actions": {          "set_priority": {            "priority": 100          }        }      },      "delete": {        "min_age": "4d",        "actions": {          "delete": {}        }      }    }  }}

I tried first to use the "index template" but it's useless, it does not apply settings to each index (maybe yes but only on creation????).

Yes, it works on index creation.

Then I put the ILM on the index by hand (another bug: you can't select all index and hit "add ILM policy" - you need to add the policy one by one) which required me to click about 600 times.

Kibana does not allow you to apply ILM policy to all index, but the elasticsearch API allows it!Simply open a kibana dev tools and run the follow request:

PUT fluentd.*/_settings{  "index": {    "lifecycle": {      "name": "fluentd"    }  }}

Now the problem was, I had "hot" phase defined but it didn't trigger (it's buggy?) - because the hot phase didn't trigger (i set to to "rollover after 1 day after index creation") - the delete phase didn't either. When I removed the hot phase and applied the ILM to index again with only delete - it worked! but adding and removing all this is buggy, I get Ooops, something went wrong errors here and there.

If rollover phase was not triggered, the ILM could not progress.

I don't understand why I have to remove the ILM and reapply it to each index when I change something in the ILM policy. It's 1000% inconvenient.

Because the ILM definition are cached on each index.see the doc: https://www.elastic.co/guide/en/elasticsearch/reference/current/ilm-index-lifecycle.html#ilm-phase-execution

A little bit late, but maybe it will help somebody.

Another reason can be like that was mentioned here:

ILM is not really intended to be used on 1m lifecycle. I Do notbelieve You will achieve your desired behavior. My understanding isthat ILM is an opportunistic background task it is not preemptive soit is not going to execute on the exact time frame.

It's designed to work on the order of hours or days not minutes.

I have the same situation at my indices and I checked - indices are deleted, but later than I sat them up.