Logstash with persistent queue
Nowadays Logstash Persistent Queues behave like this:
"When the persistent queue feature is enabled, Logstash will store events on disk. Logstash commits to disk in a mechanism called checkpointing."
https://www.elastic.co/guide/en/logstash/current/persistent-queues.htmlhttps://www.elastic.co/guide/en/logstash/current/persistent-queues.html#durability-persistent-queues
Is that all you have in logstash.yml for your pipeline? You should be defining your pipeline settings in either logstash.yml or pipelines.yml. For example, it should look like:
- pipeline.id: Beats path.config: "/LogStash/pipelines/beatspipeline.yml" queue.type: persisted path.queue: /Logstash/data/queue queue.max_bytes: 10gbThe documentation doesn't explicitly state you must configure per pipeline settings, but I know this method has always worked.