How to buffer logstash logs to elastic search How to buffer logstash logs to elastic search elasticsearch elasticsearch

How to buffer logstash logs to elastic search


Place a Buffer (Redis, RabbitMQ ...) in front of your Logstash machine that will be the entry point for all log events that are shipped to your system. It will then buffer the data until the downstream components have enough resources to index.


You don't mention what your inputs actually are. filebeat will stop sending data to logstash/elasticsearch if it is unable to connect. Since it keeps track of where in the file it is, you get a distributed cache for free. Note that you may have problems if the log file is rotated when the server is still down.