How to aggregate logs for all docker containers in mesos

We're using Docker+Mesos as well and are shipping all logs to a log analytics service (it's the service the company I work for offers, http://logz.io). There a couple of ways to achieve that:

• Have a log shipper agent within each docker - an agent like rsyslog, nxlog, logstash, logstash-forwarder - that agent would ship data to a central logging solution
• Create a Docker that is running the shipper agent (like rsyslog, nxlog, logstash, logstash-forwarder) and that agent reads logs from all Dockers on each machine and ships them to a central location - this is the path we're taking

This is a broad question but I suggest you setup an Elastic Search, Logstash, Kibana stack (ELK)

https://www.elastic.co/products/elasticsearch

https://www.elastic.co/products/logstash

https://www.elastic.co/products/kibana

Then on each one of your containers you can run the logstash forwarder/shipper to send logs to your logstash frontend.

Logs get stored in Elastic Search and then you search for them using Kibana or the Elastic Search API

Hope it helps.

I am also doing some Docker + Mesos + Marathon work so, I guess, I am going to face same doubt that you have.

I don't know if there's any native solution yet. But there's a blog by the folks at elastic.io on how they went about solving this issue.

Here's the link - Log aggregation for Docker containers in Mesos / Marathon cluster