Leave out default Logstash fields in ElasticSearch
This removes fields from output:
filter { mutate { # remove duplicate fields # this leaves timestamp from message and source_path for source remove => ["@timestamp", "@source"] } }
Some of that will depend on what web interface you are using to view your logs. I'm using Kibana, and a customer logger (c#) that indexes the following:
{ "_index": "logstash-2013.03.13", "_type": "logs", "_id": "n3GzIC68R1mcdj6Wte6jWw", "_version": 1, "_score": 1, "_source": { "@source": "File", "@message": "Shalom", "@fields": { "tempor": "hit" }, "@tags": [ "tag1" ], "level": "Info" "@timestamp": "2013-03-13T21:47:51.9838974Z" }}This shows up in Kibana, and the source fields are not there.
To exclude certain fields you can use prune filter plugin.
filter { prune { blacklist_names => [ "@timestamp", "@source" ] }}Prune filter is not a logstash default plugin and must be installed first:
bin/logstash-plugin install logstash-filter-prune