Is the @timestamp field needed when using Logstash to store in Elasticsearch? Is the @timestamp field needed when using Logstash to store in Elasticsearch? json json

Is the @timestamp field needed when using Logstash to store in Elasticsearch?


When a message is read by Logstash, Logstash treat the message as a Event. An event will have a timestamp and message log. Thus, the @timestamp field is requisite.

Therefore, if you want to delete the @timestamp field, it will causes an error.Logstash can't output the event to the elasticsearch.

Exception in thread "LogStash::Runner" org.jruby.exceptions.RaiseException: (NoMethodError) undefined method `tv_sec' for nil:NilClass    at RUBY.sprintf(file:/tmp/logstash-1.2.1-flatjar.jar!/logstash/event.rb:239)    at org.jruby.RubyString.gsub(org/jruby/RubyString.java:3062)    at RUBY.sprintf(file:/tmp/logstash-1.2.1-flatjar.jar!/logstash/event.rb:225)    at RUBY.receive(file:/tmp/logstash-1.2.1-flatjar.jar!/logstash/outputs/elasticsearch.rb:153)

So far, not all @-prefix fields causes error, only remove @timestamp will cause this error.


Any @-prefixed field is used internally by Logstash. Removing them tends to cause errors.

For example, I tried the following config file with Logstash 1.3.3:

input {     generator {        type => "timestrip"        message => "This is a test message."        count => 1    }}filter {    mutate {        remove_field => ["@timestamp"]    }}output {    elasticsearch_http {        host => "127.0.0.1"        flush_size => 1    }}

The ES output reports "Failed to flush outgoing items" with the following exception:

NoMethodError: undefined method `tv_sec' for nil:NilClass