Using logstash and elasticseach

Here is how we managed to fix this and other problems with Logstash not processing files correctly on Windows:

1. Install the ruby-filewatch patch as explained here:logstash + elasticsearch : reloads the same data

2. Properly configure the Logstash input plugin:

   input {  file {      path => ["C:/Path/To/Logs/Directory/*.log"]      codec => json { }      sincedb_path => ["C:/Path/To/Config/Dir/sincedb"]      start_position => "beginning"  }}...

"sincedb" keeps track of your log files length, so it should have one line per log file; if not, then there's something else wrong.

Hope this helps.

Your output scope looks not complete. Here's the list of the output parameters http://logstash.net/docs/1.4.2/outputs/elasticsearch

Please, try:

input {    file {        type => "nextgen-app"        path => [ "F:\NextGen-dev\RestApi\app\logs\*.log" ]        codec => "json"    }   }output {    elasticsearch {        host => "localhost"        port => 9200        protocol => "http"        index => "logstash-%{+YYYY.MM.dd}"    }}

Alternatively, you can try the transport protocol:

output {    elasticsearch {        host => "localhost"        port => 9300        protocol => "transport"        index => "logstash-%{+YYYY.MM.dd}"    }}

I also recommend using Kibana as a data viewer. You can download it at https://www.elastic.co/downloads/kibana