Indexing tuples from storm to elasticsearch with elasticsearch-hadoop library does not work Indexing tuples from storm to elasticsearch with elasticsearch-hadoop library does not work elasticsearch elasticsearch

Indexing tuples from storm to elasticsearch with elasticsearch-hadoop library does not work


Documents will only be indexed once you reach the flush size, specified by es.storm.bolt.flush.entries.size

Alternately, you may set a TICK frequency that triggers a queue flush.

config.put(Config.TOPOLOGY_TICK_TUPLE_FREQ_SECS, 5);

By default, es-hadoop flushes on tick, as per the es.storm.bolt.tick.tuple.flush parameter.


I have also got the same issue, but when I looking for the es-Hadoop documents, I find because I was miss set the frequency that triggers a queue flush.Then I add a configurations to my store topology (es.storm.bolt.flush.entries.size ), it's fine.but when we setting the value for Config.TOPOLOGY_TICK_TUPLE_FREQ_SECS .it's throw an exception :java.lang.RuntimeException:java.lang.NullPointerException in bolt execute function. then we use debug mode to test my topology, I find the input tuple in bolt execute don't contain any entries, but this empty tuple is been triggered.That's what I feel confusion. Don't the tuple will be emitted according to the setting time, Even though this tuple is empty after we set Config.TOPOLOGY_TICK_TUPLE_FREQ_SECS.i think which is a bug.enter image description hereenter image description here

more information you can see:https://www.elastic.co/guide/en/elasticsearch/hadoop/current/storm.html