how to do bulk indexing to elasticsearch from python
I prefer using the bulk method present in helpers module for bulk indexing. Try the following:
from elasticsearch import helpersres = helpers.bulk(es, jsonvalue, chunk_size=1000, request_timeout=200)Your jsonvalue needs to follow a particular format. It needs to be a list of the 10K json documents with each document having the following fields:
doc = { '_index': 'your-index', '_type': 'your-type', '_id': 'your-id', 'field_1': 'value_1', ...}So your final jsonvalue would look something like this:
jsonvalue = [ { '_index': 'your-index', '_type': 'your-type', '_id': 'your-id', 'field_1': 'value_1', ...}, { '_index': 'your-index', '_type': 'your-type', '_id': 'your-id', 'field_1': 'value_2', ...}, { '_index': 'your-index', '_type': 'your-type', '_id': 'your-id', 'field_1': 'value_3', ...}]