Elasticsearch : how to reduce disk usage
By default, elasticsearch only merges away a segment if its delete percentage is over 10 %. If you want to delete all documents marked as deleted in the index, you should change index.merge.policy.expunge_deletes_allowed in elasticsearch.yml and set it to 0, then run the optimize command:
curl -XPOST 'http://localhost:9200/myindex/_optimize?only_expunge_deletes=true'
You can have a look at this link for more details about merge policy.
I think the difference you see in size is related to indexing and document metadata which is normal for any database. Size of indexes depends on your mappings. So technically, your documents size will never be the same as size of elasticsearch data folder.
Following links might help explain this better:
You should run the following:
curl -XPOST 'http://localhost:9200/myindex/_optimize?max_num_segments=1Perhaps you should run it more than once. (Because if there is too much segments it will not join all of them in one step.)