Setting up MongoDB river for Elasticsearch

This is a rather old question, but I'll just post my answer in case others are wondering the same question, especially with new versions of ES coming out all the time. It took me a while to get my ES working with MongoDB too.

1. First, I assume you have ES and MongoDB installed. Make sure you have oplogs enabled if you're not using replica sets. Refer here on how to do it.

2. The river plugin has a dependency (elasticsearch-mapper-attachments), so MAKE SURE you install that first to prevent any problems later on. This wiki has the necessary commands you need to install the plugins. Take note of the alternate download link for the river plugin if you're using ES 0.20.2 and higher.

3. Restart ES.

4. Use the following command to enable the indexing:

   curl -XPUT 'http://localhost:9200/_river/mongodb/_meta' -d '{    "type": "mongodb",    "mongodb": {        "db": "your-database-name",        "collection": "your-collection-name"    },    "index": {        "name": "mongoindex",        "type": "your-type"    }}'

5. To do a search, use curl -XGET 'http://localhost:9200/mongoindex/_search?q=field:value'

I got most of my information from this website, but I felt it could be streamlined a lot more, hence my own approach.

It works in certain configurations. Versions of elasticsearch river plugin and mongodb could be essential for the whole system to work.

Here is steps to reproduce working environment

1) use elasticsearch version 1.2.4. Package for ubuntu it is located here: https://download.elasticsearch.org/elasticsearch/elasticsearch/elasticsearch-1.2.4.deb

install it with dpkg -i /path/to/elasticsearch-1.2.4.deb

2) install river plugin and its dependencies:

cd /usr/share/elasticsearch && bin/plugin --install elasticsearch/elasticsearch-mapper-attachments/1.9.0cd /usr/share/elasticsearch && bin/plugin --install com.github.richardwilly98.elasticsearch/elasticsearch-river-mongodb/2.0.1

Newer versions of mapper atachments plugin are available, but official wiki of mongo river plugin suggest to use 1.9.0.

3) MongoDB v2.4.9 surely works with elasticsearch river plugin.make sure mongod is launched with --replSet rs0 or you have replSet=rs0 in mongod.confNext, you must be sure that replica set is initiated in mongodb:login to mongo console:

mongo

Then type:

rs.initiate()rs.status()

4) Preparations have been done, now you need to init elasticsearch river plugin.Assuming that you have elasticsearch running on localhost:9200 and mongodb is on 10.0.2.15:27017

curl -XPUT "localhost:9200/_river/feed/_meta" -d"        {          \"type\": \"mongodb\",            \"mongodb\": {              \"servers\": [                {\"host\": \"10.0.2.15", \"port\": 27017}              ],              \"db\": \"YOUR_DB\",              \"collection\": \"YOUR_COLLECTION\"            },            \"index\": {              \"name\": \"YOUR_ELASTIC_INDEX\",              \"type\": \"item\"            }        }"

5) check rs.config() on mongo server. It should contain resolvable hostnames or ips for replica set members that could be contacted by mongodb river plugin. Elasticsearch river plugin will connect to the host specified in the mongodb.servers and fetch replica config. Then it will try to connect to the primary host of the replica set by the hostname specified in rs.config(). If you are about to use docker containers to interconnect monogdb and elasticsearch via river plugin than that's a problem.

You could update rs config by doing the following:

mongo

and then in mongo shell

cfg = rs.config()cfg.members[0].host = "12.34.56.78:27017"rs.reconfig(cfg)