How to add a new node to my Elasticsearch cluster
TIPS TO ADD ANOTHER NODE:
1) VERSIONS:
It is a good advise to check all of your nodes for the status:http://elastic-node1:9200/
Keep in mind that in most cases: VERSION NEED TO BE THE SAME, EVEN MINOR
{"name" : "node2","cluster_name" : "xxxxxxxxxxx","cluster_uuid" : "n-xxxxxxxxxxxxxxx","version" : { "number" : "5.2.2", "build_hash" : "xxxx", "build_date" : "20-02-24T17:26:45.835Z", "build_snapshot" : false, "lucene_version" : "6.4.1"},"tagline" : "You Know, for Search"}
Keep in mind that if you see a different version number in node1, e.g.
"number" : "5.2.1"
you have to update your node in that case to version 5.2.2 (same as node1).
2) NODES AND REPLICA:
What is the usecase of the node? For 3 nodes I would do this:
curl -XPUT 'localhost:9200/_cluster/settings?pretty' -H 'Content-Type: application/json' -d'{ "transient": { "discovery.zen.minimum_master_nodes": 3 }}'
Even better is to change settings in Elasticsearch's configuration file:
/etc/elasticsearch/elasticsearch.yml # need to be changed on each node (has to be unique for each node):node.name: node1# need to be the same in all nodes:cluster.name: my_clusterdiscovery.zen.ping.unicast.hosts: ["IP_ADDRESS_OR_HOSTNAME1", "IP_ADDRESS_OR_HOSTNAME2", "IP_ADDRESS_OR_HOSTNAME3"]
And if you have 3 nodes, do you want two replicas and one primary?
curl -XPUT 'localhost:9200/_settings?pretty' -H 'Content-Type: application/json' -d'{ "index" : { "number_of_replicas" : 2 }}'
3) MAKE SURE THAT NODES ARE ENABLED
There is a way to kick a node:
curl -XPUT localhost:9200/_cluster/settings -d '{ "transient" :{ "cluster.routing.allocation.exclude._ip" : "NODE_TO_REMOVE_IP_ADDRESS_OR_HOSTNAME" }}';echo
So if you did that, and now you want to add the node back:https://www.elastic.co/guide/en/elasticsearch/guide/current/_rolling_restarts.html
you can do that with following request (please read carefully mentioned link above):
curl -XPUT localhost:9200/_cluster/settings -d '{ "transient" :{ "cluster.routing.allocation.enable" : "all" }}';echo
4) NEVER FORGET, NETWORKING:
Firewall, network... Can you reach the new node at port 9200?Can you see it on your web browser?
Can you
curl http://your-elasticsearch-hostname:9200/
?
TIPS TO REMOVE NODE FROM CLUSTER:
1) REMOVE WITH API
curl -XPUT 'http://localhost:9200/_cluster/settings?pretty' -d '{ "transient" : { "cluster.routing.allocation.exclude._name" : "node3" }}'
2) CHECK YOUR CONFIG FILE
Check config file under: /etc/elasticsearch/elasticsearch.yml
and leave only the nodes you want to keep:
discovery.zen.ping.unicast.hosts:["IP_ADDRESS_OR_HOSTNAME1", "IP_ADDRESS_OR_HOSTNAME2"]
* CHECK YOUR STATUS *
Check http://elk-pipeline:9200/_cat/shardsWhat is your status? You may see: INITIALIZINGThat probably means that data is transferred. So if your data is large, (and not on SSD), wait.
DON'T FORGET
You can see if your data is currently moving by typing:
[watch] du /var/lib/elasticsearch/
That is all for now. I will try to add more information here from time to time.
Please feel free to change/add.
Complete steps on Windows Box are:
- unzip elastic, say, to C:\ELK\elastic\ run command
bin\serviceinstall elastic-search-x64-node01
which will create service namedelastic-search-x64-node01
edit
elasticsearch.yml
config file:cluster.name: Animalsnode.name: Snakenode.master: truenode.data: truepath.data: C:\ELK\storage\datapath.logs: C:\ELK\storage\logshttp.port: 9200discovery.zen.ping.multicast.enabled: falsediscovery.zen.ping.unicast.hosts: ["127.0.0.1"]
run
service manager elastic-search-x64-node01
and set-up your services rules and start the serviceunzip elastic, say, to C:\ELK\elastic2\ run command
bin\serviceinstall elastic-search-x64-node02
which will create service namedelastic-search-x64-node02
edit
elasticsearch.yml
config file:cluster.name: Animalsnode.name: Baboonnode.master: falsenode.data: truepath.data: C:\ELK\storage\datapath.logs: C:\ELK\storage\logshttp.port: 9201discovery.zen.ping.multicast.enabled: falsediscovery.zen.ping.unicast.hosts: ["127.0.0.1"]
run
service manager elastic-search-x64-node02
and set-up your services rules and start the service
At this point you have 2 separate nodes (they store data in the same folder, but I'm too laze to edit path.data for second node) as 2 separate Windows Services and GET http://localhost:9200/_cluster/health
shows something like:
{ "cluster_name": "Animals", "status": "green", "timed_out": false, "number_of_nodes": 2, "number_of_data_nodes": 2, "active_primary_shards": 4, "active_shards": 8, "relocating_shards": 0, "initializing_shards": 0, "unassigned_shards": 0, "delayed_unassigned_shards": 0, "number_of_pending_tasks": 0, "number_of_in_flight_fetch": 0, "task_max_waiting_in_queue_millis": 0, "active_shards_percent_as_number": 100}
First, you can remove the replicas to go back to a green state, you can do this even after you have created the index and added documents to it.
Here's how you set the replica count to 0:
curl -XPUT 'localhost:9200/my_index/_settings' -d '{ "index" : { "number_of_replicas" : 0 }}'
If you would like to simply add another node to your cluster, you will need to edit the elasticsearch.yml
, preferably on both of your nodes with these changes:
cluster.name: my-clusternode.name: node01discovery.zen.ping.multicast.enabled: falsediscovery.zen.ping.unicast.hosts: ["192.168.0.5"]
Set the unicast hosts on each node to reference the other, set the cluster name to be the same on both nodes and give each node a unique name, then restart both ES instances and your cluster should be online.