Is there a way to add nodes to a running Hadoop cluster?
Following steps should help you launch the new node into the running cluster.
1> Update the /etc/hadoop/conf/slaves list with the new node-name2> Sync the full configuration /etc/hadoop/conf to the new datanode from the Namenode. If the file system isn't shared. 2> Restart all the hadoop services on Namenode/Tasktracker and all the services on the new Datanode. 3> Verify the new datanode from the browser http://namenode:500704> Run the balancer script to readjust the data between the nodes.
If you don't want to restart the services on the NN, when you add a new node. I would say add the names ahead to the slaves configuration file. So they report as decommission/dead nodes until they are available. Following the above DataNode only steps. Again this not the best practice.
Updated Answer for Cloudera using CDH 5.8.5 (Hadoop 2.6)-
To add a new node to your cluster, follow these steps on ClouderaManager UI,
- Click on your cluster name.
- Go to Hosts List.
- Once on the hosts page, click 'Add New Hosts to Cluster'.
- Enter the IP of your host and Search.
- Keep following the instructions and continue to next steps.
- Finally assign roles to your new node, for example if it's a data-node,assign only datanode related roles and continue.
- Finally your new node is added to your cluster. click Finish.