Does an EMR master node know its cluster ID? Does an EMR master node know its cluster ID? hadoop hadoop

Does an EMR master node know its cluster ID?


You may look at /mnt/var/lib/info/ on Master node to find lot of info about your EMR cluster setup. More specifically /mnt/var/lib/info/job-flow.json contains the jobFlowId or ClusterID.

You can use the pre-installed json parser (jq) to get the jobflow id.

cat /mnt/var/lib/info/job-flow.json | jq -r ".jobFlowId"

(updated as per @Marboni)


You can use Amazon EC2 API to figure out. The example below uses shell commands for simplicity. In real life you should use appropriate API to do this steps.

First you should find out your instance ID:

 INSTANCE=`wget -q -O - http://169.254.169.254/latest/meta-data/instance-id`

Then you can use your instance ID to find out the cluster id :

ec2-describe-instances $INSTANCE | grep TAG | grep aws:elasticmapreduce:job-flow-id

Hope this helps.


As been specifed above, the information is in the job-flow.json file. This file has several other attributes. So, knowing where it's located, you can do it in a very easy way:

cat /mnt/var/lib/info/job-flow.json | grep jobFlowId | cut -f2 -d: | cut -f2 -d'"'

Edit: This command works in core nodes also.