How to suggest a more balanced allocation of containers in Hadoop cluster?
I was using the default scheduler (Capacity) and by default yarn.scheduler.capacity.per-node-heartbeat.maximum-container-assignments was set to -1 (infinity) which explained why every node that answer first to the RM (with Heartbeat) has been "packing" containers as much as it can.To conclude, inserting to hadoop/etc/hadoop/capacity-scheduler.xml the above parameter (using a third of the number of mappers would result in balanced scheduling of mappers) and following yarn rmadmin -refreshQueues after restarting the RM will grant you the option to balance the containers allocation in YARN.
For more details, please search my discussion here.