h2o starting on YARN not working h2o starting on YARN not working hadoop hadoop

h2o starting on YARN not working


You should setup your default queue to have available resources to run 2nodes cluster.

See warnings:

  1. WARNING: Job memory request (2.2 GB) exceeds queue available memory capacity (0.0 GB)

    • you ask 1GB per node (+overhead) but there is no available resources in the YARN queue
  2. WARNING: Job virtual cores request (2) exceeds queue available virtual cores capacity (0)

    • you ask for 2 virtual cores but no cores are available in your default queue

Please check YARN documentation - for example setup of capacity scheduler and max available resources:https://hadoop.apache.org/docs/r2.4.1/hadoop-yarn/hadoop-yarn-site/CapacityScheduler.html


I made the following changes in Cloudera Manager yarn configuration

Setting                                     Valueyarn.scheduler.maximum-allocation-vcores    8 yarn.nodemanager.resource.cpu-vcores        4yarn.nodemanager.resource.cpu-vcores        4yarn.scheduler.maximum-allocation-mb        16 GB