Difference between job, application, task, task attempt logs in Hadoop, Oozie Difference between job, application, task, task attempt logs in Hadoop, Oozie hadoop hadoop

Difference between job, application, task, task attempt logs in Hadoop, Oozie


In terms of YARN, the programs that are being run on a cluster are called applications. In terms of MapReduce they are called jobs. So, if you are running MapReduce on YARN, job and application are the same thing (if you take a close look, job ids and application ids are the same).

MapReduce job consists of several tasks (they could be either map or reduce tasks). If a task fails, it is launched again on another node. Those are task attempts.

Container is a YARN term. This is a unit of resource allocation. For example, MapReduce task would be run in a single container.