Tracking Hadoop job status via web interface? (Exposing Hadoop to internal clients in the company) Tracking Hadoop job status via web interface? (Exposing Hadoop to internal clients in the company) hadoop hadoop

Tracking Hadoop job status via web interface? (Exposing Hadoop to internal clients in the company)


I have found one way to get information about jobs on JobTracker. This is the code:

    Configuration conf = new Configuration();    conf.set("mapred.job.tracker", "URL");    JobClient client = new JobClient(new JobConf(conf));    JobStatus[] jobStatuses = client.getAllJobs();    for (JobStatus jobStatus : jobStatuses) {        long lastTaskEndTime = 0L;        TaskReport[] mapReports = client.getMapTaskReports(jobStatus.getJobID());        for (TaskReport r : mapReports) {            if (lastTaskEndTime < r.getFinishTime()) {                lastTaskEndTime = r.getFinishTime();            }        }        TaskReport[] reduceReports = client.getReduceTaskReports(jobStatus.getJobID());        for (TaskReport r : reduceReports) {            if (lastTaskEndTime < r.getFinishTime()) {                lastTaskEndTime = r.getFinishTime();            }        }        client.getSetupTaskReports(jobStatus.getJobID());        client.getCleanupTaskReports(jobStatus.getJobID());        System.out.println("JobID: " + jobStatus.getJobID().toString() +                             ", username: " + jobStatus.getUsername() +                             ", startTime: " + jobStatus.getStartTime() +                             ", endTime: " + lastTaskEndTime +                             ", Durration: " + (lastTaskEndTime - jobStatus.getStartTime()));    }


Since version 'beta 2' of Cloudera's Hadoop Distribution you can almost with no effort use Hadoop User Experience (HUE), which was earlier called Cloudera Desktop.

But since this version it has grown enormously. It comes with job designer,hive interface and many more. You should definitely check this out before deciding to build your own application.


Maybe a good place to start would be to take a look at Cloudera Destktop. It provides a web interface to enable cluster administration and job development tasks. Its free to download.