Hadoop Job Scheduling query Hadoop Job Scheduling query hadoop hadoop

Hadoop Job Scheduling query


What about calling the job from external java schedule framework, like Quartz? Then you can run the job as you want.


you might consider using Oozie (http://yahoo.github.com/oozie/). It allows (beside other things):

Frequency execution: Oozie workflow specification supports both data and time triggers. Users can specify execution frequency and can wait for data arrival to trigger an action in the workflow.

It is independent of any other Hadoop schedulers and should work with any of them, so probably nothing in you Hadoop configuration will change.


How about having a script to execute your Hadoop job and then using at command to execute at some specified time.if you want the job to run regularly, you could setup a cron job to execute your script.