Controlling logging functionality in hadoop Controlling logging functionality in hadoop hadoop hadoop

Controlling logging functionality in hadoop


What specifically are you trying to achieve with your own Log4J file? I ask because the logs are distributed across your cluster, but by logging them to the rootLogger, you should be able to see them via the job tracker (by drilling down on the Job task attempts).

If you want to utilize rolling files then you have a difficult time retrieving those files later (again because they are distributed across your task nodes).

If you want to dynamically set log levels, this should be simple enough:

public static Logger log = Logger.getLogger(MyMapper.class);@Overrideprotected void setup(Context context) throws IOException,        InterruptedException {    log.setLevel(Level.WARN);}

If you want to add you own appenders, then you should be able to do this programmatically (see this SO Question), in the setup method as above.