Oozie shell action - running sqoop command and need logging details
You have a parameter hinting that Log4J should use a specific properties file, but Sqoop is not instructed to use it (no -Dlog4j.configuration=...file name without path...
on command line).
OK, let's assume that is is done on purpose (?); the problem is that
- Log4J searchs for a properties file w/ default name i.e.
log4j.properties
- search is done in the directories present in CLASSPATH, stopping on first match
- default CLASSPATH for an Oozie shell is Hadoop JARs, then Hadoop conf dir, then current working dir (the place where all
<file>
dependencies are dumped along with Oozie JARs) and all these app/Oozie JARs - the first match for
log4j.properties
happens to be a file that Log4J cannot open - thus Log4J does not log anything anywhere (????????)
A possible workaround would be
- create a custom
log4j.properties
-- cf. 1st example in that post to log anything tagged INFO and above (i.e. INFO, WARN, ERROR but not DEBUG) to StdOut - upload that file to HDFS somewhere, then tell the Oozie Action to download it to the container with a
<file>
element - tell the Oozie Action to request that its CLASSPATH starts with current working dir, by setting a property such as
oozie.launcher.mapreduce.task.classpath.first
totrue
(actual property may depend on your Hadoop version, see that post and that JIRA)
Note that step 3 is only necessary because of the default name being present elsewhere in the CLASSPATH; if Sqoop was instructed to use a different file with a different name then there would be no ambiguity.