How can I get Hadoop with Cascading to show me debug log output?

I got an answer from the mailing list on this.

Changing it to this works:

assembly = new Each( assembly, new Fields( "line" ), function );// simpler debug statementassembly = new Each( assembly, new Debug("hello", true) );assembly = new GroupBy( assembly, new Fields( "word" ) );

That outputs this in the jobdetails UI under stderr:

Task Logs: 'attempt_201203131143_0028_m_000000_0'stdout logsstderr logs2012-03-13 16:21:41.304 java[78617:1903] Unable to load realm info from SCDynamicStorehello: ['word']hello: ['CYMBELINE']<SNIP>

I had tried this directly from the docs, and that doesn't work for me (even though I've also set the FlowConnector debugLevel to VERBOSE):

assembly = new Each( assembly, DebugLevel.VERBOSE, new Debug() );

It seems that it's something related to the DebugLevel.VERBOSE from the documentation as when I try this, I still get no output:

assembly = new Each( assembly, DebugLevel.VERBOSE, new Debug("hello", true) );

Changing it to remove the DebugLevel also gives me output

assembly = new Each( assembly, new Debug() );

I can also get it to switch to stdout by doing this:

assembly = new Each( assembly, new Debug(Debug.Output.STDOUT) );

I'm betting there's still something I've got misconfigured with the VERBOSE log level stuff, or 1.2.6 doesn't match the documentation anymore, but at least now I can see the output in the logs.

Did you try setting

flow.setDebugLevel( DebugLevel.VERBOSE );