Pig JVM java heap space error
Probably a problem with the BZip codec - the API does note that it's rather memory hungry:
- http://hadoop.apache.org/common/docs/r0.20.0/api/org/apache/hadoop/io/compress/bzip2/CBZip2OutputStream.html
The compression requires large amounts of memory
-Xms2048m
did you set the options for the pig grunt shell, or for the map/reduce jobs?set mapred.child.java.opts=-Xmx2048m
You can check by looking in the JobTracker, find the job that failed, open the job.xml and locate the value of mapred.child.java.opts