IOException: Too many open files

The problem comes from your Java application (or a library you are using).

First, you should read the entire outputs (Google for StreamGobbler), and pronto!

Javadoc says:

The parent process uses these streams to feed input to and get output from the subprocess. Because some native platforms only provide limited buffer size for standard input and output streams, failure to promptly write the input stream or read the output stream of the subprocess may cause the subprocess to block, and even deadlock.

Secondly, waitFor() your process to terminate.You then should close the input, output and error streams.

Finally destroy() your Process.

My sources:

• http://stuffthathappens.com/blog/2007/11/28/crash-boom-too-many-open-files/
• http://www.javaworld.com/javaworld/jw-12-2000/jw-1229-traps.html?page=4
• http://kylecartmell.com/?p=9

As you are running on Linux I suspect you are running out of file descriptors. Check out ulimit. Here is an article that describes the problem: http://www.cyberciti.biz/faq/linux-increase-the-maximum-number-of-open-files/

Aside from looking into root cause issues like file leaks, etc. in order to do a legitimate increase the "open files" limit and have that persist across reboots, consider editing

/etc/security/limits.conf

by adding something like this

jetty soft nofile 2048jetty hard nofile 4096

where "jetty" is the username in this case. For more details on limits.conf, see http://linux.die.net/man/5/limits.conf

log off and then log in again and run

ulimit -n

to verify that the change has taken place. New processes by this user should now comply with this change. This link seems to describe how to apply the limit on already running processes but I have not tried it.

The default limit 1024 can be too low for large Java applications.