java.net.SocketException: Too many open files Spring Hibernate Tomcat java.net.SocketException: Too many open files Spring Hibernate Tomcat nginx nginx

java.net.SocketException: Too many open files Spring Hibernate Tomcat


Ok, it turns out that the problem was the jdbc connection settings, has maxActive set to 20 connections, I changed the limit to 200 and the problem stopped.

The way I figured this was the problem was thanks to appdynamics.comĀ“s wonderful tool, which lets you check a great deal of metrics in the ApplicationInfraestructurePerformance metrics.

Also, found this wonderful article about the subject which helped me tune my app:

http://www.tomcatexpert.com/blog/2010/04/01/configuring-jdbc-pool-high-concurrency

the oficial documentation also helped:

https://tomcat.apache.org/tomcat-7.0-doc/jdbc-pool.html.

I guess that the arriving connections started a query which collapsed the server response ability first, and afterwards filled the OS socket limits, in linux, open socket are open files. I hope this helps someone !

EDIT

Hello! This solution solved the issue in the short term, but another error regarding the JDBC connection appeared, the application was not closing the connections, I opened and solved a ticket regarding that issue here


Have you checked your ulimit for the user that is running tomcat?Linux has a limit of 1024 open files by default.More onHow do I change the number of open files limit in Linux?

There is a possibility you have too many connections in the configs or for some reason you are not properly closing some IO streams(higly unlikely).

I would approach it by increasing the ulimit and then run some load test to see what is spiking the file usage.


a bit late, but maybe a help/hint for anyone struggling with this issue. we had the same strange problem every now and then (our tomcat service is restarted every day at night (which cleans the open handles), so the error was not happening that often).
We use a apache proxy with ajp protocol. The problem was a wrong protocol implementation.
Our connector config is now the following:

<Connector        port="8009"        protocol="org.apache.coyote.ajp.AjpNioProtocol"        connectionTimeout="1800000"        acceptorThreadCount="2"        maxThreads="400"        maxConnections="8192"        asyncTimeout="20000"        acceptCount="200"        minSpareThreads="40"        compression="on"        compressableMimeType="text/html,text/xml,text/plain"        enableLookups="false"        URIEncoding="UTF-8"        redirectPort="8443" />

Please mind this: protocol="org.apache.coyote.ajp.AjpNioProtocol"
This implementation did the trick for us - no more open file handles.Further information can be found here: https://itellity.wordpress.com/2013/07/12/getting-rid-of-close_waits-in-tomcat-67-with-mod_proxy/
I hope this helps someone.Have a nice day!