High I/O Java process consistently gets signal 11 SIGSEGV in a JavaThread when run in a Docker container
From the full log:
siginfo: si_signo: 11 (SIGSEGV), si_code: 0 (SI_USER)
This means a kill()
was issued. This is not a JVM issue. Something is killing the process deliberately. Probably due to running out of memory.
Based on your comment, this is likely a case where your container limits are lower than your heap space + space needed for GC.
Some insights on how to run the JVM in a container here.
You didn't post any pod specs but you can also take a look a setting limits on your Kubernetes pods.
A big hint is here
Memory: 4k page, physical 33554432k(1020k free), swap 0k(0k free)
Out of 32 GB, only 1 MB is free at the time of the crash. Most likely the process was killed as the system has run out of memory. I suggest:
- reducing the heap size significantly. e.g. 2 - 8 GB
- increasing the available memory. e.g. 4 - 16 GB
- adding some swap space. e.g. 8 - 32 GB, this doesn't fix the problem but will handle full memory a little more gracefully.