Dotnet Core Docker Container Leaks RAM on Linux and causes OOM

This could be because garbage collection (GC) is not executed.

Looking at this open issue it looks very similar:

https://github.com/dotnet/runtime/issues/851

One solution that made Ubuntu 18.04.4 work on a virtualized machine was using Workstation garbage collection (GC):

<PropertyGroup>    <ServerGarbageCollection>false</ServerGarbageCollection></PropertyGroup>

https://github.com/dotnet/runtime/issues/851#issuecomment-644648315

https://github.com/dotnet/runtime/issues/851#issuecomment-438474207

https://docs.microsoft.com/en-us/dotnet/standard/garbage-collection/workstation-server-gc

This is another finding:

After further investigations I've noticed that there is big differencebetween my servers in amount of available logical CPUs count (80 vs16). After some googling I came across this topic dotnet/runtime#622that leads me to an experiments with CPU/GC/Threads settings.

I was using --cpus constraint in stack file; explicitly setSystem.GC.Concurrent=true, System.GC.HeapCount=8,System.GC.NoAffinitize=true, System.Threading.ThreadPool.MaxThreads=16in runtimeconfig.template.json file; update image to a 3.1.301-bionicsdk and 3.1.5-bionic asp.net runtime — I made all this things in avarious combinations and all of this had no effect. Application justhangs until gets OOMKilled.

The only thing that make it work with Server GC is --cpuset-cpusconstraint. Of course, explicit setting of available processors is notan option for a docker swarm mode. But I was experimenting withavailable cpus to find any regularity. And here I got a fewinteresting facts.

What is interesting, previously I have mirgated 3 other backendservices to a new servers cluster and they all go well with a defaultsettings. Their memory limit is set to 600 Mb but in fact they needabout 400 Mb to run. Things go wrong only with memory-consumingapplications (I have two of those), it requires 3 Gb to buildin-memory structures and runs with a 6 Gb constraint.

It keeps working in any range between [1, 35] available cpus and getshanging when cpus count is 36.

https://github.com/dotnet/runtime/issues/851#issuecomment-645237830