Postgres gets out of memory errors despite having plenty of free memory

I just ran into this same issue with a ~2.5 GB plain-text SQL file I was trying to restore. I scaled my Digital Ocean server up to 64 GB RAM, created a 10 GB swap file, and tried again. I got an out-of-memory error with 50 GB free, and no swap in use.

I scaled back my server to the small 1 GB instance I was using (requiring a reboot) and figured I'd give it another shot for no other reason than I was frustrated. I started the import and realized I forgot to create my temporary swap file again.

I created it in the middle of the import. psql made it a lot further before crashing. It made it through 5 additional tables.

I think there must be a bug allocating memory in psql.

Can you check if there's any swap memory available when the error raises up?

I've remove completely the swap memory in my Linux desktop (just for testing other things...) and I got the exactly same error! I'm pretty sure that this is what is going on with you too.

It is a bit suspicious that you report the same free memory size as your shared_buffers size. Are you sure you are looking the right values?

Output of free command at the time of crash would be useful as well as the content of /proc/meminfo

Beware that setting overcommit_memory to 2 is not so effective if you see the overcommit_ratio to 100. It will basically limits the memory allocation to the size swap (0 in this case) + 100% of physical RAM, which doesn't take into account any space for shared memory and disk caches.

You should probably set overcommit_ratio to 50.