.ssh/id_rsa failed: permission denied
For some reasons, the id_rsa file in the ~/.ssh folder was in read-only mode for my user (0400). I changed that to read-write (0600) with
chmod 0600 id_rsa
and after I was obviously able to overwrite the file. I guess these are the highest permissions you can give to this file, as others wouldn't make too much sense.