How does Git-Bash for Windows 'rm' work?
I was able to figure out how this works.
Interestingly, I used Git-Bash's strace utility on it's rm command.
It turns out that Git Bash uses CygWin, and the delete routine is found in the CygWin syscalls.cc file which tries to delete a file in a few different ways.
Eventually it tries to move the file to the Recycle Bin, where it deletes the locked file by opening it with the Windows Driver call NtOpenFile() with a FILE_DELETE_ON_CLOSE flag, then closing it using NtClose().
Not sure if it would be proper to copy CygWin's code into the response here, but details on all of the above can be found in the link provided.