Git tags with quote names on Windows
Note: This answer has been torn apart and rewritten from its original version
I wonder if you're having issues with the filesystem's permitted characters in filenames. For example, according to Wikipedia, FAT does not permit double-quotes ("). According to Wikipedia, NTFS will accept double-quotes only if a filename is in the Posix namespacem, but I cannot seem to find that information anywhere else. Further, Microsoft seems to recommend avoiding double-quotes in filenames. Posix appears to only allow A-Za-z_.- in "portable" filenamesยน. See Wikipedia on filesystem limitations and filesystem capabilities for more.
Perhaps you might try the Cygwin version of Git. It appears that Cygwin manages to support quotes in filenames on Windows.