How to fix ''UnicodeDecodeError: 'charmap' codec can't decode byte 0x9d in position 29815: character maps to <undefined>''? How to fix ''UnicodeDecodeError: 'charmap' codec can't decode byte 0x9d in position 29815: character maps to <undefined>''? python python

How to fix ''UnicodeDecodeError: 'charmap' codec can't decode byte 0x9d in position 29815: character maps to <undefined>''?


As you see from https://en.wikipedia.org/wiki/Windows-1252, the code 0x9D is not defined in CP1252.

The "error" is e.g. in your open function: you do not specify the encoding, so python (just in windows) will use some system encoding. In general, if you read a file that maybe was not create in the same machine, it is really better to specify the encoding.

I recommend to put also a coding also on your open for writing the csv. It is really better to be explicit.

I do no know the original file format, but adding to open , encoding='utf-8' is usually a good thing (and it is the default in Linux and MacOs).


Add encoding in the open statementFor example:

f=open("filename.txt","r",encoding='utf-8')


The above did not work for me, try this instead: , errors='ignore'Worked wonders!