Linux/Python: encoding a unicode string for print

If you're dumping to an ASCII terminal, encode manually using unicode.encode, and specify that errors should be ignored.

u = u'\xa0'u.encode('ascii') # This failsu.encode('ascii', 'ignore') # This replaces failed encoding attempts with empty string

If you want to store unicode files, try this:

u = u'\xa0'print >>open('out', 'w'), u # This failsprint >>open('out', 'w'), u.encode('utf-8') # This is ok

I have now solved this problem. The solution was neither of the answers given. I used the method given at http://wiki.python.org/moin/PrintFails , as given by ChrisJ in one of the comments. That is, I replace sys.stdout with a wrapper that calls unicode encode with the correct arguments. Works very well.

Either wrap all your print statement through a method perform arbitrary unicode -> utf8 conversion or as last resort change the Python default encoding from ascii to utf-8 inside your site.py. In general it is a bad idea printing unicode strings unfiltered to sys.stdout since Python will trigger an implict conversion of unicode strings to the configured default encoding which is ascii.