Python Unicode Encode Error

Likely, your problem is that you parsed it okay, and now you're trying to print the contents of the XML and you can't because theres some foreign Unicode characters. Try to encode your unicode string as ascii first:

unicodeData.encode('ascii', 'ignore')

the 'ignore' part will tell it to just skip those characters. From the python docs:

>>> # Python 2: u = unichr(40960) + u'abcd' + unichr(1972)>>> u = chr(40960) + u'abcd' + chr(1972)>>> u.encode('utf-8')'\xea\x80\x80abcd\xde\xb4'>>> u.encode('ascii')Traceback (most recent call last):  File "<stdin>", line 1, in ?UnicodeEncodeError: 'ascii' codec can't encode character '\ua000' in position 0: ordinal not in range(128)>>> u.encode('ascii', 'ignore')'abcd'>>> u.encode('ascii', 'replace')'?abcd?'>>> u.encode('ascii', 'xmlcharrefreplace')'&#40960;abcd&#1972;'

You might want to read this article: http://www.joelonsoftware.com/articles/Unicode.html, which I found very useful as a basic tutorial on what's going on. After the read, you'll stop feeling like you're just guessing what commands to use (or at least that happened to me).

A better solution:

if type(value) == str:    # Ignore errors even if the string is not proper UTF-8 or has    # broken marker bytes.    # Python built-in function unicode() can do this.    value = unicode(value, "utf-8", errors="ignore")else:    # Assume the value object has proper __unicode__() method    value = unicode(value)

If you would like to read more about why:

http://docs.plone.org/manage/troubleshooting/unicode.html#id1

Don't hardcode the character encoding of your environment inside your script; print Unicode text directly instead:

assert isinstance(text, unicode) # or str on Python 3print(text)

If your output is redirected to a file (or a pipe); you could use PYTHONIOENCODING envvar, to specify the character encoding:

$ PYTHONIOENCODING=utf-8 python your_script.py >output.utf8

Otherwise, python your_script.py should work as is -- your locale settings are used to encode the text (on POSIX check: LC_ALL, LC_CTYPE, LANG envvars -- set LANG to a utf-8 locale if necessary).

To print Unicode on Windows, see this answer that shows how to print Unicode to Windows console, to a file, or using IDLE.