os.popen().read() - charmap decoding error os.popen().read() - charmap decoding error python-3.x python-3.x

os.popen().read() - charmap decoding error


os.popen is just a wrapper around subprocess.Popen along with a io.TextIOWrapper object:

The returned file object reads or writes text strings rather than bytes.

If Python's default encoding doesn't work for you, you should use subprocess.Popen directly.

The underlying issue is that cmd writes ansi garbage by default, even when the output is to a pipe. This behavior may depend on your Windows version.

You can fix this by passing /U flag to cmd:

p = subprocess.Popen('cmd /u /c dir', stdout=subprocess.PIPE)result = p.communicate()text = result[0].decode('u16')


In this case, using subprocess.Popen is too general, too verbose and too hard to remember. Use subprocess.check_output instead.

It returns a bytes object, which can be converted to str with decode function.

import subprocessx = subprocess.check_output(['ls','/'])print(x.decode('utf-8'))

Try it online!


If someone used the with-statement with the combination of readline() in python2 like me(for a timezone Util in Windows), it won't work for python3:

with os.popen("tzutil /l") as source:    key, value = self.get_key_value(source, True)    while value and key:        timezones_to_json.append({u"key": key, u"value": value, u"toolTip": key})        key, value = self,get_key_value(source, False)return timezones_to_jsondef get_key_value(self, source, first=False):    if not first:        source.readline()    value = source.stdout.readline().strip()    key = source.stdout.readline().strip()    return key, value

So my changes to python3 were:

  1. like @Josh Lee said I used the subprocess.Popen instead, but than I had an AttributeError: __exit__

  2. So you had to Insert .stdout at the end, so the object in the with-statement has __enter__ and __exit__ methods:

    with subprocess.Popen(['tzutil', '/l'], stdout=subprocess.PIPE).stdout as source: