How `[System.Console]::OutputEncoding/InputEncoding` with Python? How `[System.Console]::OutputEncoding/InputEncoding` with Python? powershell powershell

How `[System.Console]::OutputEncoding/InputEncoding` with Python?


You are piping data into Python; at that point Python's stdin is no longer attached to a TTY (your console) and won't guess at what the encoding might be. Instead, the default system locale is used; on your system that's cp1251 (the Windows Latin-1-based codepage).

Set the PYTHONIOENCODING environment variable to override:

PYTHONIOENCODING
If this is set before running the interpreter, it overrides the encoding used for stdin/stdout/stderr, in the syntax encodingname:errorhandler. Both the encodingname and the :errorhandler parts are optional and have the same meaning as in str.encode().

PowerShell doesn't appear to support per-command-line environment variables the way UNIX shells do; the easiest is to just set the variable first:

Set-Item Env:PYTHONIOENCODING "UTF-8"

or even

Set-Item Env:PYTHONIOENCODING "cp65001"

as the Windows UTF-8 codepage is apparently not quite UTF-8 really, depending on the Windows version and on wether or not pipe redirection is used.


Why not embed CPython in powershell?! CPython is so easy to embed, and powershell is very good REPL to play with .NET and COM objects. Here is a simple introduction to using pythonnet from PowerShell. Note how encoding is automatically propagated from powershell to python.

Windows PowerShellCopyright (C) 2015 Microsoft Corporation. All rights reserved.PS C:\Users\denfromufa> [system.console]::InputEncoding = [System.Text.Encoding]::UTF8;PS C:\Users\denfromufa> [system.console]::OutputEncoding = [System.Text.Encoding]::UTF8;PS C:\Users\denfromufa> [Reflection.Assembly]::LoadFile("C:\Python\Miniconda3_64b\Lib\site-packages\Python.Runtime.dll")GAC    Version        Location---    -------        --------False  v4.0.30319     C:\Python\Miniconda3_64b\Lib\site-packages\Python.Runtime.dllPS C:\Users\denfromufa> $gil = [Python.Runtime.Py]::GIL()PS C:\Users\denfromufa> $sys=[Python.Runtime.Py]::Import("sys")PS C:\Users\denfromufa> $sys.stdin.encoding.ToString()cp65001PS C:\Users\denfromufa> $sys.stdout.encoding.ToString()cp65001PS C:\Users\denfromufa> $gil.Dispose()PS C:\Users\denfromufa> [Python.Runtime.PythonEngine]::Shutdown()PS C:\Users\denfromufa>

[EDIT]

Here is snek package that was released by one of powershell developers for embedding Python in powershell:

https://github.com/adamdriscoll/snek