Why does Powershell file concatenation convert UTF8 to UTF16?

The Out-* cmdlets (like Out-File) format the data, and the default format is unicode.

You can add an -Encoding parameter to Out-file:

Get-Content $metadataPath$iFile | Out-File $cFile -Encoding UTF8 -append

or switch to Add-Content, which doesn't re-format

Get-Content $metadataPath$iFile | Add-Content $cFile

powershell utf-8 utf-16 data-conversion

First, the fact that you get 2 bytes per character indicates that fixed length UTF16 is being used. More accurately, it is called UCS-2. This article explains that file redirection in Powershell causes the output to be in UCS-2. See http://www.kongsli.net/nblog/2012/04/20/powershell-gotchas-redirect-to-file-encodes-in-unicode/. That same article also provides a fix.

CodeHunter