Curl Powershell windows 10 slower than command prompt why?

To complement briantist's helpful answer:

• In PowerShell, the redirection operators > and >> are in effect aliases of Out-File and Out-File -Append.

• > and >> are therefore not mere byte-stream conduits, and, in fact, PowerShell as of v7.2 does not support sending raw byte output to a file.

  • Instead, PowerShell invariably decodes output from any external program as text ([string] instances), based on the character encoding reported by [Console]::OutputEncoding] and then, on saving to the target file with Out-File (possibly via > / >>), re-encodes these strings, using that cmdlet's default character encoding (unless overridden with -Encoding in an explicit Out-File call).

  • Not only does this not preserve the external program's raw byteoutput, it adds significant overhead.

To get raw byte processing, call cmd.exe^[1] and use its redirection operators:

cmd /c 'curl.exe yourfileonS3 >> output.bin'

See this answer for more information.

^{[1] On Unix-like platforms, use sh -c 'curl yourfileonS3 >> output.bin'}

See mklement0's answer for full context on this (I recommend accepting that one!), and the important point that handling of byte streams in redirection is problematic and error prone in PowerShell and should be avoided.

So I looked into this and I believe the reason is that >> (file redirection) is the slow part.

I originally suspected you might be calling curl (which is aliased to Invoke-WebRequest in Windows PowerShell), but I was able to reproduce the speed difference between curl.exe directly in PowerShell vs cmd.exe, and measure it, this way:

# call curl.exe and do redirection in PowerShellMeasure-Command -Expression { curl.exe https://uploader.codecov.io/v0.1.0_6943/linux/codecov >> delme.bin }del delme.bin# call cmd.exe and do redirection thereMeasure-Command -Expression { & cmd.exe /c 'curl.exe https://uploader.codecov.io/v0.1.0_6943/linux/codecov >> delme.bin' }del delme.bin

This was enough to show a stark difference.

I also confirmed that this problem is a little bit worse in Windows PowerShell as opposed to later cross-platform versions (pwsh.exe). In Windows, with version 7.1.0, the same commands above still show a large difference.