Multithreaded compression in C#

I think your best bet is to split the data stream at equal intervals yourself, and launch threads to compress each part separately in parallel, if using non-parallelized algorithms. (After which a single thread concatenates them into a single stream (you can make a stream class that continues reading from the next stream when the current one ends)).

You may wish to take a look at SharpZipLib which is somewhat better than the intrinsic compression streams in .NET.

EDIT: You will need a header to tell where each new stream begins, of course. :)

Found this library: http://www.codeplex.com/sevenzipsharp

Looks like it wraps the unmanaged 7z.dll which does support multithreading. Obviously not ideal having to wrap unmanaged code, but it looks like this is currently the only option that's out there.

I recently found a compression library that supports multithreaded bzip compression:DotNetZip. The nice thing about this library is that the ParallelBZip2OutputStream class is derived from System.IO.Stream and takes a System.IO.Stream as output. This means that you can create a chain of classes derived from System.IO.Stream like:

• ICSharpCode.SharpZipLib.Tar.TarOutputStream
• Ionic.BZip2.ParallelBZip2OutputStream (from the DotNetZip library)
• System.Security.Cryptography.CryptoStream (for encryption)
• System.IO.FileStream

In this case we create a .tar.bz file, encrypt it (maybe with AES) and directly write it to a file.