"The operation completed successfully" exception
It looks like exception was raised when WPF tried to allocate native Win32 window handle for another new Dispatcher. Maybe you have reached the maximum amount of handles for your process?
Did you try to force garbage collection on each iteration? The most WPF classes doesn't implement IDisposable, but they still use unmanaged resources, which are managed inside WPF.
There is another way for overflow I can imagine. The CurrentDispatcher property creates a new Dispatcher for each thread. Every Dispatcher never stop until InvokeShutdown will be called. So it means that a thread where a Dispatcher works will never be terminated, because there is no Window and there is nobody to press the Close button. Perhaps it forces Parallel.For implementation to allocate a new Thread for a next iteration, which also need another new Dispatcher and another new handle. Unfortunately, I don't have much experience in using Parallel, so this is only supposition.