Does Thread.Sleep hinder other threads?

This is by design. You are seeing the threadpool manager trying to keep a limited number of threads in the executing state. Important to ensure that your program isn't running more threads than your machine has cpu cores. That's inefficient, less work gets done when Windows is forced to start swapping the cores between active threads. The threadpool manager isn't smart enough to know that the thread is sleeping and not actually performing any work.

On a dual-core machine, you'll see the first 2 threads starting right away. Then additional threads are allowed to run, one by one with a one second interval when the thread manager notices that the active threads are not making any progress and are probably blocked. The order in which threads are released and execute the Console.Write() call is not deterministic.

This is an artificial test of course, real threads don't sleep. If you have threads that block for a long time, waiting for an I/O request to complete for example then using threadpool threads (tasks) is not the best solution.

TaskCreationOptions.LongRunning will 'remove' the ThreadPool limitation.
I don't know the easy way to specify TaskCreationOptions.LongRunning for Parallel.For.
However, you can achieve the same effect using Task class:

Action<int> action = i =>    {        Console.Write("start {0} ", i);        Thread.Sleep(5000);        Console.Write("finish {0} ", i);    };var tasks = Enumerable.Range(0, 100)    .Select(arg => Task.Factory.StartNew(() => action(arg), TaskCreationOptions.LongRunning))    .ToArray();Task.WaitAll(tasks);

Without TaskCreationOptions.LongRunning it will run exactly the same way as your Parallel.For did.

I updated the code slightly to show the ThreadID when writing to the Console:

Console.WriteLine("start index:{0} thread id:{1} Time:{2} ", index, Thread.CurrentThread.ManagedThreadId.ToString(), DateTime.Now.ToLongTimeString());Thread.Sleep(5000);ConsoleWriteLine("finish index:{0} thread id:{1} Time:{2} ", index, Thread.CurrentThread.ManagedThreadId.ToString(), DateTime.Now.ToLongTimeString());

My machine is a dual core, here's the output I get. This should give you a sense of what's happening. Remember, the loop may not always run in order, i.e.. 0 to 9, being parallel, it grabs a chunk of your array and runs each item through the lambda.

Output:

start  index:1 thread id:11 Time:11:07:17 PMstart  index:0 thread id:9  Time:11:07:17 PMstart  index:5 thread id:10 Time:11:07:17 PMstart  index:6 thread id:12 Time:11:07:18 PMstart  index:2 thread id:13 Time:11:07:19 PMstart  index:7 thread id:14 Time:11:07:20 PMstart  index:3 thread id:15 Time:11:07:21 PMstart  index:8 thread id:16 Time:11:07:22 PMfinish index:0 thread id:9  Time:11:07:22 PMstart  index:4 thread id:9  Time:11:07:22 PMfinish index:1 thread id:11 Time:11:07:22 PMstart  index:9 thread id:11 Time:11:07:22 PMfinish index:5 thread id:10 Time:11:07:22 PMfinish index:6 thread id:12 Time:11:07:23 PMfinish index:2 thread id:13 Time:11:07:24 PMfinish index:7 thread id:14 Time:11:07:25 PMfinish index:3 thread id:15 Time:11:07:26 PMfinish index:8 thread id:16 Time:11:07:27 PMfinish index:4 thread id:9  Time:11:07:27 PMfinish index:9 thread id:11 Time:11:07:27 PM