Thread pooling in C++11

This is copied from my answer to another very similar post:

1. Start with maximum number of threads the system can support:

   int num_threads = std::thread::hardware_concurrency();

2. For an efficient threadpool implementation, once threads are created according to num_threads, it's better not to create new ones, or destroy old ones (by joining). There will be a performance penalty, an it might even make your application goe slower than the serial version.

Each C++11 thread should be running in their function with an infinite loop, constantly waiting for new tasks to grab and run.

Here is how to attach such a function to the thread pool:

int num_threads = std::thread::hardware_concurrency();std::vector<thread> threads;for (int i = 0; i < num_threads; i++){    pool.push_back(std::thread(Infinite_loop_function));}

3. The infinite loop function. This is a while (true) loop waiting for the task queue.

void Pool::Infinite_loop_function(){    while (true)    {        std::unique_lock<std::mutex> lock(queue_mutex);        condition.wait(lock, [this](){            return !queue.empty() || terminate_pool;        });        Job = queue.front();        queue.pop();        Job(); // function<void()> type    }};

4. Make a function to add job to your queue

void Pool::Add_Job(function<void()> New_Job){    std::unique_lock<std::mutex> lock(queue_mutex);    queue.push(New_Job);    condition.notify_one();}

5. Bind an arbitrary function to your queue

Pool_Obj.Add_Job(std::bind(&Some_Class::Some_Method, &Some_object));

Once you integrate these ingredients, you have your own dynamic threading pool. These threads always run, waiting for job to do.

I apologize if there are some syntax errors, I typed these code and and I have a bad memory. Sorry that I cannot provide you the complete thread pool code, that would violate my job integrity.

Edit: to terminate the pool, call the shutdown() method:

Pool::shutdown(){    std::unique_lock<std::mutex> lock(threadpool_mutex);    terminate_pool = true; // use this flag in condition.wait    condition.notify_all(); // wake up all threads.    // Join all threads.    for(std::thread &th : threads)    {        th.join();    }    pool.clear();      stopped = true; // use this flag in destructor, if not set, call shutdown() }

You can use C++ Thread Pool Library, https://github.com/vit-vit/ctpl.

Then the code your wrote can be replaced with the following

#include <ctpl.h>  // or <ctpl_stl.h> if ou do not have Boost libraryint main (int argc, char *argv[]) {    ctpl::thread_pool p(2 /* two threads in the pool */);    int arr[4] = {0};    std::vector<std::future<void>> results(4);    for (int i = 0; i < 8; ++i) { // for 8 iterations,        for (int j = 0; j < 4; ++j) {            results[j] = p.push([&arr, j](int){ arr[j] +=2; });        }        for (int j = 0; j < 4; ++j) {            results[j].get();        }        arr[4] = std::min_element(arr, arr + 4);    }}

You will get the desired number of threads and will not create and delete them over and over again on the iterations.

A pool of threads means that all your threads are running, all the time – in other words, the thread function never returns. To give the threads something meaningful to do, you have to design a system of inter-thread communication, both for the purpose of telling the thread that there's something to do, as well as for communicating the actual work data.

Typically this will involve some kind of concurrent data structure, and each thread would presumably sleep on some kind of condition variable, which would be notified when there's work to do. Upon receiving the notification, one or several of the threads wake up, recover a task from the concurrent data structure, process it, and store the result in an analogous fashion.

The thread would then go on to check whether there's even more work to do, and if not go back to sleep.

The upshot is that you have to design all this yourself, since there isn't a natural notion of "work" that's universally applicable. It's quite a bit of work, and there are some subtle issues you have to get right. (You can program in Go if you like a system which takes care of thread management for you behind the scenes.)