Multithread request handling in node.js (deployed in Kubernetes and behind Nginx)

I was expecting that the node.js server receives all requests at the same time and handles them in different threads (by generating new threads), but it seems it's not the case. How can I make node.js handle requests in parallel, and utilize all the cores it has?

By design, node.js runs your Javascript only in a single thread. It my use other threads for certain things such as built-in crypto operations that have an asynchronous calling interface or disk I/O, but anything you've written in pure Javascript, it will only run in a single thread. For CPU-intensive Javascript code, you will need to specifically change the design of your code in order to use multiple CPUs for processing that CPU intensive code.

Your options are to use child processes or WorkerThreads. Probably what you want to do is to set up a set of worker threads (probably one per CPU core) and then create a queue for jobs that should be processed by a worker thread. Then, as a job is inserted into the queue, you see if there is an available worker thread and if so, you immediately send the job off to the worker thread. If not, you wait until a worker thread notifies you that it is finished and is available for the next job.

In node.js WorkerThreads are entirely separate instances of the V8 Javascript engine and you can use messages between worker threads and your main process.

I used the cluster module of node.js and followed the approach mentioned here: https://medium.com/tech-tajawal/clustering-in-nodejs-utilizing-multiple-processor-cores-75d78aeb0f4f