Node JS workers - any need for them? Node JS workers - any need for them? node.js node.js

Node JS workers - any need for them?


You're right, continuations are quite nice in node and if you're running everything in a single node process there is no immediate need for a queue.However, as node is single threaded, node won't be able to handle any new incoming requests while it's busy sending that email or processing that task (if it's a cpu intensive task)

So if your tasks take a while to process cpu wise, it might still be worthwhile using an external queue and a seperate process to handle those tasks/messages.If your tasks are io intensive and take a while because they are waiting for responses from other servers for example, then there's less of a need again as node deals well with io.

If you have a cpu intensive task, but you don't want to deploy a queue you could just create more node processes and machine instances and load balance across them, all letting them process these tasks. The disadvantage of this approach would be the you can't scale the website and background processing separately. (e.g. 5 instances dealing with web requests and a 2 worker instances)


No, there are always use cases for queues, even in the Node world. A basic approach that I've seen several people use to varying degrees of success is using a Redis backed queue to store messages or tasks. You may have one Node process adding items to the queue, with another Node instance processing items from the queue. In addition, take a look at the node modules list for queue modules, and you'll see a decent amount of implementations.