Question about mongodb capped collections + tailable cursors Question about mongodb capped collections + tailable cursors database database

Question about mongodb capped collections + tailable cursors


MongoDB documents contain a thorough description of ways to achieve an atomic update. You cannot ensure that only one process receives the new document but you can implement an atomic update after receiving it to ensure that only one process acts on it.


I have recently been looking into this problem and I would be interested to know if there are other ways to have multiple readers (consumers) without relying on atomic updates.

This is what I have come up with: divide your logic into two "modules". The first module will be responsible for fetching new documents from the tailable cursor. The second module will be responsible for working with an arbitrary document. In this manner, you can have only one consumer (module one) fetching documents which later sends the document to multiple document workers (second module).

Both modules can be implemented in different processes and even in different languages. For example, a Node.js app could be fetching the documents and sending them to a pool of scripts written in Python ready to process documents concurrently.