Mongo tailable cursors vs Redis pub/sub Mongo tailable cursors vs Redis pub/sub mongodb mongodb

Mongo tailable cursors vs Redis pub/sub


Actually, they are very different beasts.

A MongoDB tailable cursor would work a bit like a queue. It can work with a capped collection so you do not have to explicitly delete items in the collection. It is quite efficient, but keep in mind that MongoDB will lock the whole collection (the database actually) at each write operation, so it limits the scalability. Another scalability limitation is the number of connections. Each client connection will add a connection thread in the mongod servers (or mongos).

Still you can expect tens of thousands of items per second without major problems, which may be enough for a range of applications.

On the other hand, Redis can generally handle much more connections simultaneously, because each connection does not create a thread (Redis is a single-theaded event loop). It is also extremely CPU efficient, because it does not queue at all the items. With Redis pub/sub, the items are propagated to the subscribers in the same event loop iteration than the publication. The items are not even stored in memory, Redis does not even have a single index to maintain. They are only retrieved from a socket buffer to be pushed in another socket buffer.

However, because there is no queuing, delivery of Redis pub/sub messages is not guaranteed at all. If a subscriber is down when a message is published, the message will be lost for this subscriber.

With Redis, you can expect hundreds of thousands of items per second on a single core, especially if you use pipelining, and multiple publication clients.