Why do we need to use rabbitmq
Let me tell you a few reasons that makes using MOM (Message Oriented Middleware) probably the best choice.
Decoupling:
It can decouple/separate the core components of the application. There is no need to bring all the benefits of the decoupled architecture here. I just want to point it out that this is one of the main requirement of writing a quality and maintainable software.
Flexibility:
It is actually very easy to connect two totally different applications written on different languages together by using AMQP protocol. These application will talk to each other by the help of a "translator" which is MOM.
Scalability:
By using MOM we can scale the system horizontally. One message producer can transmit to unlimited number of message consumers a task, a command or a message for processing and for scaling this system all we need to do is just create new message consumers. Lets say we are getting 1000 pictures per second and we must resize them. Solving this problem with traditional methods could be a headache. With MOM we can transmit images to the message consumers which can do their job asynchronously and make sure data integrity is intact.
They are other benefits of using MOM as well but these 3 are the most significant in my opinion.
Twisted is a networking library that implements a number of network protocols as well as allowing you to create your own. One of the protocols that you can use with Twisted is AMQP https://launchpad.net/txamqp
RabbitMQ is an AMQP broker, i.e. a services that runs outside of your application, probably on a separate cluster of servers. AMQP is merely the protocol that is used to communicate with a message queueing broker like RabbitMQ. You get a lot of things from RabbitMQ. You can send messages persistently with guaranteed delivery so they will arrive even if your app crashes, and even if the RabbitMQ broker ends up being restarted. You get load balancing between message consumers if you have multiple consumers on the same queue. You get interoperability with apps in other languages as long as you use a reasonably open serialization format for your message bodies. AMQP allows you to break up a monolithic app into many loosely coupled parts that can run on different servers. This is a big win for long term maintenance of an application.