Looking for clarity on Event Hubs vs Topics in Azure Service Bus
If you have the choice it's almost always easier to write a system based around a full enterprise pubsub messaging system where you can mark single events as having been consumed, retry messages, and just about every other wonderful feature. If you've already accepted partitioning your message channel (which Azure Service Bus Topics appear to support) then you could in principle scale a more full featured messaging system to the degree you require. The issue is at what cost?
An Azure Service Bus Topic has a cost at high scale of approximately $0.20 per Million messages, Amazon SQS (somewhat similar) lists $0.50 per Million. If you host it yourself you'll likely need to set up a lot of RabbitMQ servers or even multiple clusters as you partition.
Azure Event Hub costs $0.028 per Million plus an amount per throughput unit, same for Amazon Kinesis. Apache Kafka has been benchmarked at 2 Million per second on 3 machines
At say 20,000 events per second sustained the difference between some Azure Topics and Azure Event Hub is in the range of a full time developer's salary. At 2 million per second sustained (which requires contacting MS), the difference is approaching $1M/month.
Basically use the partitioned stream|log / offset tracking systems when you either don't need all the useful features of a full messaging system, or when you don't need them enough to pay the ~10X premium. (Or can't use them because you can't scale the proper messaging system enough without heroic efforts).
i wrote a post a while ago about this topic with some support from Dan on the Service Bus Team. Hopefully this should clarify for you
http://microsoftintegration.guru/2015/03/03/azure-event-hubs-vs-azure-messaging/
Service Bus (Messaging)
For messaging it’s about one application telling one or more apps to DO SOMETHING or GIVE ME SOMETHING.
Event Hub (Eventing)
The alternative is that in eventing the applications are saying SOMETHING HAS HAPPENED.
Correct!!
The fundamental difference between EventHubs and Topics is - TOPICS offer per-message semantics - whereas, EventHubs - offer Stream Semantics - implies, one should not expect any per-message feature/semantics with EventHubs.
Any middle-tier providing per-message features comes with the processing overhead (the tax)!!
For Ex: Per message Duplicate detection, Receive confirmation per message (topics have a Message.Complete to ack Msg received) - are all Topic features. EventHubs narrows-down the feature set to provide a better low-latency/high-throughput solution.
To visualize features like at-least-once delivery (of per msg is not available in EventHubs) is to translate it to stream semantics - Read until a point in a Given eventHub partition and checkpoint and let your application which is consuming those events handle the at-least-once delivery.