RabbitMQ vs Kafka

Author: Shazin Sadakath


When it comes to machine to machine messaging there are so many choices. From proprietary to open source the list is so long and filled with competing products. But there are two choices which are widely used by the tech sphere. 

RabbitMQ which was introduced in 2007 is a traditional smart queue for dumb consumers where as Apache Kafka which was introduced in 2011 is a cutting edge dumb queue for smart consumers. Lets have a look at the difference between the two while evaluating the pros and cons.

Beginning

RabbitMQ

Apache Kafka

Architecture

RabbitMQ

Simplified overall RabbitMQ architecture. Source: http://kth.diva-portal.org/smash/get/diva2:813137/FULLTEXT01.pdf 

Apache Kafka

Global Apache Kafka architecture (with 1 topic, 1 partition, replication factor 4). Source: http://kth.diva-portal.org/smash/get/diva2:813137/FULLTEXT01.pdf 

Use Cases

RabbitMQ

A general purpose messaging solution, often used to allow web servers to respond to requests quickly instead of being forced to perform resource-heavy procedures while the user waits for the result. It’s also good for distributing a message to multiple recipients for consumption or for balancing loads between workers under high load (20k+/sec).  When your requirements extend beyond throughput, RabbitMQ has a lot to offer: features for reliable delivery, routing, federation, HA, security, management tools and other features.  Let’s examine some scenarios best for RabbitMQ, like:

Apache Kafka

Includes the broker itself, which is actually the best known and the most popular part of it, and has been designed and prominently marketed towards stream processing scenarios. In addition to that, Apache Kafka has recently added Kafka Streams which positions itself as an alternative to streaming platforms such as Apache Spark, Apache Flink, Apache Beam/Google Cloud Data Flow and Spring Cloud Data Flow. The documentation does a good job of discussing popular use cases like Website Activity Tracking, Metrics, Log Aggregation, Stream Processing, Event Sourcing and Commit logs. One of those use cases it describes is messaging, which can generate some confusion.  So let’s unpack that a bit and get some clarity on which messaging scenarios are best for Kafka for, like:

Developer Friendliness 

RabbitMQ

Apache Kafka

Security and DevOps

RabbitMQ

Apache Kafka

Performance

RabbitMQ

Apache Kafka

Conclusion

This post is a summary of a post written by Pivotal named Understanding When to use RabbitMQ or Apache Kafka. They have based a Master's thesis by Nicolas Nannoni which has a lot more side by side comparison of the two products in depth. 



Tags: RabbitMQ ApacheKafka
Views: 1723
Register for more exciting articles

Comments

Please login or register to post a comment.


There are currently no comments.