RabbitMQ vs Apache Kafka

RabbitMQ

RabbitMQ is a general purpose message broker that can support various messaging protocols. It is an older technology that can support high throughput, and is still widely used for messaging use cases. It can handle message passing between micro-services or to trigger background asynchronous jobs.

RabbitMQ is based on Push based model
RabbitMQ allows the messages to be processed only once.
RabbitMQ is typically used for messages with headers, keys (typically bigger message size)
RabbitMQ supports various messaging protocols.
RabbitMQ has different variations of message broker and bindings, like one-to-one, based on topic, and other forms of pub-sub mechanism.
RabbitMQ stores messages almost in-memory using a big cluster.
RabbitMQ has a smart exchange and routing, with tight coupling with consumers.

Apache Kafka

Apache Kafka is a highly durable message broker supporting high-incoming message stream. It is extremely fast and allows messages to be processed and re-processed by other consumers.

Apache Kafka is based on topics, upon which the messages are partitioned by. Consumers subscribe to topics and consumer messages based on them.
Kafka also stores messages for a fixed retention period and hence can also be used as a storage medium. Hence it also allows messages to be replayed and consumed multiple times as long as it is within the retention period.
Kafka consumers keeps track of the last messages read in the stream, and hence supports checkpointing.
Kafka is based on pull approach, where consumers retrieves messages and controls the messages being received. This also allows to reprocess messages in case of failure .
Kafka can support millions of messages per second.
Kafka also allows streaming and can be used along with other streaming computation systems like spark for real time computations.
Kafka stores messages on disk.

Criteria

RabbitMQ/Kafka

Millions messages per second

Kafka

Low Latency

Kafka

Dumb consumers

RabbitMQ

Data Retention

Kafka

Complex messages

RabbitMQ

One time Background Jobs

RabbitMQ

Message Reprocessing

Kafka

For More Reading

PreviousMessage Queue NextConcurrency

Last updated 4 years ago

Was this helpful?

RabbitMQ

RabbitMQ is based on Push based model

RabbitMQ allows the messages to be processed only once.

RabbitMQ is typically used for messages with headers, keys (typically bigger message size)

RabbitMQ supports various messaging protocols.

RabbitMQ has different variations of message broker and bindings, like one-to-one, based on topic, and other forms of pub-sub mechanism.

RabbitMQ stores messages almost in-memory using a big cluster.

RabbitMQ has a smart exchange and routing, with tight coupling with consumers.

Apache Kafka

Apache Kafka is a highly durable message broker supporting high-incoming message stream. It is extremely fast and allows messages to be processed and re-processed by other consumers.

Apache Kafka is based on topics, upon which the messages are partitioned by. Consumers subscribe to topics and consumer messages based on them.

Kafka also stores messages for a fixed retention period and hence can also be used as a storage medium. Hence it also allows messages to be replayed and consumed multiple times as long as it is within the retention period.

Kafka consumers keeps track of the last messages read in the stream, and hence supports checkpointing.

Kafka is based on pull approach, where consumers retrieves messages and controls the messages being received. This also allows to reprocess messages in case of failure .

Kafka can support millions of messages per second.

Kafka also allows streaming and can be used along with other streaming computation systems like spark for real time computations.

Kafka stores messages on disk.

Criteria

RabbitMQ/Kafka

Millions messages per second

Kafka

Low Latency

Kafka

Dumb consumers

RabbitMQ

Data Retention

Kafka

Complex messages

RabbitMQ

One time Background Jobs

RabbitMQ

Message Reprocessing

Kafka