Kafka vs RabbitMQ vs Redis

Need advice about which tool to choose?Ask the StackShare community!

Kafka

12.7K
11.8K
+ 1
546
RabbitMQ

12.3K
10.4K
+ 1
507
Redis

35.4K
25.6K
+ 1
3.9K
Decisions about Kafka, RabbitMQ, and Redis
Kirill Mikhailov

Maybe not an obvious comparison with Kafka, since Kafka is pretty different from rabbitmq. But for small service, Rabbit as a pubsub platform is super easy to use and pretty powerful. Kafka as an alternative was the original choice, but its really a kind of overkill for a small-medium service. Especially if you are not planning to use k8s, since pure docker deployment can be a pain because of networking setup. Google PubSub was another alternative, its actually pretty cheap, but I never tested it since Rabbit was matching really good for mailing/notification services.

See more
Mickael Alliel
DevOps Engineer at Rookout · | 3 upvotes · 129.7K views

In addition to being a lot cheaper, Google Cloud Pub/Sub allowed us to not worry about maintaining any more infrastructure that needed.

We moved from a self-hosted RabbitMQ over to CloudAMQP and decided that since we use GCP anyway, why not try their managed PubSub?

It is one of the better decisions that we made, and we can just focus about building more important stuff!

See more
Colin Chartier
Founder at Distributed Containers Inc. · | 4 upvotes · 10.8K views

We needed a centralized "job" processor for our CI runs, but continuously had issues with transactions across services:

INSERT INTO ci_jobs(...) VALUES (...) RETURNING id

redis-cli LPUSH $id

wasn't good enough, since a temporary inability to connect to redis would kill the run in a strange way.

Instead, I used postgres itself as the job server with PUBLISH / SUBSCRIBE and an atomic claiming mechanism using FOR UPDATE SKIP LOCKED using Postgres.

See the blog post below for more details:

See more
Pros of Kafka
Pros of RabbitMQ
Pros of Redis
  • 116
    High-throughput
  • 111
    Distributed
  • 84
    Scalable
  • 77
    High-Performance
  • 62
    Durable
  • 34
    Publish-Subscribe
  • 17
    Simple-to-use
  • 13
    Open source
  • 10
    Written in Scala and java. Runs on JVM
  • 6
    Message broker + Streaming system
  • 4
    Avro schema integration
  • 2
    Robust
  • 2
    KSQL
  • 2
    Suport Multiple clients
  • 2
    Partioned, replayable log
  • 1
    Fun
  • 1
    Extremely good parallelism constructs
  • 1
    Flexible
  • 1
    Simple publisher / multi-subscriber model
  • 225
    It's fast and it works with good metrics/monitoring
  • 78
    Ease of configuration
  • 56
    I like the admin interface
  • 49
    Easy to set-up and start with
  • 20
    Durable
  • 18
    Standard protocols
  • 18
    Intuitive work through python
  • 10
    Written primarily in Erlang
  • 7
    Simply superb
  • 6
    Completeness of messaging patterns
  • 3
    Scales to 1 million messages per second
  • 3
    Reliable
  • 2
    Better than most traditional queue based message broker
  • 2
    Distributed
  • 2
    Supports AMQP
  • 1
    Great ui
  • 1
    Better routing system
  • 1
    Inubit Integration
  • 1
    Reliability
  • 1
    High performance
  • 1
    Runs on Open Telecom Platform
  • 1
    Clusterable
  • 1
    Clear documentation with different scripting language
  • 875
    Performance
  • 535
    Super fast
  • 510
    Ease of use
  • 442
    In-memory cache
  • 321
    Advanced key-value cache
  • 189
    Open source
  • 179
    Easy to deploy
  • 163
    Stable
  • 152
    Free
  • 120
    Fast
  • 39
    High-Performance
  • 38
    High Availability
  • 34
    Data Structures
  • 32
    Very Scalable
  • 23
    Replication
  • 20
    Great community
  • 19
    Pub/Sub
  • 17
    "NoSQL" key-value data store
  • 14
    Hashes
  • 12
    Sets
  • 10
    Sorted Sets
  • 9
    Lists
  • 8
    BSD licensed
  • 8
    NoSQL
  • 7
    Async replication
  • 7
    Integrates super easy with Sidekiq for Rails background
  • 7
    Bitmaps
  • 6
    Open Source
  • 6
    Keys with a limited time-to-live
  • 5
    Strings
  • 5
    Lua scripting
  • 4
    Awesomeness for Free!
  • 4
    Hyperloglogs
  • 3
    outstanding performance
  • 3
    Runs server side LUA
  • 3
    Networked
  • 3
    LRU eviction of keys
  • 3
    Written in ANSI C
  • 3
    Feature Rich
  • 3
    Transactions
  • 2
    Data structure server
  • 2
    Performance & ease of use
  • 1
    Existing Laravel Integration
  • 1
    Automatic failover
  • 1
    Easy to use
  • 1
    Object [key/value] size each 500 MB
  • 1
    Simple
  • 1
    Channels concept
  • 1
    Scalable
  • 1
    Temporarily kept on disk
  • 1
    Dont save data if no subscribers are found
  • 0
    Jk

Sign up to add or upvote prosMake informed product decisions

Cons of Kafka
Cons of RabbitMQ
Cons of Redis
  • 26
    Non-Java clients are second-class citizens
  • 25
    Needs Zookeeper
  • 7
    Operational difficulties
  • 1
    Terrible Packaging
  • 9
    Too complicated cluster/HA config and management
  • 6
    Needs Erlang runtime. Need ops good with Erlang runtime
  • 5
    Configuration must be done first, not by your code
  • 4
    Slow
  • 11
    Cannot query objects directly
  • 1
    No WAL
  • 1
    No secondary indexes for non-numeric data types

Sign up to add or upvote consMake informed product decisions

What is Kafka?

Kafka is a distributed, partitioned, replicated commit log service. It provides the functionality of a messaging system, but with a unique design.

What is RabbitMQ?

RabbitMQ gives your applications a common platform to send and receive messages, and your messages a safe place to live until received.

What is Redis?

Redis is an open source, BSD licensed, advanced key-value store. It is often referred to as a data structure server since keys can contain strings, hashes, lists, sets and sorted sets.

Need advice about which tool to choose?Ask the StackShare community!

What companies use Kafka?
What companies use RabbitMQ?
What companies use Redis?

Sign up to get full access to all the companiesMake informed product decisions

What tools integrate with Kafka?
What tools integrate with RabbitMQ?
What tools integrate with Redis?

Sign up to get full access to all the tool integrationsMake informed product decisions

Blog Posts

Jun 24 2020 at 4:42PM
https://img.stackshare.io/stack/673323/default_9dbdf311e98bc0550cc0b928a4f7144ba95c89fc.png logo

Pinterest

Amazon S3KafkaHBase+4
4
949
MySQLKafkaApache Spark+6
2
1351
Jan 7 2020 at 5:09PM
https://img.stackshare.io/stack/527058/default_0abbbd50e9a4e209c9b9b328d003c8cdbce2e9b3.png logo

Ably Realtime

KafkaAbly+2
7
1677
Nov 20 2019 at 3:38AM
https://img.stackshare.io/stack/517248/default_4bf5f3d2d3ef627f563fd3b2e94dee6cc37a38d6.jpg logo

OneSignal

PostgreSQLRedisRuby+8
7
3634
What are some alternatives to Kafka, RabbitMQ, and Redis?
ActiveMQ
Apache ActiveMQ is fast, supports many Cross Language Clients and Protocols, comes with easy to use Enterprise Integration Patterns and many advanced features while fully supporting JMS 1.1 and J2EE 1.4. Apache ActiveMQ is released under the Apache 2.0 License.
Amazon Kinesis
Amazon Kinesis can collect and process hundreds of gigabytes of data per second from hundreds of thousands of sources, allowing you to easily write applications that process information in real-time, from sources such as web site click-streams, marketing and financial information, manufacturing instrumentation and social media, and operational logs and metering data.
Apache Spark
Spark is a fast and general processing engine compatible with Hadoop data. It can run in Hadoop clusters through YARN or Spark's standalone mode, and it can process data in HDFS, HBase, Cassandra, Hive, and any Hadoop InputFormat. It is designed to perform both batch processing (similar to MapReduce) and new workloads like streaming, interactive queries, and machine learning.
Akka
Akka is a toolkit and runtime for building highly concurrent, distributed, and resilient message-driven applications on the JVM.
Apache Storm
Apache Storm is a free and open source distributed realtime computation system. Storm makes it easy to reliably process unbounded streams of data, doing for realtime processing what Hadoop did for batch processing. Storm has many use cases: realtime analytics, online machine learning, continuous computation, distributed RPC, ETL, and more. Storm is fast: a benchmark clocked it at over a million tuples processed per second per node. It is scalable, fault-tolerant, guarantees your data will be processed, and is easy to set up and operate.
See all alternatives
Interest over time
News about RabbitMQ
More news