Need advice about which tool to choose?Ask the StackShare community!
Decisions about Kafka, NSQ, and RabbitMQ
Kirill Mikhailov
GO/C developer · | 6 upvotes · 9.3K views
Maybe not an obvious comparison with Kafka, since Kafka is pretty different from rabbitmq. But for small service, Rabbit as a pubsub platform is super easy to use and pretty powerful. Kafka as an alternative was the original choice, but its really a kind of overkill for a small-medium service. Especially if you are not planning to use k8s, since pure docker deployment can be a pain because of networking setup. Google PubSub was another alternative, its actually pretty cheap, but I never tested it since Rabbit was matching really good for mailing/notification services.
Mickael Alliel
DevOps Engineer at Rookout · | 3 upvotes · 126.1K views
In addition to being a lot cheaper, Google Cloud Pub/Sub allowed us to not worry about maintaining any more infrastructure that needed.
We moved from a self-hosted RabbitMQ over to CloudAMQP and decided that since we use GCP anyway, why not try their managed PubSub?
It is one of the better decisions that we made, and we can just focus about building more important stuff!
Pros of Kafka
Pros of NSQ
Pros of RabbitMQ
Pros of Kafka
- High-throughput116
- Distributed110
- Scalable83
- High-Performance77
- Durable62
- Publish-Subscribe34
- Simple-to-use17
- Open source13
- Written in Scala and java. Runs on JVM9
- Message broker + Streaming system6
- Avro schema integration4
- Suport Multiple clients2
- KSQL2
- Partioned, replayable log2
- Robust1
- Extremely good parallelism constructs1
- Simple publisher / multi-subscriber model1
- Fun1
Pros of NSQ
- It's in golang29
- Lightweight20
- Distributed19
- Easy setup18
- High throughput16
- Publish-Subscribe10
- Save data if no subscribers are found7
- Scalable7
- Open source6
- Temporarily kept on disk5
- Simple-to use2
- Free1
- Topics and channels concept1
- Load balanced1
- Primarily in-memory1
Pros of RabbitMQ
- It's fast and it works with good metrics/monitoring224
- Ease of configuration77
- I like the admin interface56
- Easy to set-up and start with49
- Durable20
- Standard protocols18
- Intuitive work through python18
- Written primarily in Erlang10
- Simply superb7
- Completeness of messaging patterns6
- Scales to 1 million messages per second3
- Reliable3
- Better than most traditional queue based message broker2
- Distributed2
- Supports AMQP2
- Great ui1
- Better routing system1
- Inubit Integration1
- Reliability1
- High performance1
- Runs on Open Telecom Platform1
- Clusterable1
- Clear documentation with different scripting language1
Sign up to add or upvote prosMake informed product decisions
Cons of Kafka
Cons of NSQ
Cons of RabbitMQ
Cons of Kafka
- Needs Zookeeper25
- Non-Java clients are second-class citizens24
- Operational difficulties6
- Terrible Packaging1
Cons of NSQ
- Long term persistence1
- HA1
- Get NSQ behavior out of Kafka but not inverse1
Cons of RabbitMQ
- Too complicated cluster/HA config and management8
- Needs Erlang runtime. Need ops good with Erlang runtime6
- Configuration must be done first, not by your code5
- Slow4
Sign up to add or upvote consMake informed product decisions
What is Kafka?
Kafka is a distributed, partitioned, replicated commit log service. It provides the functionality of a messaging system, but with a unique design.
What is NSQ?
NSQ is a realtime distributed messaging platform designed to operate at scale, handling billions of messages per day. It promotes distributed and decentralized topologies without single points of failure, enabling fault tolerance and high availability coupled with a reliable message delivery guarantee. See features & guarantees.
What is RabbitMQ?
RabbitMQ gives your applications a common platform to send and receive messages, and your messages a safe place to live until received.
Need advice about which tool to choose?Ask the StackShare community!
Jobs that mention Kafka, NSQ, and RabbitMQ as a desired skillset
What companies use Kafka?
What companies use NSQ?
What companies use RabbitMQ?
What companies use Kafka?
Sign up to get full access to all the companiesMake informed product decisions
What tools integrate with Kafka?
What tools integrate with NSQ?
What tools integrate with RabbitMQ?
What tools integrate with Kafka?
What tools integrate with NSQ?
No integrations found
What tools integrate with RabbitMQ?
Sign up to get full access to all the tool integrationsMake informed product decisions
Blog Posts
+4 2
582
+8 4
1301
+4 4
937
+6 2
1336
+2 5
1275
+2 7
1675
+17 12
5509
+9 15
1040
What are some alternatives to Kafka, NSQ, and RabbitMQ?
ActiveMQ
Apache ActiveMQ is fast, supports many Cross Language Clients and Protocols, comes with easy to use Enterprise Integration Patterns and many advanced features while fully supporting JMS 1.1 and J2EE 1.4. Apache ActiveMQ is released under the Apache 2.0 License.
Amazon Kinesis
Amazon Kinesis can collect and process hundreds of gigabytes of data per second from hundreds of thousands of sources, allowing you to easily write applications that process information in real-time, from sources such as web site click-streams, marketing and financial information, manufacturing instrumentation and social media, and operational logs and metering data.
Apache Spark
Spark is a fast and general processing engine compatible with Hadoop data. It can run in Hadoop clusters through YARN or Spark's standalone mode, and it can process data in HDFS, HBase, Cassandra, Hive, and any Hadoop InputFormat. It is designed to perform both batch processing (similar to MapReduce) and new workloads like streaming, interactive queries, and machine learning.
Akka
Akka is a toolkit and runtime for building highly concurrent, distributed, and resilient message-driven applications on the JVM.
Apache Storm
Apache Storm is a free and open source distributed realtime computation system. Storm makes it easy to reliably process unbounded streams of data, doing for realtime processing what Hadoop did for batch processing. Storm has many use cases: realtime analytics, online machine learning, continuous computation, distributed RPC, ETL, and more. Storm is fast: a benchmark clocked it at over a million tuples processed per second per node. It is scalable, fault-tolerant, guarantees your data will be processed, and is easy to set up and operate.
Interest over time
News about Kafka
Spring Messaging Projects Maintenance Releases - Integration, AMQP, Kafka
(spring.io)
Jan 29, 2018
Containerizing a Data Ingest Pipeline: Making the JVM Play Nice with Kafka
(blog.newrelic.com)
Jan 29, 2018
Kafka Streams on Heroku
(blog.heroku.com)
Dec 19, 2017
Kafkapocalypse: Monitoring Kafka Without Losing Your Mind
(blog.newrelic.com)
Dec 12, 2017
Apache Kafka - How to Load Test with JMeter
(www.blazemeter.com)
Dec 6, 2017
News about NSQ
More newsNews about RabbitMQ
More news