Alternatives to RabbitMQ logo

Alternatives to RabbitMQ

Kafka, ActiveMQ, ZeroMQ, Amazon SNS, and Redis are the most popular alternatives and competitors to RabbitMQ.
21.3K
557

What is RabbitMQ and what are its top alternatives?

RabbitMQ is a popular open-source message broker that implements the Advanced Message Queuing Protocol (AMQP). It is designed to facilitate communication between different systems by acting as a middleman for message exchange. RabbitMQ features include message queuing, publish/subscribe messaging, routing, and reliability mechanisms. However, some limitations of RabbitMQ include potential message loss in certain scenarios, high memory usage, and limited support for complex message routing scenarios.

  1. Apache Kafka: Apache Kafka is a distributed streaming platform with high-throughput, fault-tolerant messaging capabilities. Its key features include horizontal scalability, fault tolerance, and real-time data streaming. Compared to RabbitMQ, Kafka is better suited for handling large volumes of data and enables real-time processing, but it may have a steeper learning curve for beginners.

  2. ActiveMQ: Apache ActiveMQ is an open-source message broker that supports multiple messaging protocols and provides features such as message queuing, topic-based messaging, and message persistence. Compared to RabbitMQ, ActiveMQ has better support for complex messaging patterns like message groups and message selectors, but it may face performance issues under heavy loads.

  3. ZeroMQ: ZeroMQ is a lightweight messaging library that allows building distributed messaging systems with low latency and high throughput. Its key features include support for various messaging patterns, scalability, and simplicity. Compared to RabbitMQ, ZeroMQ is more lightweight and faster, but it lacks built-in features like message queuing and persistence.

  4. NATS: NATS is a high-performance messaging system that provides publish/subscribe and request/reply messaging patterns. Its key features include simplicity, scalability, and high performance. Compared to RabbitMQ, NATS is more lightweight and faster, but it may not offer as many advanced features for complex messaging scenarios.

  5. Redis: Redis is an open-source, in-memory data structure store that can be used as a message broker for real-time messaging. Its key features include high performance, persistence, and support for various data structures. Compared to RabbitMQ, Redis is faster for simple messaging scenarios but may lack advanced features like message queuing and routing.

  6. Amazon SQS: Amazon Simple Queue Service (SQS) is a fully managed message queuing service provided by AWS. Its key features include scalability, reliability, and seamless integration with other AWS services. Compared to RabbitMQ, Amazon SQS is a managed service that offloads maintenance tasks but may have limitations in terms of customization and control.

  7. Google Cloud Pub/Sub: Google Cloud Pub/Sub is a scalable and durable real-time messaging service provided by Google Cloud Platform. Its key features include global availability, horizontal scalability, and integration with other GCP services. Compared to RabbitMQ, Google Cloud Pub/Sub is a fully managed service with high availability and scalability but may have higher costs for usage.

  8. IBM MQ: IBM MQ is a reliable, secure, and scalable messaging middleware that enables communication between applications and systems. Its key features include support for multiple messaging protocols, high availability, and data encryption. Compared to RabbitMQ, IBM MQ is better suited for enterprise environments with advanced security and compliance requirements but may have higher costs and complexity.

  9. KubeMQ: KubeMQ is a Kubernetes-native message broker and message queueing system that simplifies the process of building scalable and resilient microservices architecture. Its key features include seamless integration with Kubernetes, high performance, and automatic failover. Compared to RabbitMQ, KubeMQ is easier to deploy and manage in containerized environments but may have limited support for complex messaging patterns.

  10. Mosquitto: Eclipse Mosquitto is an open-source MQTT broker that provides lightweight messaging capabilities for IoT applications. Its key features include support for the MQTT protocol, scalability, and low resource usage. Compared to RabbitMQ, Mosquitto is optimized for IoT use cases with low bandwidth and resource constraints but may lack features for more general-purpose messaging scenarios.

Top Alternatives to RabbitMQ

  • Kafka
    Kafka

    Kafka is a distributed, partitioned, replicated commit log service. It provides the functionality of a messaging system, but with a unique design. ...

  • ActiveMQ
    ActiveMQ

    Apache ActiveMQ is fast, supports many Cross Language Clients and Protocols, comes with easy to use Enterprise Integration Patterns and many advanced features while fully supporting JMS 1.1 and J2EE 1.4. Apache ActiveMQ is released under the Apache 2.0 License. ...

  • ZeroMQ
    ZeroMQ

    The 0MQ lightweight messaging kernel is a library which extends the standard socket interfaces with features traditionally provided by specialised messaging middleware products. 0MQ sockets provide an abstraction of asynchronous message queues, multiple messaging patterns, message filtering (subscriptions), seamless access to multiple transport protocols and more. ...

  • Amazon SNS
    Amazon SNS

    Amazon Simple Notification Service makes it simple and cost-effective to push to mobile devices such as iPhone, iPad, Android, Kindle Fire, and internet connected smart devices, as well as pushing to other distributed services. Besides pushing cloud notifications directly to mobile devices, SNS can also deliver notifications by SMS text message or email, to Simple Queue Service (SQS) queues, or to any HTTP endpoint. ...

  • Redis
    Redis

    Redis is an open source (BSD licensed), in-memory data structure store, used as a database, cache, and message broker. Redis provides data structures such as strings, hashes, lists, sets, sorted sets with range queries, bitmaps, hyperloglogs, geospatial indexes, and streams. ...

  • Gearman
    Gearman

    Gearman allows you to do work in parallel, to load balance processing, and to call functions between languages. It can be used in a variety of applications, from high-availability web sites to the transport of database replication events. ...

  • Beanstalkd
    Beanstalkd

    Beanstalks's interface is generic, but was originally designed for reducing the latency of page views in high-volume web applications by running time-consuming tasks asynchronously. ...

  • gRPC
    gRPC

    gRPC is a modern open source high performance RPC framework that can run in any environment. It can efficiently connect services in and across data centers with pluggable support for load balancing, tracing, health checking... ...

RabbitMQ alternatives & related posts

Kafka logo

Kafka

23.5K
22K
607
Distributed, fault tolerant, high throughput pub-sub messaging system
23.5K
22K
+ 1
607
PROS OF KAFKA
  • 126
    High-throughput
  • 119
    Distributed
  • 92
    Scalable
  • 86
    High-Performance
  • 66
    Durable
  • 38
    Publish-Subscribe
  • 19
    Simple-to-use
  • 18
    Open source
  • 12
    Written in Scala and java. Runs on JVM
  • 9
    Message broker + Streaming system
  • 4
    KSQL
  • 4
    Avro schema integration
  • 4
    Robust
  • 3
    Suport Multiple clients
  • 2
    Extremely good parallelism constructs
  • 2
    Partioned, replayable log
  • 1
    Simple publisher / multi-subscriber model
  • 1
    Fun
  • 1
    Flexible
CONS OF KAFKA
  • 32
    Non-Java clients are second-class citizens
  • 29
    Needs Zookeeper
  • 9
    Operational difficulties
  • 5
    Terrible Packaging

related Kafka posts

Nick Rockwell
SVP, Engineering at Fastly · | 46 upvotes · 4.1M views

When I joined NYT there was already broad dissatisfaction with the LAMP (Linux Apache HTTP Server MySQL PHP) Stack and the front end framework, in particular. So, I wasn't passing judgment on it. I mean, LAMP's fine, you can do good work in LAMP. It's a little dated at this point, but it's not ... I didn't want to rip it out for its own sake, but everyone else was like, "We don't like this, it's really inflexible." And I remember from being outside the company when that was called MIT FIVE when it had launched. And been observing it from the outside, and I was like, you guys took so long to do that and you did it so carefully, and yet you're not happy with your decisions. Why is that? That was more the impetus. If we're going to do this again, how are we going to do it in a way that we're gonna get a better result?

So we're moving quickly away from LAMP, I would say. So, right now, the new front end is React based and using Apollo. And we've been in a long, protracted, gradual rollout of the core experiences.

React is now talking to GraphQL as a primary API. There's a Node.js back end, to the front end, which is mainly for server-side rendering, as well.

Behind there, the main repository for the GraphQL server is a big table repository, that we call Bodega because it's a convenience store. And that reads off of a Kafka pipeline.

See more
Ashish Singh
Tech Lead, Big Data Platform at Pinterest · | 38 upvotes · 3.3M views

To provide employees with the critical need of interactive querying, we’ve worked with Presto, an open-source distributed SQL query engine, over the years. Operating Presto at Pinterest’s scale has involved resolving quite a few challenges like, supporting deeply nested and huge thrift schemas, slow/ bad worker detection and remediation, auto-scaling cluster, graceful cluster shutdown and impersonation support for ldap authenticator.

Our infrastructure is built on top of Amazon EC2 and we leverage Amazon S3 for storing our data. This separates compute and storage layers, and allows multiple compute clusters to share the S3 data.

We have hundreds of petabytes of data and tens of thousands of Apache Hive tables. Our Presto clusters are comprised of a fleet of 450 r4.8xl EC2 instances. Presto clusters together have over 100 TBs of memory and 14K vcpu cores. Within Pinterest, we have close to more than 1,000 monthly active users (out of total 1,600+ Pinterest employees) using Presto, who run about 400K queries on these clusters per month.

Each query submitted to Presto cluster is logged to a Kafka topic via Singer. Singer is a logging agent built at Pinterest and we talked about it in a previous post. Each query is logged when it is submitted and when it finishes. When a Presto cluster crashes, we will have query submitted events without corresponding query finished events. These events enable us to capture the effect of cluster crashes over time.

Each Presto cluster at Pinterest has workers on a mix of dedicated AWS EC2 instances and Kubernetes pods. Kubernetes platform provides us with the capability to add and remove workers from a Presto cluster very quickly. The best-case latency on bringing up a new worker on Kubernetes is less than a minute. However, when the Kubernetes cluster itself is out of resources and needs to scale up, it can take up to ten minutes. Some other advantages of deploying on Kubernetes platform is that our Presto deployment becomes agnostic of cloud vendor, instance types, OS, etc.

#BigData #AWS #DataScience #DataEngineering

See more
ActiveMQ logo

ActiveMQ

609
1.3K
77
A message broker written in Java together with a full JMS client
609
1.3K
+ 1
77
PROS OF ACTIVEMQ
  • 18
    Easy to use
  • 14
    Open source
  • 13
    Efficient
  • 10
    JMS compliant
  • 6
    High Availability
  • 5
    Scalable
  • 3
    Distributed Network of brokers
  • 3
    Persistence
  • 3
    Support XA (distributed transactions)
  • 1
    Docker delievery
  • 1
    Highly configurable
  • 0
    RabbitMQ
CONS OF ACTIVEMQ
  • 1
    ONLY Vertically Scalable
  • 1
    Support
  • 1
    Low resilience to exceptions and interruptions
  • 1
    Difficult to scale

related ActiveMQ posts

I want to choose Message Queue with the following features - Highly Available, Distributed, Scalable, Monitoring. I have RabbitMQ, ActiveMQ, Kafka and Apache RocketMQ in mind. But I am confused which one to choose.

See more
Naushad Warsi
software developer at klingelnberg · | 1 upvote · 786.6K views
Shared insights
on
ActiveMQActiveMQRabbitMQRabbitMQ

I use ActiveMQ because RabbitMQ have stopped giving the support for AMQP 1.0 or above version and the earlier version of AMQP doesn't give the functionality to support OAuth.

If OAuth is not required and we can go with AMQP 0.9 then i still recommend rabbitMq.

See more
ZeroMQ logo

ZeroMQ

260
585
71
Fast, lightweight messaging library that allows you to design complex communication system without much effort
260
585
+ 1
71
PROS OF ZEROMQ
  • 23
    Fast
  • 20
    Lightweight
  • 11
    Transport agnostic
  • 7
    No broker required
  • 4
    Low level APIs are in C
  • 4
    Low latency
  • 1
    Open source
  • 1
    Publish-Subscribe
CONS OF ZEROMQ
  • 5
    No message durability
  • 3
    Not a very reliable system - message delivery wise
  • 1
    M x N problem with M producers and N consumers

related ZeroMQ posts

Shared insights
on
MongoDBMongoDBZeroMQZeroMQSpring BootSpring Boot

In our Spring Boot application, which encompasses various projects, we employ ZeroMQ (ZMQ) for communication via a req/resp pattern. Recently, I observed that data is persisted in the MongoDB database before being transmitted to other applications. I've identified a method to monitor changes to the database, and I'm contemplating whether to utilize this monitoring approach to detect changes and execute the necessary instructions.

Which approach is more advisable in this scenario: leveraging the database monitoring mechanism or sticking with the current ZMQ req/resp communication?

Essentially, I'm seeking guidance on whether to rely on database monitoring for change detection and subsequent actions or to continue with the existing ZMQ communication pattern.

See more
Meili Triantafyllidi
Software engineer at Digital Science · | 6 upvotes · 476.4K views
Shared insights
on
Amazon SQSAmazon SQSRabbitMQRabbitMQZeroMQZeroMQ

Hi, we are in a ZMQ set up in a push/pull pattern, and we currently start to have more traffic and cases that the service is unavailable or stuck. We want to: * Not loose messages in services outages * Safely restart service without losing messages (ZeroMQ seems to need to close the socket in the receiver before restart manually)

Do you have experience with this setup with ZeroMQ? Would you suggest RabbitMQ or Amazon SQS (we are in AWS setup) instead? Something else?

Thank you for your time

See more
Amazon SNS logo

Amazon SNS

1.4K
1.2K
18
Fully managed push messaging service
1.4K
1.2K
+ 1
18
PROS OF AMAZON SNS
  • 12
    Low cost
  • 6
    Supports multi subscribers
CONS OF AMAZON SNS
    Be the first to leave a con

    related Amazon SNS posts

    Cyril Duchon-Doris

    We decided to use AWS Lambda for several serverless tasks such as

    • Managing AWS backups
    • Processing emails received on Amazon SES and stored to Amazon S3 and notified via Amazon SNS, so as to push a message on our Redis so our Sidekiq Rails workers can process inbound emails
    • Pushing some relevant Amazon CloudWatch metrics and alarms to Slack
    See more
    Ravi Sathanapalli
    Director Product Management at Centime · | 7 upvotes · 125.6K views
    Shared insights
    on
    TwilioTwilioAmazon SNSAmazon SNS

    Hi, We are looking to implement 2FA - so that users would be sent a Verification code over their Email and SMS to their phone.

    We faced some limitations with Amazon SNS where we could either send the verification code to email OR to the phone number, while we want to send it to both.

    We also are looking to make the 2FA more flexible by adding any other options later on.

    What are the best alternatives to SNS for this use case and purpose? Looked at Twilio but want to explore other options before making a decision.

    Would be great to know what the experience with Twilio has been, especially the limitations/issues with Twilio...

    Appreciate any input from users of Twilio and others who have had similar use cases.

    See more
    Redis logo

    Redis

    59.4K
    45.6K
    3.9K
    Open source (BSD licensed), in-memory data structure store
    59.4K
    45.6K
    + 1
    3.9K
    PROS OF REDIS
    • 886
      Performance
    • 542
      Super fast
    • 513
      Ease of use
    • 444
      In-memory cache
    • 324
      Advanced key-value cache
    • 194
      Open source
    • 182
      Easy to deploy
    • 164
      Stable
    • 155
      Free
    • 121
      Fast
    • 42
      High-Performance
    • 40
      High Availability
    • 35
      Data Structures
    • 32
      Very Scalable
    • 24
      Replication
    • 22
      Great community
    • 22
      Pub/Sub
    • 19
      "NoSQL" key-value data store
    • 16
      Hashes
    • 13
      Sets
    • 11
      Sorted Sets
    • 10
      NoSQL
    • 10
      Lists
    • 9
      Async replication
    • 9
      BSD licensed
    • 8
      Bitmaps
    • 8
      Integrates super easy with Sidekiq for Rails background
    • 7
      Keys with a limited time-to-live
    • 7
      Open Source
    • 6
      Lua scripting
    • 6
      Strings
    • 5
      Awesomeness for Free
    • 5
      Hyperloglogs
    • 4
      Transactions
    • 4
      Outstanding performance
    • 4
      Runs server side LUA
    • 4
      LRU eviction of keys
    • 4
      Feature Rich
    • 4
      Written in ANSI C
    • 4
      Networked
    • 3
      Data structure server
    • 3
      Performance & ease of use
    • 2
      Dont save data if no subscribers are found
    • 2
      Automatic failover
    • 2
      Easy to use
    • 2
      Temporarily kept on disk
    • 2
      Scalable
    • 2
      Existing Laravel Integration
    • 2
      Channels concept
    • 2
      Object [key/value] size each 500 MB
    • 2
      Simple
    CONS OF REDIS
    • 15
      Cannot query objects directly
    • 3
      No secondary indexes for non-numeric data types
    • 1
      No WAL

    related Redis posts

    Russel Werner
    Lead Engineer at StackShare · | 32 upvotes · 2.8M views

    StackShare Feed is built entirely with React, Glamorous, and Apollo. One of our objectives with the public launch of the Feed was to enable a Server-side rendered (SSR) experience for our organic search traffic. When you visit the StackShare Feed, and you aren't logged in, you are delivered the Trending feed experience. We use an in-house Node.js rendering microservice to generate this HTML. This microservice needs to run and serve requests independent of our Rails web app. Up until recently, we had a mono-repo with our Rails and React code living happily together and all served from the same web process. In order to deploy our SSR app into a Heroku environment, we needed to split out our front-end application into a separate repo in GitHub. The driving factor in this decision was mostly due to limitations imposed by Heroku specifically with how processes can't communicate with each other. A new SSR app was created in Heroku and linked directly to the frontend repo so it stays in-sync with changes.

    Related to this, we need a way to "deploy" our frontend changes to various server environments without building & releasing the entire Ruby application. We built a hybrid Amazon S3 Amazon CloudFront solution to host our Webpack bundles. A new CircleCI script builds the bundles and uploads them to S3. The final step in our rollout is to update some keys in Redis so our Rails app knows which bundles to serve. The result of these efforts were significant. Our frontend team now moves independently of our backend team, our build & release process takes only a few minutes, we are now using an edge CDN to serve JS assets, and we have pre-rendered React pages!

    #StackDecisionsLaunch #SSR #Microservices #FrontEndRepoSplit

    See more
    Simon Reymann
    Senior Fullstack Developer at QUANTUSflow Software GmbH · | 30 upvotes · 11.1M views

    Our whole DevOps stack consists of the following tools:

    • GitHub (incl. GitHub Pages/Markdown for Documentation, GettingStarted and HowTo's) for collaborative review and code management tool
    • Respectively Git as revision control system
    • SourceTree as Git GUI
    • Visual Studio Code as IDE
    • CircleCI for continuous integration (automatize development process)
    • Prettier / TSLint / ESLint as code linter
    • SonarQube as quality gate
    • Docker as container management (incl. Docker Compose for multi-container application management)
    • VirtualBox for operating system simulation tests
    • Kubernetes as cluster management for docker containers
    • Heroku for deploying in test environments
    • nginx as web server (preferably used as facade server in production environment)
    • SSLMate (using OpenSSL) for certificate management
    • Amazon EC2 (incl. Amazon S3) for deploying in stage (production-like) and production environments
    • PostgreSQL as preferred database system
    • Redis as preferred in-memory database/store (great for caching)

    The main reason we have chosen Kubernetes over Docker Swarm is related to the following artifacts:

    • Key features: Easy and flexible installation, Clear dashboard, Great scaling operations, Monitoring is an integral part, Great load balancing concepts, Monitors the condition and ensures compensation in the event of failure.
    • Applications: An application can be deployed using a combination of pods, deployments, and services (or micro-services).
    • Functionality: Kubernetes as a complex installation and setup process, but it not as limited as Docker Swarm.
    • Monitoring: It supports multiple versions of logging and monitoring when the services are deployed within the cluster (Elasticsearch/Kibana (ELK), Heapster/Grafana, Sysdig cloud integration).
    • Scalability: All-in-one framework for distributed systems.
    • Other Benefits: Kubernetes is backed by the Cloud Native Computing Foundation (CNCF), huge community among container orchestration tools, it is an open source and modular tool that works with any OS.
    See more
    Gearman logo

    Gearman

    76
    144
    45
    A generic application framework to farm out work to other machines or processes
    76
    144
    + 1
    45
    PROS OF GEARMAN
    • 11
      Ease of use and very simple APIs
    • 11
      Free
    • 6
      Polyglot
    • 5
      No single point of failure
    • 3
      Scalable
    • 3
      High-throughput
    • 2
      Foreground & background processing
    • 2
      Very fast
    • 1
      Different Programming Languages Channel
    • 1
      Many supported programming languages
    CONS OF GEARMAN
      Be the first to leave a con

      related Gearman posts

      Beanstalkd logo

      Beanstalkd

      111
      161
      74
      A simple, fast work queue
      111
      161
      + 1
      74
      PROS OF BEANSTALKD
      • 23
        Fast
      • 12
        Free
      • 12
        Does one thing well
      • 9
        Scalability
      • 8
        Simplicity
      • 3
        External admin UI developer friendly
      • 3
        Job delay
      • 2
        Job prioritization
      • 2
        External admin UI
      CONS OF BEANSTALKD
        Be the first to leave a con

        related Beanstalkd posts

        Frédéric MARAND
        Core Developer at OSInet · | 2 upvotes · 234.2K views

        I used Kafka originally because it was mandated as part of the top-level IT requirements at a Fortune 500 client. What I found was that it was orders of magnitude more complex ...and powerful than my daily Beanstalkd , and far more flexible, resilient, and manageable than RabbitMQ.

        So for any case where utmost flexibility and resilience are part of the deal, I would use Kafka again. But due to the complexities involved, for any time where this level of scalability is not required, I would probably just use Beanstalkd for its simplicity.

        I tend to find RabbitMQ to be in an uncomfortable middle place between these two extremities.

        See more
        gRPC logo

        gRPC

        2.2K
        1.4K
        63
        A high performance, open-source universal RPC framework
        2.2K
        1.4K
        + 1
        63
        PROS OF GRPC
        • 24
          Higth performance
        • 15
          The future of API
        • 13
          Easy setup
        • 5
          Contract-based
        • 4
          Polyglot
        • 2
          Garbage
        CONS OF GRPC
          Be the first to leave a con

          related gRPC posts

          Noah Zoschke
          Engineering Manager at Segment · | 30 upvotes · 304.8K views

          We just launched the Segment Config API (try it out for yourself here) — a set of public REST APIs that enable you to manage your Segment configuration. Behind the scenes the Config API is built with Go , GRPC and Envoy.

          At Segment, we build new services in Go by default. The language is simple so new team members quickly ramp up on a codebase. The tool chain is fast so developers get immediate feedback when they break code, tests or integrations with other systems. The runtime is fast so it performs great at scale.

          For the newest round of APIs we adopted the GRPC service #framework.

          The Protocol Buffer service definition language makes it easy to design type-safe and consistent APIs, thanks to ecosystem tools like the Google API Design Guide for API standards, uber/prototool for formatting and linting .protos and lyft/protoc-gen-validate for defining field validations, and grpc-gateway for defining REST mapping.

          With a well designed .proto, its easy to generate a Go server interface and a TypeScript client, providing type-safe RPC between languages.

          For the API gateway and RPC we adopted the Envoy service proxy.

          The internet-facing segmentapis.com endpoint is an Envoy front proxy that rate-limits and authenticates every request. It then transcodes a #REST / #JSON request to an upstream GRPC request. The upstream GRPC servers are running an Envoy sidecar configured for Datadog stats.

          The result is API #security , #reliability and consistent #observability through Envoy configuration, not code.

          We experimented with Swagger service definitions, but the spec is sprawling and the generated clients and server stubs leave a lot to be desired. GRPC and .proto and the Go implementation feels better designed and implemented. Thanks to the GRPC tooling and ecosystem you can generate Swagger from .protos, but it’s effectively impossible to go the other way.

          See more
          Dylan Krupp
          Shared insights
          on
          gRPCgRPCGraphQLGraphQL

          I used GraphQL extensively at a previous employer a few years ago and really appreciated the data-driven schema etc alongside the many other benefits it provided. At that time, it seemed like it was set to replace RESTful APIs and many companies were adopting it.

          However, as of late, it seems like interest has been waning for GraphQL as opposed to increasing as I had assumed it would. Am I missing something here? What is the current perspective regarding this technology?

          Currently, I'm working with gRPC and was curious as to the state of everything now.

          See more