RabbitMQ vs. Kafka vs. NSQ

Implements the TaskQueue API.

The poster child for scalable messaging systems, RabbitMQ has been used in countless large scale systems as the messaging backbone of any large cluster, and has proven itself time and again in many production settings.

MQ

RabbitMQ is an all purpose queuing service for our stack. We use it for user facing jobs as well as keeping track of behind the scenes jobs.

celery broker 로써 erlang 기반의 rabbitmq 를 클러스터링 구성하여 적용 하였습니다.

Rabbit acts as our coordinator for all actions that happen during game time. All worker containers connect to rabbit in order to receive game events and emit their own events when applicable.

RabbitMQ is used by Celery as a task queue.

Services inner communication and logging

Work queue and delayed messages are still managed in RabbitMQ, phasing out

RabbitMQ is the enterprise message bus for our platform, providing infrastructure for managing our ETL queues, real-time event notifications for applications, and audit logging.

Our primary message queuing system.

RabbitMQ 学习总结

To support event-driven architecture.

All communication between our services is asynchronous and goes through a message queue.

AMQP 0-9-1 with topic exchanges for fast & reliable messaging. Built on the rock-solid Erlang OTP.

We used RabbitMQ in conjunction with Celery (Celery-Beat package) to run periodic tasks on the server.

Communication between microservices.

Message bus for application messages (commands, events) and logs. Creates transparency into pending units of work. Stored pending work in case of application failures.

default message queue replacing generic crons

We use RabbitMQ quite heavily across both legacy and current products for everything from work queues to communications products and messaging.

All changes to our resources are published to RabbitMQ, and is the core of our evented architecture.

Queue for agents to pick up jobs from, and provide logs back to front-end UI consumers and the log server.

Decoupling various backend services

Messaging Bus

Used as central Message Broker; off-loading tasks to be executed asynchronous, used as communication tool between different microservices, used as tool to handle peaks in incoming data, etc.

wonderful messaging service. used it at a couple companies.

Experience with RabbitMQ in the context of handling asynchronous api events + inter-server communication.

Used at cloudfind as one of the job queue technologies.

Task queue

Use like a service bus for microservices

IoT communication, topic based message system

消息队列

Set-and-forget backend for NServicebus. Easy to deploy and develop for.

Data pipeline

We attentively enroll in classes and seminars to keep on our education. Every year we attain far more hours of knowledge than required to sustain our licenses.

Asynchronous communication from Backend to Worker processes

Reliable inter-service messaging with failover and error routing. Used heavily in backend infrastructure for long-lived tasks.

Front-end messages are logged to Kafka by our API and application servers. We have batch processing (on the middle-left) and real-time processing (on the middle-right) pipelines to process the experiment data. For batch processing, after daily raw log get to s3, we start our nightly experiment workflow to figure out experiment users groups and experiment metrics. We use our in-house workflow management system Pinball to manage the dependencies of all these MapReduce jobs.

Durable, high throughput data streaming

as a log aggregator

We are using Kafka as a message queue to process our widget logs.

Used as a integration middleware by messaging interchanging.

realtime distributed broker

Distributed system messaging.

消息队列

消息队列，主要用于处理各个微服务之间的通信以及异步处理相关逻辑

Event bus

The built-in Gamification that comes with our Playbooks application uses NSQ for work queues and microservice communication.