Gearman vs NSQ vs ZeroMQ

Hi, we are in a ZMQ set up in a push/pull pattern, and we currently start to have more traffic and cases that the service is unavailable or stuck. We want to:

• Not loose messages in services outages
• Safely restart service without losing messages (@{ZeroMQ}|tool:1064| seems to need to close the socket in the receiver before restart manually)

Do you have experience with this setup with ZeroMQ? Would you suggest RabbitMQ or Amazon SQS (we are in AWS setup) instead? Something else?

Thank you for your time

This depends on your needs, but basically Kafka is the de-facto solution to go for. RabbitMQ, ZeroMQ or similar message queuing systems have their advantages too. Check for parallel consuming, in-flight queue (topic for Kafka) creation needs, consumer <-> message relations (how many consumers are interested in a message, all consumers are interested in all messages) etc...

I am looking into IoT World Solution where we have MQTT Broker. This MQTT Broker Sits in one of the Data Center. We are doing a lot of Alert and Alarm related processing on that Data, Currently, we are looking into Solution which can do distributed persistence of log/alert primarily on remote Disk.

Our primary need is to use lightweight where operational complexity and maintenance costs can be significantly reduced. We want to do it on-premise so we are not considering cloud solutions.

We looked into the following alternatives:

Apache Kafka - Great choice but operation and maintenance wise very complex. Rabbit MQ - High availability is the issue, Apache Pulsar - Operational Complexity. NATS - Absence of persistence. Akka Streams - Big learning curve and operational streams.

So we are looking into a lightweight library that can do distributed persistence preferably with publisher and subscriber model. Preferable on JVM stack.