Need advice about which tool to choose?Ask the StackShare community!
Amazon SQS vs Apache Pulsar: What are the differences?
Introduction
This Markdown document provides a comparison between Amazon Simple Queue Service (SQS) and Apache Pulsar, highlighting the key differences between the two messaging services.
Scalability: Amazon SQS is highly scalable and can handle a large number of messages per second, making it suitable for applications with high throughputs. On the other hand, Apache Pulsar offers topic-level scalability, allowing individual topics to handle high message rates independent of other topics in the system.
Latency: Amazon SQS generally has higher latency compared to Apache Pulsar. SQS is primarily designed for high durability and availability, sacrificing real-time responsiveness. Pulsar, on the other hand, offers low latency messaging with its distributed architecture and efficient message publication and consumption mechanisms.
Message Ordering: Amazon SQS offers ordered message processing within a single queue, ensuring that messages are delivered and processed in the order in which they were sent. In Apache Pulsar, message ordering is maintained within a single partition or topic, but not across multiple partitions or topics by default.
Event Time: Exactly Once Semantics: In Amazon SQS, the exactly-once processing of messages is not provided out of the box. It offers at-least-once message delivery, and deduplication mechanisms need to be implemented by the application developer. Apache Pulsar, on the other hand, provides built-in support for exactly-once semantics using event time in its log-based architecture.
Multi-tenancy: Amazon SQS is designed to be a fully isolated service with per-queue resource allocation, offering dedicated resources for each queue. In contrast, Apache Pulsar supports multi-tenancy by allowing multiple topics or namespaces to share the same cluster resources while maintaining data isolation.
Message Retention: Amazon SQS retains messages for a configurable period, but there is no built-in message retention mechanism beyond the configured retention period. Apache Pulsar provides configurable message retention policies at both the topic and cluster level, allowing message expiration and automatic cleanup based on time or size.
In Summary, Amazon SQS and Apache Pulsar have key differences in scalability, latency, message ordering, event time processing, multi-tenancy, and message retention. Each messaging service offers distinct features and trade-offs, allowing developers to choose the most suitable technology for their specific use cases.
I want to schedule a message. Amazon SQS provides a delay of 15 minutes, but I want it in some hours.
Example: Let's say a Message1 is consumed by a consumer A but somehow it failed inside the consumer. I would want to put it in a queue and retry after 4hrs. Can I do this in Amazon MQ? I have seen in some Amazon MQ videos saying scheduling messages can be done. But, I'm not sure how.
Mithiridi, I believe you are talking about two different things. 1. If you need to process messages with delays of more 15m or at specific times, it's not a good idea to use queues, independently of tool SQM, Rabbit or Amazon MQ. you should considerer another approach using a scheduled job. 2. For dead queues and policy retries RabbitMQ, for example, doesn't support your use case. https://medium.com/@kiennguyen88/rabbitmq-delay-retry-schedule-with-dead-letter-exchange-31fb25a440fc I'm not sure if that is possible SNS/SQS support, they have a maximum delay for delivery (maxDelayTarget) in seconds but it's not clear the number. You can check this out: https://docs.aws.amazon.com/sns/latest/dg/sns-message-delivery-retries.html
Pros of Amazon SQS
- Easy to use, reliable62
- Low cost40
- Simple28
- Doesn't need to maintain it14
- It is Serverless8
- Has a max message size (currently 256K)4
- Triggers Lambda3
- Easy to configure with Terraform3
- Delayed delivery upto 15 mins only3
- Delayed delivery upto 12 hours3
- JMS compliant1
- Support for retry and dead letter queue1
- D1
Pros of Apache Pulsar
- Simple7
- Scalable4
- High-throughput3
- Geo-replication2
- Multi-tenancy2
- Pulsar Functions1
- Secure1
- Stream SQL1
- Horizontally scaleable1
- Easy to deploy1
- Fast1
Sign up to add or upvote prosMake informed product decisions
Cons of Amazon SQS
- Has a max message size (currently 256K)2
- Proprietary2
- Difficult to configure2
- Has a maximum 15 minutes of delayed messages only1
Cons of Apache Pulsar
- Very few commercial vendors for support1
- LImited Language support(6)1
- No one and only one delivery1
- No guaranteed dliefvery1
- Not jms compliant1
- Only Supports Topics1