Get Advice Icon

Need advice about which tool to choose?Ask the StackShare community!

Amazon SQS
Amazon SQS

1.4K
981
+ 1
154
Sidekiq
Sidekiq

894
443
+ 1
405
Add tool

Amazon SQS vs Sidekiq: What are the differences?

Amazon SQS: Fully managed message queuing service. Transmit any volume of data, at any level of throughput, without losing messages or requiring other services to be always available. With SQS, you can offload the administrative burden of operating and scaling a highly available messaging cluster, while paying a low price for only what you use; Sidekiq: Simple, efficient background processing for Ruby. Sidekiq uses threads to handle many jobs at the same time in the same process. It does not require Rails but will integrate tightly with Rails 3/4 to make background processing dead simple.

Amazon SQS belongs to "Message Queue" category of the tech stack, while Sidekiq can be primarily classified under "Background Processing".

"Easy to use, reliable" is the top reason why over 45 developers like Amazon SQS, while over 120 developers mention "Simple" as the leading cause for choosing Sidekiq.

Sidekiq is an open source tool with 9.68K GitHub stars and 1.66K GitHub forks. Here's a link to Sidekiq's open source repository on GitHub.

According to the StackShare community, Amazon SQS has a broader approval, being mentioned in 384 company stacks & 103 developers stacks; compared to Sidekiq, which is listed in 348 company stacks and 77 developer stacks.

- No public GitHub repository available -

What is Amazon SQS?

Transmit any volume of data, at any level of throughput, without losing messages or requiring other services to be always available. With SQS, you can offload the administrative burden of operating and scaling a highly available messaging cluster, while paying a low price for only what you use.

What is Sidekiq?

Sidekiq uses threads to handle many jobs at the same time in the same process. It does not require Rails but will integrate tightly with Rails 3/4 to make background processing dead simple.
Get Advice Icon

Need advice about which tool to choose?Ask the StackShare community!

Why do developers choose Amazon SQS?
Why do developers choose Sidekiq?

Sign up to add, upvote and see more prosMake informed product decisions

    Be the first to leave a con
    What companies use Amazon SQS?
    What companies use Sidekiq?

    Sign up to get full access to all the companiesMake informed product decisions

    What tools integrate with Amazon SQS?
    What tools integrate with Sidekiq?

    Sign up to get full access to all the tool integrationsMake informed product decisions

    What are some alternatives to Amazon SQS and Sidekiq?
    Amazon MQ
    Amazon MQ is a managed message broker service for Apache ActiveMQ that makes it easy to set up and operate message brokers in the cloud.
    Kafka
    Kafka is a distributed, partitioned, replicated commit log service. It provides the functionality of a messaging system, but with a unique design.
    Redis
    Redis is an open source, BSD licensed, advanced key-value store. It is often referred to as a data structure server since keys can contain strings, hashes, lists, sets and sorted sets.
    ActiveMQ
    Apache ActiveMQ is fast, supports many Cross Language Clients and Protocols, comes with easy to use Enterprise Integration Patterns and many advanced features while fully supporting JMS 1.1 and J2EE 1.4. Apache ActiveMQ is released under the Apache 2.0 License.
    Amazon SNS
    Amazon Simple Notification Service makes it simple and cost-effective to push to mobile devices such as iPhone, iPad, Android, Kindle Fire, and internet connected smart devices, as well as pushing to other distributed services. Besides pushing cloud notifications directly to mobile devices, SNS can also deliver notifications by SMS text message or email, to Simple Queue Service (SQS) queues, or to any HTTP endpoint.
    See all alternatives
    Decisions about Amazon SQS and Sidekiq
    Jerome Dalbert
    Jerome Dalbert
    Senior Backend Engineer at StackShare · | 4 upvotes · 57.7K views
    atGratify CommerceGratify Commerce
    delayed_job
    delayed_job
    Rails
    Rails
    AWS Elastic Beanstalk
    AWS Elastic Beanstalk
    Sidekiq
    Sidekiq
    Ruby
    Ruby
    Amazon SQS
    Amazon SQS
    #BackgroundProcessing

    delayed_job is a great Rails background job library for new projects, as it only uses what you already have: a relational database. We happily used it during the company’s first two years.

    But it started to falter as our web and database transactions significantly grew. Our app interacted with users via SMS texts sent inside background jobs. Because the delayed_job daemon ran every couple seconds, this meant that users often waited several long seconds before getting text replies, which was not acceptable. Moreover, job processing was done inside AWS Elastic Beanstalk web instances, which were already under stress and not meant to handle jobs.

    We needed a fast background job system that could process jobs in near real-time and integrate well with AWS. Sidekiq is a fast and popular Ruby background job library, but it does not leverage the Elastic Beanstalk worker architecture, and you have to maintain a Redis instance.

    We ended up choosing active-elastic-job, which seamlessly integrates with worker instances and Amazon SQS. SQS is a fast queue and you don’t need to worry about infrastructure or scaling, as AWS handles it for you.

    We noticed significant performance gains immediately after making the switch.

    #BackgroundProcessing

    See more
    Jerome Dalbert
    Jerome Dalbert
    Senior Backend Engineer at StackShare · | 3 upvotes · 37.1K views
    atStackShareStackShare
    Sidekiq
    Sidekiq
    Ruby
    Ruby
    delayed_job
    delayed_job
    Redis
    Redis

    We use Sidekiq to process millions of Ruby background jobs a day under normal loads. We sometimes process more than that when running one-off backfill tasks.

    With so many jobs, it wouldn't really make sense to use delayed_job, as it would put our main database under unnecessary load, which would make it a bottleneck with most DB queries serving jobs and not end users. I suppose you could create a separate DB just for jobs, but that can be a hassle. Sidekiq uses a separate Redis instance so you don't have this problem. And it is very performant!

    I also like that its free version comes "batteries included" with:

    • A web monitoring UI that provides some nice stats.
    • An API that can come in handy for one-off tasks, like changing the queue of certain already enqueued jobs.

    Sidekiq is a pleasure to use. All our engineers love it!

    See more
    Simon Bettison
    Simon Bettison
    Managing Director at Bettison.org Limited · | 7 upvotes · 212.9K views
    atBettison.org LimitedBettison.org Limited
    PostgreSQL
    PostgreSQL
    Elasticsearch
    Elasticsearch
    Sidekiq
    Sidekiq
    Redis
    Redis
    Amazon ElastiCache
    Amazon ElastiCache
    Rails
    Rails
    RSpec
    RSpec
    Selenium
    Selenium
    Travis CI
    Travis CI
    Ruby
    Ruby
    Unicorn
    Unicorn
    nginx
    nginx
    Amazon CloudFront
    Amazon CloudFront
    Amazon SES
    Amazon SES
    Amazon SQS
    Amazon SQS
    Amazon Route 53
    Amazon Route 53
    Amazon VPC
    Amazon VPC
    Docker
    Docker
    Amazon EC2 Container Service
    Amazon EC2 Container Service

    In 2010 we made the very difficult decision to entirely re-engineer our existing monolithic LAMP application from the ground up in order to address some growing concerns about it's long term viability as a platform.

    Full application re-write is almost always never the answer, because of the risks involved. However the situation warranted drastic action as it was clear that the existing product was going to face severe scaling issues. We felt it better address these sooner rather than later and also take the opportunity to improve the international architecture and also to refactor the database in. order that it better matched the changes in core functionality.

    PostgreSQL was chosen for its reputation as being solid ACID compliant database backend, it was available as an offering AWS RDS service which reduced the management overhead of us having to configure it ourselves. In order to reduce read load on the primary database we implemented an Elasticsearch layer for fast and scalable search operations. Synchronisation of these indexes was to be achieved through the use of Sidekiq's Redis based background workers on Amazon ElastiCache. Again the AWS solution here looked to be an easy way to keep our involvement in managing this part of the platform at a minimum. Allowing us to focus on our core business.

    Rails ls was chosen for its ability to quickly get core functionality up and running, its MVC architecture and also its focus on Test Driven Development using RSpec and Selenium with Travis CI providing continual integration. We also liked Ruby for its terse, clean and elegant syntax. Though YMMV on that one!

    Unicorn was chosen for its continual deployment and reputation as a reliable application server, nginx for its reputation as a fast and stable reverse-proxy. We also took advantage of the Amazon CloudFront CDN here to further improve performance by caching static assets globally.

    We tried to strike a balance between having control over management and configuration of our core application with the convenience of being able to leverage AWS hosted services for ancillary functions (Amazon SES , Amazon SQS Amazon Route 53 all hosted securely inside Amazon VPC of course!).

    Whilst there is some compromise here with potential vendor lock in, the tasks being performed by these ancillary services are no particularly specialised which should mitigate this risk. Furthermore we have already containerised the stack in our development using Docker environment, and looking to how best to bring this into production - potentially using Amazon EC2 Container Service

    See more
    Cyril Duchon-Doris
    Cyril Duchon-Doris
    CTO at My Job Glasses · | 5 upvotes · 45.1K views
    atMy Job GlassesMy Job Glasses
    Redis
    Redis
    Rails
    Rails
    Sidekiq
    Sidekiq
    Amazon SQS
    Amazon SQS

    We migrated from Amazon SQS + Shoryuken to Sidekiq in order to have at-most-once delivery out of the box and more flexibility.

    The UI builtin Rails makes it smoother for development and QA. Through the sidekiq rails engine we can easily see & understand which job is/was/will be executed, and even get some stats for free. Compared to SQS, we lose in scalability (need to manage the underlying Redis instance) but this is not so critical right now for our business size and the PROs clearly outweigh the CONs. Plugins allow to easily add distributed CRON scheduled jobs in there for almost free, and this is a core feature for us, so we no longer need to maintain a "scheduler" instance and we make our CRON jobs more resilient. The Sidekiq UI can easily be tweaked and for instance we have added a column that translates the CRON syntax into a human readable string, so it's easy for our Q/A to check whether the job is scheduled appropriately.

    We still use Amazon SQS for some other apps, but no longer for our main Rails app.

    See more
    Interest over time
    Reviews of Amazon SQS and Sidekiq
    Review ofSidekiqSidekiq

    Pretty good post. I found your website perfect for my needs bullet force

    How developers use Amazon SQS and Sidekiq
    Avatar of Karma
    Karma uses Amazon SQSAmazon SQS

    In the beginning we thought we wanted to start using something like RabbitMQ or maybe Kafka or maybe ActiveMQ. Back then we only had a few developers and no ops people. That has changed now, but we didn't really look forward to setting up a queuing cluster and making sure that all works.

    What we did instead was we looked at what services Amazon offers to see if we can use those to build our own messaging system within those services. That's basically what we did. We wrote some clients in Ruby that can basically do the entire orchestration for us, and we run all our messaging on both SNS and SQS. Basically what you can do in Amazon services is you can use Amazon Simple Notification Service, so SNS, for creating topics and you can use queues to subscribe to these topics. That's basically all you need for a messaging system. You don't have to worry about scalability at all. That's what really appealed to us.

    Avatar of Brandon Adams
    Brandon Adams uses Amazon SQSAmazon SQS

    This isn't exactly low-latency (10s to 100s of milliseconds), but it has good throughput and a simple API. There is good reliability, and there is no configuration necessary to get up and running. A hosted queue is important when trying to move fast.

    Avatar of SmartLogic
    SmartLogic uses SidekiqSidekiq

    We turn to Sidekiq when we need to run background jobs in a Rails app, which we do for just about every Rails app we write. We especially like the ops tools that come with Sidekiq, which make it easy to monitor and maintain.

    Avatar of Tim Lucas
    Tim Lucas uses SidekiqSidekiq

    Background processing of Pushover push notifications to admins when sales occur, payments processing via Pin Payments, Campaign Monitor transaction email sending, and Intercom event API posting.

    Avatar of Simple Merchant
    Simple Merchant uses Amazon SQSAmazon SQS

    SQS is the bridge between our new Lambda services and our incumbent Rails applications. Extremely easy to use when you're already using other AWS infrastructure.

    Avatar of Told
    Told uses SidekiqSidekiq

    Sidekiq is used extensively for a multitude of background jobs, everything from audio/video post-processing to sending push notifications.

    Avatar of Jeff Flynn
    Jeff Flynn uses SidekiqSidekiq

    We offload our background processing tasks (photo sizing, watermarking, etc.) to Sidekiq to keep our app's performance optimal.

    Avatar of Olo
    Olo uses Amazon SQSAmazon SQS

    Primary message queue. Enqueueing operations revert to a local file-system-based queue when SQS is unavailable.

    Avatar of IndiTip
    IndiTip uses Amazon SQSAmazon SQS

    I can't afford to lose data if Dynamo throttles my writes, so everything goes into a message queue first.

    How much does Amazon SQS cost?
    How much does Sidekiq cost?
    Pricing unavailable