What is Sidekiq?

Sidekiq uses threads to handle many jobs at the same time in the same process. It does not require Rails but will integrate tightly with Rails 3/4 to make background processing dead simple.
Sidekiq is a tool in the Background Processing category of a tech stack.
Sidekiq is an open source tool with 9.7K GitHub stars and 1.7K GitHub forks. Here’s a link to Sidekiq's open source repository on GitHub

Who uses Sidekiq?

Companies
344 companies use Sidekiq in their tech stacks, including StackShare, Product Hunt, and SendGrid.

Developers
76 developers use Sidekiq.

Sidekiq Integrations

Why developers like Sidekiq?

Here’s a list of reasons why companies and developers use Sidekiq
Sidekiq Reviews

Here are some stack decisions, common use cases and reviews by companies and developers who chose Sidekiq in their tech stack.

Cyril Duchon-Doris
Cyril Duchon-Doris
CTO at My Job Glasses · | 9 upvotes · 5.5K views
atMy Job Glasses
Slack
Amazon CloudWatch
Rails
Sidekiq
Redis
Amazon SNS
Amazon S3
Amazon SES
AWS Lambda

We decided to use AWS Lambda for several serverless tasks such as

  • Managing AWS backups
  • Processing emails received on Amazon SES and stored to Amazon S3 and notified via Amazon SNS, so as to push a message on our Redis so our Sidekiq Rails workers can process inbound emails
  • Pushing some relevant Amazon CloudWatch metrics and alarms to Slack
See more
Yarn
Redux
React
jQuery
vuex
Vue.js
MongoDB
Redis
PostgreSQL
Sidekiq
Rails
#Font-awesome
#Bulma.io

I'm building a new process management tool. I decided to build with Rails as my backend, using Sidekiq for background jobs. I chose to work with these tools because I've worked with them before and know that they're able to get the job done. They may not be the sexiest tools, but they work and are reliable, which is what I was optimizing for. For data stores, I opted for PostgreSQL and Redis. Because I'm planning on offering dashboards, I wanted a SQL database instead of something like MongoDB that might work early on, but be difficult to use as soon as I want to facilitate aggregate queries.

On the front-end I'm using Vue.js and vuex in combination with #Turbolinks. In effect, I want to render most pages on the server side without key interactions being managed by Vue.js . This is the first project I'm working on where I've explicitly decided not to include jQuery . I have found React and Redux.js more confusing to setup. I appreciate the opinionated approach from the Vue.js community and that things just work together the way that I'd expect. To manage my javascript dependencies, I'm using Yarn .

For CSS frameworks, I'm using #Bulma.io. I really appreciate it's minimal nature and that there are no hard javascript dependencies. And to add a little spice, I'm using #font-awesome.

See more
Simon Bettison
Simon Bettison
Managing Director at Bettison.org Limited · | 6 upvotes · 27.4K views
atBettison.org Limited
Amazon EC2 Container Service
Docker
Amazon VPC
Amazon Route 53
Amazon SQS
Amazon SES
Amazon CloudFront
nginx
Unicorn
Ruby
Travis CI
Selenium
RSpec
Rails
Amazon ElastiCache
Redis
Sidekiq
Elasticsearch
PostgreSQL

In 2010 we made the very difficult decision to entirely re-engineer our existing monolithic LAMP application from the ground up in order to address some growing concerns about it's long term viability as a platform.

Full application re-write is almost always never the answer, because of the risks involved. However the situation warranted drastic action as it was clear that the existing product was going to face severe scaling issues. We felt it better address these sooner rather than later and also take the opportunity to improve the international architecture and also to refactor the database in. order that it better matched the changes in core functionality.

PostgreSQL was chosen for its reputation as being solid ACID compliant database backend, it was available as an offering AWS RDS service which reduced the management overhead of us having to configure it ourselves. In order to reduce read load on the primary database we implemented an Elasticsearch layer for fast and scalable search operations. Synchronisation of these indexes was to be achieved through the use of Sidekiq's Redis based background workers on Amazon ElastiCache. Again the AWS solution here looked to be an easy way to keep our involvement in managing this part of the platform at a minimum. Allowing us to focus on our core business.

Rails ls was chosen for its ability to quickly get core functionality up and running, its MVC architecture and also its focus on Test Driven Development using RSpec and Selenium with Travis CI providing continual integration. We also liked Ruby for its terse, clean and elegant syntax. Though YMMV on that one!

Unicorn was chosen for its continual deployment and reputation as a reliable application server, nginx for its reputation as a fast and stable reverse-proxy. We also took advantage of the Amazon CloudFront CDN here to further improve performance by caching static assets globally.

We tried to strike a balance between having control over management and configuration of our core application with the convenience of being able to leverage AWS hosted services for ancillary functions (Amazon SES , Amazon SQS Amazon Route 53 all hosted securely inside Amazon VPC of course!).

Whilst there is some compromise here with potential vendor lock in, the tasks being performed by these ancillary services are no particularly specialised which should mitigate this risk. Furthermore we have already containerised the stack in our development using Docker environment, and looking to how best to bring this into production - potentially using Amazon EC2 Container Service

See more
Cyril Duchon-Doris
Cyril Duchon-Doris
CTO at My Job Glasses · | 5 upvotes · 3.3K views
atMy Job Glasses
Rails
Redis
Amazon SQS
Sidekiq

We migrated from Amazon SQS + Shoryuken to Sidekiq in order to have at-most-once delivery out of the box and more flexibility.

The UI builtin Rails makes it smoother for development and QA. Through the sidekiq rails engine we can easily see & understand which job is/was/will be executed, and even get some stats for free. Compared to SQS, we lose in scalability (need to manage the underlying Redis instance) but this is not so critical right now for our business size and the PROs clearly outweigh the CONs. Plugins allow to easily add distributed CRON scheduled jobs in there for almost free, and this is a core feature for us, so we no longer need to maintain a "scheduler" instance and we make our CRON jobs more resilient. The Sidekiq UI can easily be tweaked and for instance we have added a column that translates the CRON syntax into a human readable string, so it's easy for our Q/A to check whether the job is scheduled appropriately.

We still use Amazon SQS for some other apps, but no longer for our main Rails app.

See more
Cyril Duchon-Doris
Cyril Duchon-Doris
CTO at My Job Glasses · | 4 upvotes · 159.5K views
atMy Job Glasses
Sidekiq
Rollbar
redux-saga
Redux
React
Rails

After splitting our monolith into a Rails API + a React Redux.js frontend app, it became a necessity to monitor frontend errors. Our frontend application is not your typical website, and features a lot of interesting SPA mechanics that need to be followed closely (many async flows, redux-saga , etc.) in addition to regular browser incompatibility issues. Rollbar kicks in so that we can monitor every bug that happens on our frontend, and aggregate this with almost 0 work. The number of occurrences and affected browsers on each occurence helps us understand the priority and severity of bugs even when our users don't tell us about them, so we can decide whether we need to fix this bug that was encountered by 1k users in less than a few days days VERSUS telling this SINGLE user to switch browsers because he's using a very outdated version that no one else uses. Now we also use Rollbar with Rails, Sidekiq and even AWS Lambda errors since the interface is quite convenient.

See more
Jerome Dalbert
Jerome Dalbert
Senior Backend Engineer at StackShare · | 4 upvotes · 11.8K views
atGratify Commerce
Amazon SQS
Ruby
Sidekiq
AWS Elastic Beanstalk
Rails
delayed_job
#BackgroundProcessing

delayed_job is a great Rails background job library for new projects, as it only uses what you already have: a relational database. We happily used it during the company’s first two years.

But it started to falter as our web and database transactions significantly grew. Our app interacted with users via SMS texts sent inside background jobs. Because the delayed_job daemon ran every couple seconds, this meant that users often waited several long seconds before getting text replies, which was not acceptable. Moreover, job processing was done inside AWS Elastic Beanstalk web instances, which were already under stress and not meant to handle jobs.

We needed a fast background job system that could process jobs in near real-time and integrate well with AWS. Sidekiq is a fast and popular Ruby background job library, but it does not leverage the Elastic Beanstalk worker architecture, and you have to maintain a Redis instance.

We ended up choosing active-elastic-job, which seamlessly integrates with worker instances and Amazon SQS. SQS is a fast queue and you don’t need to worry about infrastructure or scaling, as AWS handles it for you.

We noticed significant performance gains immediately after making the switch.

#BackgroundProcessing

See more

Sidekiq Alternatives & Comparisons

What are some alternatives to Sidekiq?
Resque
Background jobs can be any Ruby class or module that responds to perform. Your existing classes can easily be converted to background jobs or you can create new classes specifically to do work. Or, you can do both.
Celery
Celery is an asynchronous task queue/job queue based on distributed message passing. It is focused on real-time operation, but supports scheduling as well.
RabbitMQ
RabbitMQ gives your applications a common platform to send and receive messages, and your messages a safe place to live until received.
delayed_job
Delayed_job (or DJ) encapsulates the common pattern of asynchronously executing longer tasks in the background. It is a direct extraction from Shopify where the job table is responsible for a multitude of core tasks.
Kafka
Kafka is a distributed, partitioned, replicated commit log service. It provides the functionality of a messaging system, but with a unique design.
See all alternatives

Sidekiq's Stats