delayed_job vs Resque: What are the differences?
delayed_job: Database backed asynchronous priority queue -- Extracted from Shopify. Delayed_job (or DJ) encapsulates the common pattern of asynchronously executing longer tasks in the background. It is a direct extraction from Shopify where the job table is responsible for a multitude of core tasks; Resque: A Redis-backed Ruby library for creating background jobs, placing them on multiple queues, and processing them later. Background jobs can be any Ruby class or module that responds to perform. Your existing classes can easily be converted to background jobs or you can create new classes specifically to do work. Or, you can do both.
delayed_job and Resque can be primarily classified as "Background Processing" tools.
"Easy to get started" is the primary reason why developers consider delayed_job over the competitors, whereas "Free" was stated as the key factor in picking Resque.
delayedjob and Resque are both open source tools. It seems that Resque with 8.53K GitHub stars and 1.58K forks on GitHub has more adoption than delayedjob with 4.45K GitHub stars and 915 GitHub forks.
MAK IT, Stitched, and Youboox are some of the popular companies that use Resque, whereas delayedjob is used by Ubiqua, WeLab Limited, and feedforce Inc.. Resque has a broader approval, being mentioned in 34 company stacks & 8 developers stacks; compared to delayedjob, which is listed in 8 company stacks and 5 developer stacks.
What is delayed_job?
What is Resque?
Need advice about which tool to choose?Ask the StackShare community!
What are the cons of using delayed_job?
What are the cons of using Resque?
Sign up to get full access to all the companiesMake informed product decisions
What tools integrate with delayed_job?
delayed_job is a great Rails background job library for new projects, as it only uses what you already have: a relational database. We happily used it during the company’s first two years.
But it started to falter as our web and database transactions significantly grew. Our app interacted with users via SMS texts sent inside background jobs. Because the delayed_job daemon ran every couple seconds, this meant that users often waited several long seconds before getting text replies, which was not acceptable. Moreover, job processing was done inside AWS Elastic Beanstalk web instances, which were already under stress and not meant to handle jobs.
We needed a fast background job system that could process jobs in near real-time and integrate well with AWS. Sidekiq is a fast and popular Ruby background job library, but it does not leverage the Elastic Beanstalk worker architecture, and you have to maintain a Redis instance.
We ended up choosing active-elastic-job, which seamlessly integrates with worker instances and Amazon SQS. SQS is a fast queue and you don’t need to worry about infrastructure or scaling, as AWS handles it for you.
We noticed significant performance gains immediately after making the switch.
We use Sidekiq to process millions of Ruby background jobs a day under normal loads. We sometimes process more than that when running one-off backfill tasks.
With so many jobs, it wouldn't really make sense to use delayed_job, as it would put our main database under unnecessary load, which would make it a bottleneck with most DB queries serving jobs and not end users. I suppose you could create a separate DB just for jobs, but that can be a hassle. Sidekiq uses a separate Redis instance so you don't have this problem. And it is very performant!
I also like that its free version comes "batteries included" with:
- A web monitoring UI that provides some nice stats.
- An API that can come in handy for one-off tasks, like changing the queue of certain already enqueued jobs.
Sidekiq is a pleasure to use. All our engineers love it!