delayed_job logo

delayed_job

Database backed asynchronous priority queue -- Extracted from Shopify
39
36
+ 1
6

What is delayed_job?

Delayed_job (or DJ) encapsulates the common pattern of asynchronously executing longer tasks in the background. It is a direct extraction from Shopify where the job table is responsible for a multitude of core tasks.
delayed_job is a tool in the Background Processing category of a tech stack.
delayed_job is an open source tool with 4.5K GitHub stars and 919 GitHub forks. Here’s a link to delayed_job's open source repository on GitHub

Who uses delayed_job?

Companies
12 companies reportedly use delayed_job in their tech stacks, including WeLend.hk, Shore GmbH, and Ubiqua.

Developers
27 developers on StackShare have stated that they use delayed_job.

Why developers like delayed_job?

Here’s a list of reasons why companies and developers use delayed_job
delayed_job Reviews

Here are some stack decisions, common use cases and reviews by companies and developers who chose delayed_job in their tech stack.

John Barton
John Barton
Founder at Hecate · | 6 upvotes · 11.1K views
atHecateHecate
Docker Compose
Docker Compose
Rails
Rails
Create React App
Create React App
ngrok
ngrok
PostgreSQL
PostgreSQL
delayed_job
delayed_job

Docker Compose might have been a bit of overkill for a dev environment as a solo founder, but I'd found so much with past side projects (though this is no longer a side project) that I would frequently waste time every time I came back to work on the project getting my dev env sorted again.

Made the conscious choice to make a "prod-ish" docker-compose config up front to make sure that didn't bite me again.

Structured it so I have the following containers running

  • server - the Rails app in API style
  • client - the Create React App
  • ngrok - ngrok to receive webhooks in dev
  • db - PostgreSQL
  • queues - delayed_job worker
See more
Jerome Dalbert
Jerome Dalbert
Senior Backend Engineer at StackShare · | 4 upvotes · 28.9K views
atGratify CommerceGratify Commerce
delayed_job
delayed_job
Rails
Rails
AWS Elastic Beanstalk
AWS Elastic Beanstalk
Sidekiq
Sidekiq
Ruby
Ruby
Amazon SQS
Amazon SQS
#BackgroundProcessing

delayed_job is a great Rails background job library for new projects, as it only uses what you already have: a relational database. We happily used it during the company’s first two years.

But it started to falter as our web and database transactions significantly grew. Our app interacted with users via SMS texts sent inside background jobs. Because the delayed_job daemon ran every couple seconds, this meant that users often waited several long seconds before getting text replies, which was not acceptable. Moreover, job processing was done inside AWS Elastic Beanstalk web instances, which were already under stress and not meant to handle jobs.

We needed a fast background job system that could process jobs in near real-time and integrate well with AWS. Sidekiq is a fast and popular Ruby background job library, but it does not leverage the Elastic Beanstalk worker architecture, and you have to maintain a Redis instance.

We ended up choosing active-elastic-job, which seamlessly integrates with worker instances and Amazon SQS. SQS is a fast queue and you don’t need to worry about infrastructure or scaling, as AWS handles it for you.

We noticed significant performance gains immediately after making the switch.

#BackgroundProcessing

See more
Jerome Dalbert
Jerome Dalbert
Senior Backend Engineer at StackShare · | 3 upvotes · 12.7K views
atStackShareStackShare
Sidekiq
Sidekiq
Ruby
Ruby
delayed_job
delayed_job
Redis
Redis

We use Sidekiq to process millions of Ruby background jobs a day under normal loads. We sometimes process more than that when running one-off backfill tasks.

With so many jobs, it wouldn't really make sense to use delayed_job, as it would put our main database under unnecessary load, which would make it a bottleneck with most DB queries serving jobs and not end users. I suppose you could create a separate DB just for jobs, but that can be a hassle. Sidekiq uses a separate Redis instance so you don't have this problem. And it is very performant!

I also like that its free version comes "batteries included" with:

  • A web monitoring UI that provides some nice stats.
  • An API that can come in handy for one-off tasks, like changing the queue of certain already enqueued jobs.

Sidekiq is a pleasure to use. All our engineers love it!

See more

delayed_job Alternatives & Comparisons

What are some alternatives to delayed_job?
Sidekiq
Sidekiq uses threads to handle many jobs at the same time in the same process. It does not require Rails but will integrate tightly with Rails 3/4 to make background processing dead simple.
Resque
Background jobs can be any Ruby class or module that responds to perform. Your existing classes can easily be converted to background jobs or you can create new classes specifically to do work. Or, you can do both.
Beanstalkd
Beanstalks's interface is generic, but was originally designed for reducing the latency of page views in high-volume web applications by running time-consuming tasks asynchronously.
PHP-FPM
It is an alternative PHP FastCGI implementation with some additional features useful for sites of any size, especially busier sites. It includes Adaptive process spawning, Advanced process management with graceful stop/start, Emergency restart in case of accidental opcode cache destruction etc.
Hangfire
It is an open-source framework that helps you to create, process and manage your background jobs, i.e. operations you don't want to put in your request processing pipeline. It supports all kind of background tasks – short-running and long-running, CPU intensive and I/O intensive, one shot and recurrent.
See all alternatives

delayed_job's Followers
36 developers follow delayed_job to keep up with related blogs and decisions.
Yoshinori Kawasaki
Beth Skurrie
Sergio Espeja
Mohamma76685757
Gary Damm
Peter Berkenbosch
Stanislas Boyet
Mike Mayer
puneet18190
i-arindam