Alternatives to Resque logo

Alternatives to Resque

Sidekiq, delayed_job, Celery, Beanstalkd, and RabbitMQ are the most popular alternatives and competitors to Resque.
109
98
+ 1
9

What is Resque and what are its top alternatives?

Background jobs can be any Ruby class or module that responds to perform. Your existing classes can easily be converted to background jobs or you can create new classes specifically to do work. Or, you can do both.
Resque is a tool in the Background Processing category of a tech stack.
Resque is an open source tool with 8.9K GitHub stars and 1.6K GitHub forks. Here’s a link to Resque's open source repository on GitHub

Top Alternatives to Resque

  • Sidekiq

    Sidekiq

    Sidekiq uses threads to handle many jobs at the same time in the same process. It does not require Rails but will integrate tightly with Rails 3/4 to make background processing dead simple. ...

  • delayed_job

    delayed_job

    Delayed_job (or DJ) encapsulates the common pattern of asynchronously executing longer tasks in the background. It is a direct extraction from Shopify where the job table is responsible for a multitude of core tasks. ...

  • Celery

    Celery

    Celery is an asynchronous task queue/job queue based on distributed message passing. It is focused on real-time operation, but supports scheduling as well. ...

  • Beanstalkd

    Beanstalkd

    Beanstalks's interface is generic, but was originally designed for reducing the latency of page views in high-volume web applications by running time-consuming tasks asynchronously. ...

  • RabbitMQ

    RabbitMQ

    RabbitMQ gives your applications a common platform to send and receive messages, and your messages a safe place to live until received. ...

  • RAKE

    RAKE

    It is a software task management and build automation tool. It allows the user to specify tasks and describe dependencies as well as to group tasks in a namespace. ...

  • Hangfire

    Hangfire

    It is an open-source framework that helps you to create, process and manage your background jobs, i.e. operations you don't want to put in your request processing pipeline. It supports all kind of background tasks – short-running and long-running, CPU intensive and I/O intensive, one shot and recurrent. ...

  • PHP-FPM

    PHP-FPM

    It is an alternative PHP FastCGI implementation with some additional features useful for sites of any size, especially busier sites. It includes Adaptive process spawning, Advanced process management with graceful stop/start, Emergency restart in case of accidental opcode cache destruction etc. ...

Resque alternatives & related posts

Sidekiq logo

Sidekiq

1K
532
407
Simple, efficient background processing for Ruby
1K
532
+ 1
407
PROS OF SIDEKIQ
  • 123
    Simple
  • 99
    Efficient background processing
  • 60
    Scalability
  • 37
    Better then resque
  • 26
    Great documentation
  • 15
    Admin tool
  • 14
    Great community
  • 8
    Integrates with redis automatically, with zero config
  • 7
    Great support
  • 7
    Stupidly simple to integrate and run on Rails/Heroku
  • 3
    Freeium
  • 3
    Ruby
  • 2
    Pro version
  • 1
    Dashboard w/live polling
  • 1
    Great ecosystem of addons
  • 1
    Fast
CONS OF SIDEKIQ
    Be the first to leave a con

    related Sidekiq posts

    Cyril Duchon-Doris

    We decided to use AWS Lambda for several serverless tasks such as

    • Managing AWS backups
    • Processing emails received on Amazon SES and stored to Amazon S3 and notified via Amazon SNS, so as to push a message on our Redis so our Sidekiq Rails workers can process inbound emails
    • Pushing some relevant Amazon CloudWatch metrics and alarms to Slack
    See more

    I'm building a new process management tool. I decided to build with Rails as my backend, using Sidekiq for background jobs. I chose to work with these tools because I've worked with them before and know that they're able to get the job done. They may not be the sexiest tools, but they work and are reliable, which is what I was optimizing for. For data stores, I opted for PostgreSQL and Redis. Because I'm planning on offering dashboards, I wanted a SQL database instead of something like MongoDB that might work early on, but be difficult to use as soon as I want to facilitate aggregate queries.

    On the front-end I'm using Vue.js and vuex in combination with #Turbolinks. In effect, I want to render most pages on the server side without key interactions being managed by Vue.js . This is the first project I'm working on where I've explicitly decided not to include jQuery . I have found React and Redux.js more confusing to setup. I appreciate the opinionated approach from the Vue.js community and that things just work together the way that I'd expect. To manage my javascript dependencies, I'm using Yarn .

    For CSS frameworks, I'm using #Bulma.io. I really appreciate it's minimal nature and that there are no hard javascript dependencies. And to add a little spice, I'm using #font-awesome.

    See more
    delayed_job logo

    delayed_job

    47
    53
    6
    Database backed asynchronous priority queue -- Extracted from Shopify
    47
    53
    + 1
    6
    PROS OF DELAYED_JOB
    • 3
      Easy to get started
    • 2
      Reliable
    • 1
      Doesn't require Redis
    CONS OF DELAYED_JOB
      Be the first to leave a con

      related delayed_job posts

      Jerome Dalbert
      Senior Backend Engineer at StackShare · | 4 upvotes · 73.3K views

      delayed_job is a great Rails background job library for new projects, as it only uses what you already have: a relational database. We happily used it during the company’s first two years.

      But it started to falter as our web and database transactions significantly grew. Our app interacted with users via SMS texts sent inside background jobs. Because the delayed_job daemon ran every couple seconds, this meant that users often waited several long seconds before getting text replies, which was not acceptable. Moreover, job processing was done inside AWS Elastic Beanstalk web instances, which were already under stress and not meant to handle jobs.

      We needed a fast background job system that could process jobs in near real-time and integrate well with AWS. Sidekiq is a fast and popular Ruby background job library, but it does not leverage the Elastic Beanstalk worker architecture, and you have to maintain a Redis instance.

      We ended up choosing active-elastic-job, which seamlessly integrates with worker instances and Amazon SQS. SQS is a fast queue and you don’t need to worry about infrastructure or scaling, as AWS handles it for you.

      We noticed significant performance gains immediately after making the switch.

      #BackgroundProcessing

      See more
      Jerome Dalbert
      Senior Backend Engineer at StackShare · | 3 upvotes · 47.8K views

      We use Sidekiq to process millions of Ruby background jobs a day under normal loads. We sometimes process more than that when running one-off backfill tasks.

      With so many jobs, it wouldn't really make sense to use delayed_job, as it would put our main database under unnecessary load, which would make it a bottleneck with most DB queries serving jobs and not end users. I suppose you could create a separate DB just for jobs, but that can be a hassle. Sidekiq uses a separate Redis instance so you don't have this problem. And it is very performant!

      I also like that its free version comes "batteries included" with:

      • A web monitoring UI that provides some nice stats.
      • An API that can come in handy for one-off tasks, like changing the queue of certain already enqueued jobs.

      Sidekiq is a pleasure to use. All our engineers love it!

      See more
      Celery logo

      Celery

      1.3K
      1.2K
      260
      Distributed task queue
      1.3K
      1.2K
      + 1
      260
      PROS OF CELERY
      • 93
        Task queue
      • 59
        Python integration
      • 36
        Django integration
      • 28
        Scheduled Task
      • 18
        Publish/subsribe
      • 6
        Easy to use
      • 6
        Various backend broker
      • 5
        Great community
      • 4
        Free
      • 4
        Workflow
      • 1
        Dynamic
      CONS OF CELERY
      • 4
        Sometimes loses tasks
      • 1
        Depends on broker

      related Celery posts

      James Cunningham
      Operations Engineer at Sentry · | 18 upvotes · 1.2M views
      Shared insights
      on
      Celery
      RabbitMQ
      at

      As Sentry runs throughout the day, there are about 50 different offline tasks that we execute—anything from “process this event, pretty please” to “send all of these cool people some emails.” There are some that we execute once a day and some that execute thousands per second.

      Managing this variety requires a reliably high-throughput message-passing technology. We use Celery's RabbitMQ implementation, and we stumbled upon a great feature called Federation that allows us to partition our task queue across any number of RabbitMQ servers and gives us the confidence that, if any single server gets backlogged, others will pitch in and distribute some of the backlogged tasks to their consumers.

      #MessageQueue

      See more
      Pulkit Sapra

      Hi! I am creating a scraping system in Django, which involves long running tasks between 1 minute & 1 Day. As I am new to Message Brokers and Task Queues, I need advice on which architecture to use for my system. ( Amazon SQS, RabbitMQ, or Celery). The system should be autoscalable using Kubernetes(K8) based on the number of pending tasks in the queue.

      See more
      Beanstalkd logo

      Beanstalkd

      105
      128
      74
      A simple, fast work queue
      105
      128
      + 1
      74
      PROS OF BEANSTALKD
      • 23
        Fast
      • 12
        Free
      • 12
        Does one thing well
      • 9
        Scalability
      • 8
        Simplicity
      • 3
        External admin UI developer friendly
      • 3
        Job delay
      • 2
        Job prioritization
      • 2
        External admin UI
      CONS OF BEANSTALKD
        Be the first to leave a con

        related Beanstalkd posts

        Frédéric MARAND
        Core Developer at OSInet · | 2 upvotes · 179K views

        I used Kafka originally because it was mandated as part of the top-level IT requirements at a Fortune 500 client. What I found was that it was orders of magnitude more complex ...and powerful than my daily Beanstalkd , and far more flexible, resilient, and manageable than RabbitMQ.

        So for any case where utmost flexibility and resilience are part of the deal, I would use Kafka again. But due to the complexities involved, for any time where this level of scalability is not required, I would probably just use Beanstalkd for its simplicity.

        I tend to find RabbitMQ to be in an uncomfortable middle place between these two extremities.

        See more
        RabbitMQ logo

        RabbitMQ

        13.6K
        11.7K
        511
        Open source multiprotocol messaging broker
        13.6K
        11.7K
        + 1
        511
        PROS OF RABBITMQ
        • 226
          It's fast and it works with good metrics/monitoring
        • 79
          Ease of configuration
        • 57
          I like the admin interface
        • 49
          Easy to set-up and start with
        • 20
          Durable
        • 18
          Intuitive work through python
        • 18
          Standard protocols
        • 10
          Written primarily in Erlang
        • 7
          Simply superb
        • 6
          Completeness of messaging patterns
        • 3
          Reliable
        • 3
          Scales to 1 million messages per second
        • 2
          Distributed
        • 2
          Supports AMQP
        • 2
          Better than most traditional queue based message broker
        • 1
          High performance
        • 1
          Reliability
        • 1
          Clusterable
        • 1
          Inubit Integration
        • 1
          Clear documentation with different scripting language
        • 1
          Great ui
        • 1
          Runs on Open Telecom Platform
        • 1
          Better routing system
        • 1
          Supports MQTT
        CONS OF RABBITMQ
        • 9
          Too complicated cluster/HA config and management
        • 6
          Needs Erlang runtime. Need ops good with Erlang runtime
        • 5
          Configuration must be done first, not by your code
        • 4
          Slow

        related RabbitMQ posts

        James Cunningham
        Operations Engineer at Sentry · | 18 upvotes · 1.2M views
        Shared insights
        on
        Celery
        RabbitMQ
        at

        As Sentry runs throughout the day, there are about 50 different offline tasks that we execute—anything from “process this event, pretty please” to “send all of these cool people some emails.” There are some that we execute once a day and some that execute thousands per second.

        Managing this variety requires a reliably high-throughput message-passing technology. We use Celery's RabbitMQ implementation, and we stumbled upon a great feature called Federation that allows us to partition our task queue across any number of RabbitMQ servers and gives us the confidence that, if any single server gets backlogged, others will pitch in and distribute some of the backlogged tasks to their consumers.

        #MessageQueue

        See more
        Tim Abbott
        Shared insights
        on
        RabbitMQ
        Python
        Redis
        at

        We've been using RabbitMQ as Zulip's queuing system since we needed a queuing system. What I like about it is that it scales really well and has good libraries for a wide range of platforms, including our own Python. So aside from getting it running, we've had to put basically 0 effort into making it scale for our needs.

        However, there's several things that could be better about it: * It's error messages are absolutely terrible; if ever one of our users ends up getting an error with RabbitMQ (even for simple things like a misconfigured hostname), they always end up needing to get help from the Zulip team, because the errors logs are just inscrutable. As an open source project, we've handled this issue by really carefully scripting the installation to be a failure-proof configuration (in this case, setting the RabbitMQ hostname to 127.0.0.1, so that no user-controlled configuration can break it). But it was a real pain to get there and the process of determining we needed to do that caused a significant amount of pain to folks installing Zulip. * The pika library for Python takes a lot of time to startup a RabbitMQ connection; this means that Zulip server restarts are more disruptive than would be ideal. * It's annoying that you need to run the rabbitmqctl management commands as root.

        But overall, I like that it has clean, clear semanstics and high scalability, and haven't been tempted to do the work to migrate to something like Redis (which has its own downsides).

        See more
        RAKE logo

        RAKE

        55
        30
        0
        A software task management and build automation tool
        55
        30
        + 1
        0
        PROS OF RAKE
          Be the first to leave a pro
          CONS OF RAKE
            Be the first to leave a con

            related RAKE posts

            Hangfire logo

            Hangfire

            119
            135
            15
            Perform background processing in .NET and .NET Core applications
            119
            135
            + 1
            15
            PROS OF HANGFIRE
            • 6
              Integrated UI dashboard
            • 5
              Simple
            • 2
              In Memory
            • 2
              Robust
            • 0
              Cons
            • 0
              Simole
            CONS OF HANGFIRE
              Be the first to leave a con

              related Hangfire posts

              PHP-FPM logo

              PHP-FPM

              81
              83
              0
              An alternative FastCGI daemon for PHP
              81
              83
              + 1
              0
              PROS OF PHP-FPM
                Be the first to leave a pro
                CONS OF PHP-FPM
                  Be the first to leave a con

                  related PHP-FPM posts