Need advice about which tool to choose?Ask the StackShare community!

Celery

1.6K
1.6K
+ 1
280
Resque

118
126
+ 1
9
Add tool

Celery vs Resque: What are the differences?

1. Language: Celery is written in Python, while Resque is written in Ruby. 2. Serialization: Celery uses JSON and MessagePack for serialization, whereas Resque uses only JSON. 3. Dependencies: Celery has more dependencies compared to Resque, making setup a bit more complex. 4. Database support: Celery supports various backends like RabbitMQ, Redis, and Amazon SQS, whereas Resque primarily uses Redis. 5. Monitoring: Celery provides real-time monitoring with tools like Flower, while Resque does not have built-in monitoring tools. 6. Community: Celery has a larger community and more extensive documentation compared to Resque.

In Summary, Celery and Resque differ in language, serialization, dependencies, database support, monitoring, and community size.

Advice on Celery and Resque
Needs advice
on
CeleryCelery
and
RabbitMQRabbitMQ

I am just a beginner at these two technologies.

Problem statement: I am getting lakh of users from the sequel server for whom I need to create caches in MongoDB by making different REST API requests.

Here these users can be treated as messages. Each REST API request is a task.

I am confused about whether I should go for RabbitMQ alone or Celery.

If I have to go with RabbitMQ, I prefer to use python with Pika module. But the challenge with Pika is, it is not thread-safe. So I am not finding a way to execute a lakh of API requests in parallel using multiple threads using Pika.

If I have to go with Celery, I don't know how I can achieve better scalability in executing these API requests in parallel.

See more
Replies (1)
Recommends
on
rqrqRedisRedis

For large amounts of small tasks and caches I have had good luck with Redis and RQ. I have not personally used celery but I am fairly sure it would scale well, and I have not used RabbitMQ for anything besides communication between services. If you prefer python my suggestions should feel comfortable.

Sorry I do not have a more information

See more
Get Advice from developers at your company using StackShare Enterprise. Sign up for StackShare Enterprise.
Learn More
Pros of Celery
Pros of Resque
  • 99
    Task queue
  • 63
    Python integration
  • 40
    Django integration
  • 30
    Scheduled Task
  • 19
    Publish/subsribe
  • 8
    Various backend broker
  • 6
    Easy to use
  • 5
    Great community
  • 5
    Workflow
  • 4
    Free
  • 1
    Dynamic
  • 5
    Free
  • 3
    Scalable
  • 1
    Easy to use on heroku

Sign up to add or upvote prosMake informed product decisions

Cons of Celery
Cons of Resque
  • 4
    Sometimes loses tasks
  • 1
    Depends on broker
    Be the first to leave a con

    Sign up to add or upvote consMake informed product decisions

    What is Celery?

    Celery is an asynchronous task queue/job queue based on distributed message passing. It is focused on real-time operation, but supports scheduling as well.

    What is Resque?

    Background jobs can be any Ruby class or module that responds to perform. Your existing classes can easily be converted to background jobs or you can create new classes specifically to do work. Or, you can do both.

    Need advice about which tool to choose?Ask the StackShare community!

    Jobs that mention Celery and Resque as a desired skillset
    What companies use Celery?
    What companies use Resque?
    See which teams inside your own company are using Celery or Resque.
    Sign up for StackShare EnterpriseLearn More

    Sign up to get full access to all the companiesMake informed product decisions

    What tools integrate with Celery?
    What tools integrate with Resque?

    Sign up to get full access to all the tool integrationsMake informed product decisions

    Blog Posts

    GitHubPythonNode.js+47
    55
    72467
    JavaScriptGitHubPython+42
    53
    21973
    GitHubPythonSlack+25
    7
    3181
    GitHubPythonDocker+24
    13
    17038
    What are some alternatives to Celery and Resque?
    RabbitMQ
    RabbitMQ gives your applications a common platform to send and receive messages, and your messages a safe place to live until received.
    Kafka
    Kafka is a distributed, partitioned, replicated commit log service. It provides the functionality of a messaging system, but with a unique design.
    Airflow
    Use Airflow to author workflows as directed acyclic graphs (DAGs) of tasks. The Airflow scheduler executes your tasks on an array of workers while following the specified dependencies. Rich command lines utilities makes performing complex surgeries on DAGs a snap. The rich user interface makes it easy to visualize pipelines running in production, monitor progress and troubleshoot issues when needed.
    Cucumber
    Cucumber is a tool that supports Behaviour-Driven Development (BDD) - a software development process that aims to enhance software quality and reduce maintenance costs.
    JavaScript
    JavaScript is most known as the scripting language for Web pages, but used in many non-browser environments as well such as node.js or Apache CouchDB. It is a prototype-based, multi-paradigm scripting language that is dynamic,and supports object-oriented, imperative, and functional programming styles.
    See all alternatives