Get Advice Icon

Need advice about which tool to choose?Ask the StackShare community!

Celery
Celery

889
532
+ 1
239
Redis
Redis

14.6K
9.7K
+ 1
3.8K
Add tool

Celery vs Redis: What are the differences?

What is Celery? Distributed task queue. Celery is an asynchronous task queue/job queue based on distributed message passing. It is focused on real-time operation, but supports scheduling as well.

What is Redis? An in-memory database that persists on disk. Redis is an open source, BSD licensed, advanced key-value store. It is often referred to as a data structure server since keys can contain strings, hashes, lists, sets and sorted sets.

Celery belongs to "Message Queue" category of the tech stack, while Redis can be primarily classified under "In-Memory Databases".

"Task queue" is the primary reason why developers consider Celery over the competitors, whereas "Performance" was stated as the key factor in picking Redis.

Celery and Redis are both open source tools. It seems that Redis with 37.4K GitHub stars and 14.4K forks on GitHub has more adoption than Celery with 12.9K GitHub stars and 3.33K GitHub forks.

According to the StackShare community, Redis has a broader approval, being mentioned in 3261 company stacks & 1781 developers stacks; compared to Celery, which is listed in 272 company stacks and 77 developer stacks.

What is Celery?

Celery is an asynchronous task queue/job queue based on distributed message passing. It is focused on real-time operation, but supports scheduling as well.

What is Redis?

Redis is an open source, BSD licensed, advanced key-value store. It is often referred to as a data structure server since keys can contain strings, hashes, lists, sets and sorted sets.
Get Advice Icon

Need advice about which tool to choose?Ask the StackShare community!

Why do developers choose Celery?
Why do developers choose Redis?

Sign up to add, upvote and see more prosMake informed product decisions

What companies use Celery?
What companies use Redis?

Sign up to get full access to all the companiesMake informed product decisions

What tools integrate with Celery?
What tools integrate with Redis?

Sign up to get full access to all the tool integrationsMake informed product decisions

What are some alternatives to Celery and Redis?
RabbitMQ
RabbitMQ gives your applications a common platform to send and receive messages, and your messages a safe place to live until received.
Kafka
Kafka is a distributed, partitioned, replicated commit log service. It provides the functionality of a messaging system, but with a unique design.
Amazon SQS
Transmit any volume of data, at any level of throughput, without losing messages or requiring other services to be always available. With SQS, you can offload the administrative burden of operating and scaling a highly available messaging cluster, while paying a low price for only what you use.
ActiveMQ
Apache ActiveMQ is fast, supports many Cross Language Clients and Protocols, comes with easy to use Enterprise Integration Patterns and many advanced features while fully supporting JMS 1.1 and J2EE 1.4. Apache ActiveMQ is released under the Apache 2.0 License.
ZeroMQ
The 0MQ lightweight messaging kernel is a library which extends the standard socket interfaces with features traditionally provided by specialised messaging middleware products. 0MQ sockets provide an abstraction of asynchronous message queues, multiple messaging patterns, message filtering (subscriptions), seamless access to multiple transport protocols and more.
See all alternatives
Decisions about Celery and Redis
James Cunningham
James Cunningham
Operations Engineer at Sentry · | 22 upvotes · 137.1K views
atSentrySentry
Redis
Redis
PostgreSQL
PostgreSQL
Celery
Celery
Django
Django
#InMemoryDatabases
#MessageQueue

Sentry started as (and remains) an open-source project, growing out of an error logging tool built in 2008. That original build nine years ago was Django and Celery (Python’s asynchronous task codebase), with PostgreSQL as the database and Redis as the power behind Celery.

We displayed a truly shrewd notion of branding even then, giving the project a catchy name that companies the world over remain jealous of to this day: django-db-log. For the longest time, Sentry’s subtitle on GitHub was “A simple Django app, built with love.” A slightly more accurate description probably would have included Starcraft and Soylent alongside love; regardless, this captured what Sentry was all about.

#MessageQueue #InMemoryDatabases

See more
James Cunningham
James Cunningham
Operations Engineer at Sentry · | 18 upvotes · 110.2K views
atSentrySentry
RabbitMQ
RabbitMQ
Celery
Celery
#MessageQueue

As Sentry runs throughout the day, there are about 50 different offline tasks that we execute—anything from “process this event, pretty please” to “send all of these cool people some emails.” There are some that we execute once a day and some that execute thousands per second.

Managing this variety requires a reliably high-throughput message-passing technology. We use Celery's RabbitMQ implementation, and we stumbled upon a great feature called Federation that allows us to partition our task queue across any number of RabbitMQ servers and gives us the confidence that, if any single server gets backlogged, others will pitch in and distribute some of the backlogged tasks to their consumers.

#MessageQueue

See more
Eli Hooten
Eli Hooten
CTO at Codecov · | 6 upvotes · 12K views
atCodecovCodecov
Redis
Redis
Celery
Celery
Python
Python

A major aspect of Codecov is the use of long running asynchronous tasks to process large amounts of test coverage data uploaded by our users. Being a Python stack, Celery felt like a natural fit to manage codecov's long running tasks. We rely on Celery to manage all our background queues and asyncronous scheduling. Celery enables us to set timeouts for different tasks which has been instrumental in maintaining our queue in production. Celery also interfaces easily with Redis as a backend store, which allowed it to slot neatly into our existing infrastructure.

See more
Django
Django
Redis
Redis

I use Redis because, based on the case studies I have reviewed, it appears to be the most performant cache database for my Django projects. The ease of configuration and deployment is also a big plus.

Using both higher level view caching as well as low-level QuerySet caching with Redis has allowed me to improve HTTP request times by an order of magnitude.

See more
Michael Mota
Michael Mota
CEO & Founder at AlterEstate · | 4 upvotes · 10.9K views
atAlterEstateAlterEstate
Django
Django
RabbitMQ
RabbitMQ
Celery
Celery

Automations are what makes a CRM powerful. With Celery and RabbitMQ we've been able to make powerful automations that truly works for our clients. Such as for example, automatic daily reports, reminders for their activities, important notifications regarding their client activities and actions on the website and more.

We use Celery basically for everything that needs to be scheduled for the future, and using RabbitMQ as our Queue-broker is amazing since it fully integrates with Django and Celery storing on our database results of the tasks done so we can see if anything fails immediately.

See more
Interest over time
Reviews of Celery and Redis
Review ofRedisRedis

Redis is a good caching tool for a cluster, but our application had performance issues while using Aws Elasticache Redis since some page had 3000 cache hits per a page load and Redis just couldn't quickly process them all in once + latency and object deseialization time - page load took 8-9 seconds. We create a custom hybrid caching based on Redis and EhCache which worked great for our goals. Check it out on github, it's called HybriCache - https://github.com/batir-akhmerov/hybricache.

How developers use Celery and Redis
Avatar of Cloudcraft
Cloudcraft uses RedisRedis

Redis is used for storing all ephemeral (that's data you don't necessarily want to store permanently) user data, such as mapping of session IDs (stored in cookies) to current session variables at Cloudcraft.co. The many datastructures supported by Redis also makes it an excellent caching and realtime statistics layer. It doesn't hurt that the author, Antirez, is the nicest guy ever! These days, I would be really hard pressed to find any situation where I would pick something like Memcached over Redis.

Avatar of Trello
Trello uses RedisRedis

Trello uses Redis for ephemeral data that needs to be shared between server processes but not persisted to disk. Things like the activity level of a session or a temporary OpenID key are stored in Redis, and the application is built to recover gracefully if any of these (or all of them) are lost. We run with allkeys-lru enabled and about five times as much space as its actual working set needs, so Redis automatically discards data that hasn’t been accessed lately, and reconstructs it when necessary.

Avatar of Stack Exchange
Stack Exchange uses RedisRedis

The UI has message inbox that is sent a message when you get a new badge, receive a message, significant event, etc. Done using WebSockets and is powered by redis. Redis has 2 slaves, SQL has 2 replicas, tag engine has 3 nodes, elastic has 3 nodes - any other service has high availability as well (and exists in both data centers).

Avatar of Brandon Adams
Brandon Adams uses RedisRedis

Redis makes certain operations very easy. When I need a high-availability store, I typically look elsewhere, but for rapid development with the ability to land on your feet in prod, Redis is great. The available data types make it easy to build non-trivial indexes that would require complex queries in postgres.

Avatar of Kalibrr
Kalibrr uses CeleryCelery

All of our background jobs (e.g., image resizing, file uploading, email and SMS sending) are done through Celery (using Redis as its broker). Celery's scheduling and retrying features are especially useful for error-prone tasks, such as email and SMS sending.

Avatar of Kent Steiner
Kent Steiner uses RedisRedis

I use Redis for cacheing, data storage, mining and augmentation, proprietary distributed event system for disparate apps and services to talk to each other, and more. Redis has some very useful native data types for tracking, slicing and dicing information.

Avatar of Cloudify
Cloudify uses CeleryCelery

For orchestrating the creation of the correct number of instances, managing errors and retries, and finally managing the deallocation of resources we use RabbitMQ in conjunction with the Celery Project framework, along with a self-developed workflow engine.

Avatar of MOKA Analytics
MOKA Analytics uses CeleryCelery

We maintain a fork of Celery 3 that adds HTTPS support for Redis brokers. The Winning Model currently uses Celery 3 because Celery 4 dropped support for Windows.

We plan on migrating to Celery 4 once Azure ASE supports Linux apps

Avatar of Yaakov Gesher
Yaakov Gesher uses CeleryCelery

We used celery, in combination with RabbitMQ and celery-beat, to run periodic tasks, as well as some user-initiated long-running tasks on the server.

Avatar of Dieter Adriaenssens
Dieter Adriaenssens uses CeleryCelery

Using Celery, the web service creates tasks that are executed by a background worker. Celery uses a RabbitMQ instance as a task queue.

How much does Celery cost?
How much does Redis cost?
Pricing unavailable
Pricing unavailable