Celery vs Kafka Manager: What are the differences?
Developers describe Celery as "Distributed task queue". Celery is an asynchronous task queue/job queue based on distributed message passing. It is focused on real-time operation, but supports scheduling as well. On the other hand, Kafka Manager is detailed as "A tool for managing Apache Kafka, developed by Yahoo". This interface makes it easier to identify topics which are unevenly distributed across the cluster or have partition leaders unevenly distributed across the cluster. It supports management of multiple clusters, preferred replica election, replica re-assignment, and topic creation. It is also great for getting a quick bird’s eye view of the cluster.
Celery and Kafka Manager belong to "Message Queue" category of the tech stack.
Celery and Kafka Manager are both open source tools. It seems that Celery with 12.9K GitHub stars and 3.33K forks on GitHub has more adoption than Kafka Manager with 7.55K GitHub stars and 1.84K GitHub forks.
Udemy, Robinhood, and Sentry are some of the popular companies that use Celery, whereas Kafka Manager is used by Yahoo!, IgnitionOne, and Ocado Technology. Celery has a broader approval, being mentioned in 272 company stacks & 77 developers stacks; compared to Kafka Manager, which is listed in 8 company stacks and 5 developer stacks.
What is Celery?
What is Kafka Manager?
Want advice about which of these to choose?Ask the StackShare community!
Why do developers choose Kafka Manager?
Sign up to add, upvote and see more prosMake informed product decisions
What are the cons of using Kafka Manager?
Sign up to get full access to all the companiesMake informed product decisions
All of our background jobs (e.g., image resizing, file uploading, email and SMS sending) are done through Celery (using Redis as its broker). Celery's scheduling and retrying features are especially useful for error-prone tasks, such as email and SMS sending.
For orchestrating the creation of the correct number of instances, managing errors and retries, and finally managing the deallocation of resources we use RabbitMQ in conjunction with the Celery Project framework, along with a self-developed workflow engine.
We maintain a fork of Celery 3 that adds HTTPS support for Redis brokers. The Winning Model currently uses Celery 3 because Celery 4 dropped support for Windows.
We plan on migrating to Celery 4 once Azure ASE supports Linux apps
We used celery, in combination with RabbitMQ and celery-beat, to run periodic tasks, as well as some user-initiated long-running tasks on the server.