Celery vs. ActiveMQ vs. Starling

Implements the TaskQueue API.

For orchestrating the creation of the correct number of instances, managing errors and retries, and finally managing the deallocation of resources we use RabbitMQ in conjunction with the Celery Project framework, along with a self-developed workflow engine.

Task queues and cron jobs

백엔드, 어플리케이션 서버 모두 시간이 걸리는 작업들 및 주기적으로 수행해야 하는 작업들은 celery/celerybeat 을 통해 동작됩니다. 신뢰도가 아주 높으며, 유연합니다. gevent 로 동작시키면 더욱 깔끔한 관리가 가능합니다.

All of our background jobs (e.g., image resizing, file uploading, email and SMS sending) are done through Celery (using Redis as its broker). Celery's scheduling and retrying features are especially useful for error-prone tasks, such as email and SMS sending.

Using Celery, the web service creates tasks that are executed by a background worker. Celery uses a RabbitMQ instance as a task queue.

Automated distributed tasks.

We used celery, in combination with RabbitMQ and celery-beat, to run periodic tasks, as well as some user-initiated long-running tasks on the server.

Distributed Task Queue

万能胶也要省着用

http request 이후 바로 response가 즉시 오직 않는 많은 양의 무거운 작업은 celery에 task로 등록합니다.

그래서 유저에겐 빠른 response를 주고 celery에서 해당 업무를 수행합니다.

For offloading CPU intensive tasks to separate workers instead of running them on our application server.

background tasks.

python remote workers. simple and easy to use.

Task queues

Background Django/Python tasks. Populated Redis with server cache

Message passing.

Remote broker and local client for incoming data feeds. Local broker for republishing data feeds to other systems.