Help developers discover the tools you use. Get visibility for your team's tech choices and contribute to the community's knowledge.
Production-grade workflow automation. No drag-and-drop required. Build, version, and deploy your workflows with YAML. | Armada is an orchestration platform for running bots and scrapers at scale on Kubernetes. You write a Python script (Playwright, Selenium, or nodriver), add a JSON config, and Armada handles the rest : distributing jobs across as many pods as you need, rotating proxies, managing browser fingerprints, and monitoring everything in real time through a dashboard. Going from 1 worker to 100+ requires zero code changes. No SaaS, no credits, fully self-hosted. |
✓ 100 credits included, ✓ All integrations, ✓ Custom python integrations, ✓ Community support, ✓ 10,000 credits included | Task Distribution, Proxy Manager, Anti-Detection, Unlimited Scaling, Centralized Monitoring |
Statistics | |
Stacks 1 | Stacks 0 |
Followers 2 | Followers 1 |
Votes 1 | Votes 1 |

Sidekiq uses threads to handle many jobs at the same time in the same process. It does not require Rails but will integrate tightly with Rails 3/4 to make background processing dead simple.

Use Airflow to author workflows as directed acyclic graphs (DAGs) of tasks. The Airflow scheduler executes your tasks on an array of workers while following the specified dependencies. Rich command lines utilities makes performing complex surgeries on DAGs a snap. The rich user interface makes it easy to visualize pipelines running in production, monitor progress and troubleshoot issues when needed.

It is a Node library to automate the Chromium, WebKit and Firefox browsers with a single API. It enables cross-browser web automation that is ever-green, capable, reliable and fast.

Beanstalks's interface is generic, but was originally designed for reducing the latency of page views in high-volume web applications by running time-consuming tasks asynchronously.

It makes it easy to automate all your software workflows, now with world-class CI/CD. Build, test, and deploy your code right from GitHub. Make code reviews, branch management, and issue triaging work the way you want.

Puppeteer is a Node library which provides a high-level API to control headless Chrome over the DevTools Protocol. It can also be configured to use full (non-headless) Chrome.

PhantomJS is a headless WebKit scriptable with JavaScript. It is used by hundreds of developers and dozens of organizations for web-related development workflow.

It is an open-source framework that helps you to create, process and manage your background jobs, i.e. operations you don't want to put in your request processing pipeline. It supports all kind of background tasks – short-running and long-running, CPU intensive and I/O intensive, one shot and recurrent.

It implements batch and streaming data processing jobs that run on any execution engine. It executes pipelines on multiple execution environments.

Developer framework to orchestrate multiple services and APIs into your software application using logic triggered by events and time. Build ETL processes, A/B testing, real-time alerts and personalized user experiences with custom logic.