Help developers discover the tools you use. Get visibility for your team's tech choices and contribute to the community's knowledge.
Developer-friendly API to scrape websites and get content in HTML, JSON, or markdown for AI apps and agents. | Armada is an orchestration platform for running bots and scrapers at scale on Kubernetes. You write a Python script (Playwright, Selenium, or nodriver), add a JSON config, and Armada handles the rest : distributing jobs across as many pods as you need, rotating proxies, managing browser fingerprints, and monitoring everything in real time through a dashboard. Going from 1 worker to 100+ requires zero code changes. No SaaS, no credits, fully self-hosted. |
Rotating Proxies, Headless Browser, Multiple Outputs | Task Distribution, Proxy Manager, Anti-Detection, Unlimited Scaling, Centralized Monitoring |
Statistics | |
Stacks 0 | Stacks 0 |
Followers 1 | Followers 1 |
Votes 1 | Votes 1 |

Sidekiq uses threads to handle many jobs at the same time in the same process. It does not require Rails but will integrate tightly with Rails 3/4 to make background processing dead simple.

It is a Node library to automate the Chromium, WebKit and Firefox browsers with a single API. It enables cross-browser web automation that is ever-green, capable, reliable and fast.

Beanstalks's interface is generic, but was originally designed for reducing the latency of page views in high-volume web applications by running time-consuming tasks asynchronously.

Puppeteer is a Node library which provides a high-level API to control headless Chrome over the DevTools Protocol. It can also be configured to use full (non-headless) Chrome.

import.io is a free web-based platform that puts the power of the machine readable web in your hands. Using our tools you can create an API or crawl an entire website in a fraction of the time of traditional methods, no coding required.

Web Scraping and Data Extraction ParseHub is a free and powerful web scraping tool. With our advanced web scraper, extracting data is as easy as clicking on the data you need. ParseHub lets you turn any website into a spreadsheet or API w

PhantomJS is a headless WebKit scriptable with JavaScript. It is used by hundreds of developers and dozens of organizations for web-related development workflow.

It is an open-source framework that helps you to create, process and manage your background jobs, i.e. operations you don't want to put in your request processing pipeline. It supports all kind of background tasks – short-running and long-running, CPU intensive and I/O intensive, one shot and recurrent.

Extract data from websites and turn them to API. We will handle all the rotating proxies and Chrome rendering for you. Many specialists have to handle Javascript rendering, headless browser update and maintenance, proxies diversity and rotation. It is a simple API that does all the above for you.

It is a free client-side Windows web scraping software that turns unstructured or semi-structured data from websites into structured data sets, no coding necessary. Extracted data can be exported as API, CSV, Excel or exported into a database.