What is Scrapy?
It is the most popular web scraping framework in Python. An open source and collaborative framework for extracting the data you need from websites. In a fast, simple, yet extensible way.
Scrapy is a tool in the Web Scraping API category of a tech stack.
Scrapy is an open source tool with 41K GitHub stars and 9.2K GitHub forks. Here’s a link to Scrapy's open source repository on GitHub
Who uses Scrapy?
23 companies reportedly use Scrapy in their tech stacks, including WISESIGHT, Yintrust, and Witei.
143 developers on StackShare have stated that they use Scrapy.
Decisions about Scrapy
Here are some stack decisions, common use cases and reviews by companies and developers who chose Scrapy in their tech stack.
There is no doubt that Scrapy is the preferred Python crawler framework.
Scrapy Alternatives & Comparisons
What are some alternatives to Scrapy?
See all alternatives
Selenium automates browsers. That's it! What you do with that power is entirely up to you. Primarily, it is for automating web applications for testing purposes, but is certainly not limited to just that. Boring web-based administration tasks can (and should!) also be automated as well.
import.io is a free web-based platform that puts the power of the machine readable web in your hands. Using our tools you can create an API or crawl an entire website in a fraction of the time of traditional methods, no coding required.
It works with your favorite parser to provide idiomatic ways of navigating, searching, and modifying the parse tree. It commonly saves programmers hours or days of work.
Puppeteer is a Node library which provides a high-level API to control headless Chrome over the DevTools Protocol. It can also be configured to use full (non-headless) Chrome.
Web Scraping and Data Extraction ParseHub is a free and powerful web scraping tool. With our advanced web scraper, extracting data is as easy as clicking on the data you need. ParseHub lets you turn any website into a spreadsheet or API w