Scrapy logo
A fast high-level web crawling & scraping framework for Python
37
15
+ 1
0

What is Scrapy?

It is the most popular web scraping framework in Python. An open source and collaborative framework for extracting the data you need from websites. In a fast, simple, yet extensible way.
Scrapy is a tool in the Web Scraping API category of a tech stack.
Scrapy is an open source tool with 34.1K GitHub stars and 8K GitHub forks. Here’s a link to Scrapy's open source repository on GitHub

Who uses Scrapy?

Companies
15 companies reportedly use Scrapy in their tech stacks, including Jobs2Careers, NITOZA, and RANSEL.ru.

Developers
20 developers on StackShare have stated that they use Scrapy.

Why developers like Scrapy?

Here’s a list of reasons why companies and developers use Scrapy
Top Reasons
Be the first to leave a pro

Scrapy Alternatives & Comparisons

What are some alternatives to Scrapy?
Selenium
Selenium automates browsers. That's it! What you do with that power is entirely up to you. Primarily, it is for automating web applications for testing purposes, but is certainly not limited to just that. Boring web-based administration tasks can (and should!) also be automated as well.
import.io
import.io is a free web-based platform that puts the power of the machine readable web in your hands. Using our tools you can create an API or crawl an entire website in a fraction of the time of traditional methods, no coding required.
BeautifulSoup
It works with your favorite parser to provide idiomatic ways of navigating, searching, and modifying the parse tree. It commonly saves programmers hours or days of work.
Kimono
You don't need to write any code or install any software to extract data with Kimono. The easiest way to use Kimono is to add our bookmarklet to your browser's bookmark bar. Then go to the website you want to get data from and click the bookmarklet. Select the data you want and Kimono does the rest. We take care of hosting the APIs that you build with Kimono and running them on the schedule you specify. Use the API output in JSON or as CSV files that you can easily paste into a spreadsheet.
Portia
Portia is an open source tool that lets you get data from websites. It facilitates and automates the process of data extraction. This visual web scraper works straight from your browser, so you don't need to download or install anything.
See all alternatives

Scrapy's Stats

Scrapy's Followers
15 developers follow Scrapy to keep up with related blogs and decisions.
g4ood3
Xing Sun
Chung Jaehoon
Garima Singh
Sudharshan Ravindran
Karl Leicht
Roscoe Cairney
Shawn Niederriter
Changkun Ou
Michael Mejia