Scrapy vs Selenium: What are the differences?
What is Scrapy? A fast high-level web crawling & scraping framework for Python. It is the most popular web scraping framework in Python. An open source and collaborative framework for extracting the data you need from websites. In a fast, simple, yet extensible way.
What is Selenium? Web Browser Automation. Selenium automates browsers. That's it! What you do with that power is entirely up to you. Primarily, it is for automating web applications for testing purposes, but is certainly not limited to just that. Boring web-based administration tasks can (and should!) also be automated as well.
Scrapy and Selenium are primarily classified as "Web Scraping API" and "Browser Testing" tools respectively.
Scrapy and Selenium are both open source tools. It seems that Scrapy with 33.5K GitHub stars and 7.87K forks on GitHub has more adoption than Selenium with 14.9K GitHub stars and 4.97K GitHub forks.
According to the StackShare community, Selenium has a broader approval, being mentioned in 777 company stacks & 443 developers stacks; compared to Scrapy, which is listed in 11 company stacks and 5 developer stacks.
Sign up to add or upvote prosMake informed product decisions
Sign up to add or upvote consMake informed product decisions
What is Scrapy?
What is Selenium?
Need advice about which tool to choose?Ask the StackShare community!
Sign up to get full access to all the companiesMake informed product decisions
Sign up to get full access to all the tool integrationsMake informed product decisions