Scrapy vs Selenium: What are the differences?
What is Scrapy? A fast high-level web crawling & scraping framework for Python. It is the most popular web scraping framework in Python. An open source and collaborative framework for extracting the data you need from websites. In a fast, simple, yet extensible way.
What is Selenium? Web Browser Automation. Selenium automates browsers. That's it! What you do with that power is entirely up to you. Primarily, it is for automating web applications for testing purposes, but is certainly not limited to just that. Boring web-based administration tasks can (and should!) also be automated as well.
Scrapy and Selenium are primarily classified as "Web Scraping API" and "Browser Testing" tools respectively.
Scrapy and Selenium are both open source tools. It seems that Scrapy with 33.5K GitHub stars and 7.87K forks on GitHub has more adoption than Selenium with 14.9K GitHub stars and 4.97K GitHub forks.
According to the StackShare community, Selenium has a broader approval, being mentioned in 777 company stacks & 443 developers stacks; compared to Scrapy, which is listed in 11 company stacks and 5 developer stacks.
What is Scrapy?
What is Selenium?
Need advice about which tool to choose?Ask the StackShare community!
Why do developers choose Scrapy?
Sign up to add, upvote and see more prosMake informed product decisions
What are the cons of using Scrapy?
What are the cons of using Selenium?
Sign up to get full access to all the companiesMake informed product decisions
Sign up to get full access to all the tool integrationsMake informed product decisions
가장 강력한 웹 UI / UX 테스트 툴 이지만, 문제도 많다. iframe 안에 있는 건 잘 select 가 안되기도 하고, 화면이 작아 컴포넌트가 화면에 보이지 않으면 또 select 할 수 없다. 하지만 브라우저를 있는 그대로 제어 할 수 있다는 게 가장 큰 매력이다.
ps. selenide 와 같이 쓰면 코드가 많이 개선된다.
Maintaining and expanding automated regression tests is one of my primary responsibilities at CrowdTwist. We use Selenium to test the front-end parts of our platform.
Used Selenium web-driver for automated QA testing in continuous integration and continuous deployment.
- Navigating our site for tests
- Navigating other sites that discriminate against non-human visitors