What is Scrapy and what are its top alternatives?
Top Alternatives to Scrapy
Selenium automates browsers. That's it! What you do with that power is entirely up to you. Primarily, it is for automating web applications for testing purposes, but is certainly not limited to just that. Boring web-based administration tasks can (and should!) also be automated as well. ...
import.io is a free web-based platform that puts the power of the machine readable web in your hands. Using our tools you can create an API or crawl an entire website in a fraction of the time of traditional methods, no coding required. ...
It works with your favorite parser to provide idiomatic ways of navigating, searching, and modifying the parse tree. It commonly saves programmers hours or days of work. ...
Puppeteer is a Node library which provides a high-level API to control headless Chrome over the DevTools Protocol. It can also be configured to use full (non-headless) Chrome. ...
Web Scraping and Data Extraction ParseHub is a free and powerful web scraping tool. With our advanced web scraper, extracting data is as easy as clicking on the data you need. ParseHub lets you turn any website into a spreadsheet or API w ...
Portia is an open source tool that lets you get data from websites. It facilitates and automates the process of data extraction. This visual web scraper works straight from your browser, so you don't need to download or install anything. ...
You don't need to write any code or install any software to extract data with Kimono. The easiest way to use Kimono is to add our bookmarklet to your browser's bookmark bar. Then go to the website you want to get data from and click the bookmarklet. Select the data you want and Kimono does the rest. We take care of hosting the APIs that you build with Kimono and running them on the schedule you specify. Use the API output in JSON or as CSV files that you can easily paste into a spreadsheet. ...
Scrapy alternatives & related posts
- Automates browsers169
- Essential tool for running test automation101
- Remote Control24
- Data crawling8
- Supports end to end testing7
- Functional testing6
- Easy set up6
- The Most flexible monitoring system4
- Easy to integrate with build tools3
- End to End Testing3
- Integration Tests2
- Comparing the performance selenium is faster than jasm2
- Record and playback2
- Compatible with Python2
- Easy to scale2
- Integrated into Selenium-Jupiter framework0
- Flaky tests7
- Slow as needs to make browser (even with no gui)2
related Selenium posts
With this structure, we're able to combine the automation efforts of each team member into a centralized repository while also providing new relevant metrics to business owners.
- Easy setup7
- Free lead generation tool5
- Native desktop app5
- Continuous updates3
- Features based on users suggestions3
related import.io posts
- Parsed html even when poorly formed2
- It just works1
- Scriptable web browser8
- Very well documented8
- Promise based5
- Chrome only7
related Puppeteer posts
Currently, we are using Protractor in our project. Since Protractor isn't updated anymore, we are looking for a new tool. The strongest suggestions are WebdriverIO or Puppeteer. Please help me figure out what tool would make the transition fastest and easiest. Please note that Protractor uses its own locator system, and we want the switch to be as simple as possible. Thank you!
I work in a company building web apps with AngularJS. I started using Selenium for tests automation, as I am more familiar with Python. However, I found some difficulties, like the impossibility of using IDs and fixed lists of classes, ending up with using xpaths most, which unfortunately could change with fixes and modifications in the code.
Any comments on this comparison and also on comparisons with similar tools are welcome! :)
- Great support5
- Easy setup4
- Complex websites3
- Native Desktop App2
related Portia posts
- Easy setup2
- Extracting data from ecommerce sites1
- Integrate API with web application1
- Data Scraping1