Alternatives to Scrapy logo

Alternatives to Scrapy

Selenium, import.io, BeautifulSoup, Puppeteer, and ParseHub are the most popular alternatives and competitors to Scrapy.
122
95
+ 1
0

What is Scrapy and what are its top alternatives?

It is the most popular web scraping framework in Python. An open source and collaborative framework for extracting the data you need from websites. In a fast, simple, yet extensible way.
Scrapy is a tool in the Web Scraping API category of a tech stack.
Scrapy is an open source tool with 38K GitHub stars and 8.7K GitHub forks. Here鈥檚 a link to Scrapy's open source repository on GitHub

Top Alternatives to Scrapy

Scrapy alternatives & related posts

related Selenium posts

Kamil Kowalski
Kamil Kowalski
Engineering Manager at Fresha | 27 upvotes 路 598.1K views

When you think about test automation, it鈥檚 crucial to make it everyone鈥檚 responsibility (not just QA Engineers'). We started with Selenium and Java, but with our platform revolving around Ruby, Elixir and JavaScript, QA Engineers were left alone to automate tests. Cypress was the answer, as we could switch to JS and simply involve more people from day one. There's a downside too, as it meant testing on Chrome only, but that was "good enough" for us + if really needed we can always cover some specific cases in a different way.

See more
Benjamin Poon
Benjamin Poon
QA Manager - Engineering at HBC Digital | 8 upvotes 路 534.8K views

For our digital QA organization to support a complex hybrid monolith/microservice architecture, our team took on the lofty goal of building out a commonized UI test automation framework. One of the primary requisites included a technical minimalist threshold such that an engineer or analyst with fundamental knowledge of JavaScript could automate their tests with greater ease. Just to list a few: - Nightwatchjs - Selenium - Cucumber - GitHub - Go.CD - Docker - ExpressJS - React - PostgreSQL

With this structure, we're able to combine the automation efforts of each team member into a centralized repository while also providing new relevant metrics to business owners.

See more
import.io logo

import.io

31
65
23
Extract data from the web
31
65
+ 1
23

related import.io posts

BeautifulSoup logo

BeautifulSoup

39
33
2
A Python library for pulling data out of HTML and XML files
39
33
+ 1
2
CONS OF BEAUTIFULSOUP
    No cons available

    related BeautifulSoup posts

    Puppeteer logo

    Puppeteer

    232
    277
    9
    Headless Chrome Node API
    232
    277
    + 1
    9

    related Puppeteer posts

    ParseHub logo

    ParseHub

    20
    40
    8
    Turn dynamic websites into APIs
    20
    40
    + 1
    8
    CONS OF PARSEHUB
      No cons available

      related ParseHub posts

      Portia logo

      Portia

      16
      46
      0
      Visual web scraping tool that lets you extract data without writing a single line of code
      16
      46
      + 1
      0
      PROS OF PORTIA
        No pros available
        CONS OF PORTIA
          No cons available

          related Portia posts

          Kimono logo

          Kimono

          13
          22
          5
          Turn websites into structured APIs from your browser in seconds
          13
          22
          + 1
          5

          related Kimono posts

          Apifier logo

          Apifier

          11
          50
          3
          Hosted web crawler for developers
          11
          50
          + 1
          3
          CONS OF APIFIER
            No cons available

            related Apifier posts