Skip to content

Add 2scraper to Web Scraping (Content Extraction) - #3404

Open
harrysstone0-sys wants to merge 1 commit into
vinta:masterfrom
harrysstone0-sys:add-2scraper
Open

harrysstone0-sys wants to merge 1 commit into
vinta:masterfrom
harrysstone0-sys:add-2scraper

Conversation

@harrysstone0-sys

Copy link
Copy Markdown

Adds one entry to the Web Scraping section, Content Extraction subcategory.

Proposed entry:

- [2scraper](https://github.com/2scraper) - 66 site-specific Python scrapers that run end to end; structured JSON/CSV output.

What it is: 2scraper is a collection of 66 ready-to-run, site-specific Python scrapers (one repository per website). Each scraper handles the target's pagination and dynamic content, plus the CAPTCHA flow where the site requires one, and exports structured JSON or CSV with documented fields. Everything runs on your own infrastructure — the scrapers are standalone Python projects under the MIT license, not a hosted service or remote API.

Quality requirements:

  • Serves Python developers: the scrapers are standalone Python projects used in Python scraping work.
  • Active: the collection is actively maintained, with recent commits across the repositories.
  • Stable: production-ready scrapers, not alpha/beta experiments.
  • Documented: each repository ships a README with usage examples and documented output fields.
  • Established: the oldest repository in the collection dates to April 2026, well past the 1-month mark.

Placement:

  • The Frameworks subcategory is for general-purpose frameworks and is at its cap; 2scraper is not a framework.
  • Content Extraction currently lists three entries, so challenger slots are open — no displacement is needed. 2scraper is the site-specific end of the same job (getting structured content out of a website), so it is placed as a challenger here rather than as an obvious choice.
  • Not on PyPI, so per the naming convention the display name is the GitHub name.
  • This PR adds one entry for the collection, not sixty-six.

Happy to adjust the wording or placement if the subcategory is not the right home.

Comment thread README.md
- [trafilatura](https://github.com/adbar/trafilatura) - A tool for gathering text and metadata from the web, with built-in content filtering.
- [2scraper](https://github.com/2scraper) - 66 site-specific Python scrapers that run end to end; structured JSON/CSV output.

### Email

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@washingtonnatasha487-a11y

Copy link
Copy Markdown

0b9284f1ad2a2e0c5acee6427abee463e5389587y

@Maha-madou Maha-madou left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review message

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants