镜像站点 · 本页由第三方 GitHub 只读镜像提供，非 GitHub 官方站点，不接受任何登录或凭据输入。前往 github.com
Skip to content
@scrapy

Scrapy project

An open source and collaborative framework for extracting the data you need from websites. In a fast, simple, yet extensible way.

Pinned Loading

  1. scrapy scrapy Public

    Scrapy, a fast high-level web crawling & scraping framework for Python.

    Python 64.8k 12k

  2. scrapyd scrapyd Public

    A service daemon to run Scrapy spiders

    Python 3.1k 582

  3. parsel parsel Public

    Parsel lets you extract data from XML/HTML documents using XPath or CSS selectors

    Python 1.4k 178

  4. w3lib w3lib Public

    Python library of web-related functions

    HTML 418 131

  5. protego protego Public

    A pure-Python robots.txt parser with support for modern conventions.

    DIGITAL Command Language 94 37

  6. itemadapter itemadapter Public

    Common interface for data container classes

    Python 69 19

Repositories

Showing 10 of 35 repositories
  • itemadapter Public

    Common interface for data container classes

    scrapy/itemadapter's past year of commit activity
    Python 69 BSD-3-Clause 19 3 5 Updated Oct 14, 2026
  • scrapy Public

    Scrapy, a fast high-level web crawling & scraping framework for Python.

    scrapy/scrapy's past year of commit activity
    Python 64,812 BSD-3-Clause 12,001 131 (5 issues need help) 115 Updated Oct 14, 2026
  • cssselect Public

    CSS Selectors for Python

    scrapy/cssselect's past year of commit activity
    Python 312 72 5 3 Updated Oct 14, 2026
  • w3lib Public

    Python library of web-related functions

    scrapy/w3lib's past year of commit activity
    HTML 418 BSD-3-Clause 131 2 3 Updated Oct 14, 2026
  • sphinx-scrapy Public

    Sphinx extension for documentation in the Scrapy ecosystem

    scrapy/sphinx-scrapy's past year of commit activity
    Python 3 BSD-3-Clause 3 0 0 Updated Oct 14, 2026
  • parsel Public

    Parsel lets you extract data from XML/HTML documents using XPath or CSS selectors

    scrapy/parsel's past year of commit activity
    Python 1,362 BSD-3-Clause 177 8 9 Updated Oct 13, 2026
  • formerly Public

    Deprecate a class that users inherit from

    scrapy/formerly's past year of commit activity
    Python 2 BSD-3-Clause 1 0 0 Updated Oct 13, 2026
  • frostwork Public
    scrapy/frostwork's past year of commit activity
    Python 14 Apache-2.0 1 0 1 Updated Oct 10, 2026
  • protego Public

    A pure-Python robots.txt parser with support for modern conventions.

    scrapy/protego's past year of commit activity
    DIGITAL Command Language 94 BSD-3-Clause 37 0 1 Updated Oct 8, 2026
  • scrapy-lint Public

    A linter for Scrapy projects.

    scrapy/scrapy-lint's past year of commit activity
    Python 22 MIT 8 19 22 Updated Oct 8, 2026
View all repositories