Python · Web Scraping

Web Scraping

• browser-use/browser-use on GitHub, opens in a new tab

  109.5Kstars

  Make websites accessible for AI agents with easy browser automation.

  Python·MIT License·active·21 hours ago

• unclecode/crawl4ai on GitHub, opens in a new tab

  78.5Kstars

  An open-source, LLM-friendly web crawler that provides lightning-fast, structured data extraction specifically designed for AI agents.

  Python·Apache License 2.0·active·7 hours ago

• scrapy/scrapy on GitHub, opens in a new tab

  63.9Kstars

  A fast high-level screen scraping and web crawling framework.

  Python·BSD 3-Clause "New" or "Revised" License·active·4 hours ago

• adbar/trafilatura on GitHub, opens in a new tab

  6.6Kstars

  A tool for gathering text and metadata from the web, with built-in content filtering.

  Python·Apache License 2.0·active·2 days ago

• kurtmckee/feedparser on GitHub, opens in a new tab

  2.4Kstars

  Universal feed parser.

  Python·Other·active·2 weeks ago

• Alir3z4/html2text on GitHub, opens in a new tab

  2.2Kstars

  Convert HTML to Markdown-formatted text.

  Python·GNU General Public License v3.0·steady·10 months ago