Web Crawling

Libraries to automate web scraping.

scrapy53.8K

A fast high-level screen scraping and web crawling framework.

mechanicalsoup4.7K

A Python library for automating interaction with websites.

grab2.4K

Site scraping framework.

feedparser2K

Universal feed parser.