This library intends to make parsing HTML (e.g. scraping the web) as simple and intuitive as possible. When using this library you automatically get full JavaScript support! (Using Chromium, thanks to puppeteer) CSS Selectors (a.k.a jQuery-style, thanks to PyQuery). XPath Selectors, for the faint of heart. Mocked user-agent (like a real web browser). Automatic following of redirects. Connection–pooling and cookie persistence. The Requests experience you know and love, with magical parsing abilities, and async support. The rest of the code operates the same way as the synchronous version except that results is a list containing multiple response objects however the same basic processes can be applied as above to extract the data you want.

Features

  • Full JavaScript support! (Using Chromium, thanks to pyppeteer)
  • CSS Selectors (a.k.a jQuery-style, thanks to PyQuery)
  • XPath Selectors, for the faint of heart
  • Mocked user-agent (like a real web browser)
  • Connection–pooling and cookie persistence
  • Automatic following of redirects

Project Samples

Project Activity

See All Activity >

Categories

Web Scrapers

License

MIT License

Follow Requests-HTML

Requests-HTML Web Site

Other Useful Business Software
Skillfully - The future of skills based hiring Icon
Skillfully - The future of skills based hiring

Realistic Workplace Simulations that Show Applicant Skills in Action

Skillfully transforms hiring through AI-powered skill simulations that show you how candidates actually perform before you hire them. Our platform helps companies cut through AI-generated resumes and rehearsed interviews by validating real capabilities in action. Through dynamic job specific simulations and skill-based assessments, companies like Bloomberg and McKinsey have cut screening time by 50% while dramatically improving hire quality.
Learn More
Rate This Project
Login To Rate This Project

User Reviews

Be the first to post a review of Requests-HTML!

Additional Project Details

Operating Systems

Windows

Programming Language

Python

Related Categories

Python Web Scrapers

Registered

2023-04-10