Showing 688 open source projects for "python web crawler"

View related business solutions
  • The Most Powerful Software Platform for EHSQ and ESG Management Icon
    The Most Powerful Software Platform for EHSQ and ESG Management

    Addresses the needs of small businesses and large global organizations with thousands of users in multiple locations.

    Choose from a complete set of software solutions across EHSQ that address all aspects of top performing Environmental, Health and Safety, and Quality management programs.
    Learn More
  • Collect! is a highly configurable debt collection software Icon
    Collect! is a highly configurable debt collection software

    Everything that matters to debt collection, all in one solution.

    The flexible & scalable debt collection software built to automate your workflow. From startup to enterprise, we have the solution for you.
    Learn More
  • 1
    BlackSheep

    BlackSheep

    Fast ASGI web framework for Python

    BlackSheep is an asynchronous web framework to build event-based web applications with Python. A rich code API, based on dependency injection and inspired by Flask and ASP.NET Core. A typing-friendly codebase, which enables a comfortable development experience thanks to hints when coding with IDEs. Built-in generation of OpenAPI Documentation, supporting version 3, YAML, and JSON.
    Downloads: 5 This Week
    Last Update:
    See Project
  • 2
    Responder

    Responder

    A familiar HTTP Service Framework for Python

    ...Class-based views without inheritance. ASGI framework, the future of Python web services. WebSocket support! The ability to mount any ASGI / WSGI app at a subroute. f-string syntax route declaration. Mutable response object passed into each view. No need to return anything. Background tasks spawned off in a ThreadPoolExecutor. GraphQL (with GraphiQL) support! OpenAPI schema generation, with interactive documentation!
    Downloads: 5 This Week
    Last Update:
    See Project
  • 3
    Frappe

    Frappe

    Low code web framework for real world applications

    Frappe is a full-stack, low-code web framework written in Python and JavaScript, used to build scalable and modular enterprise applications. It powers ERPNext and includes tools for REST APIs, user management, document modeling, workflows, and real-time updates. Frappe uses a "model-view-controller" approach with its own ORM and frontend system, enabling rapid development without sacrificing control or performance.
    Downloads: 21 This Week
    Last Update:
    See Project
  • 4
    Locust

    Locust

    Scalable open source load testing tool

    Locust is an open source user load testing tool written in Python. The idea behind Locust is to swarm your web site or other systems with attacks from simulated users during a test, with each user behavior defined by you using Python code. This swarming process is then monitored from a web UI in real-time, and will help identify any bottlenecks in your code before real users can come in. As it is completely event-based, Locust can have thousands or even millions of simultaneous users distributed over multiple machines swarming your system. ...
    Downloads: 13 This Week
    Last Update:
    See Project
  • Premier Construction Software Icon
    Premier Construction Software

    Premier is a global leader in financial construction ERP software.

    Rated #1 Construction Accounting Software by Forbes Advisor in 2022 & 2023. Our modern SAAS solution is designed to meet the needs of General Contractors, Developers/Owners, Homebuilders & Specialty Contractors.
    Learn More
  • 5
    Streamlit

    Streamlit

    The fastest way to build data apps in Python

    ...Effortlessly share, manage and deploy your apps, directly from Streamlit. Streamlit lets you turn data scripts into sharable web apps in minutes, not weeks. It’s all Python, open-source, and free! And once you’ve created an app you can use our cloud platform to deploy, manage, and share your app! Streamlit can also be installed in a virtual environment on Windows, Mac, and Linux.
    Downloads: 48 This Week
    Last Update:
    See Project
  • 6
    AWS Lambda Python Runtime Interf Client

    AWS Lambda Python Runtime Interf Client

    Seamlessly extend your preferred base images to be Lambda compatible

    ...The Lambda Runtime Interface Client is a lightweight interface that allows your runtime to receive requests from and send requests to the Lambda service. The Lambda Python Runtime Interface Client is vended through pip. You can include this package in your preferred base image to make that base image Lambda compatible. To make it easy to locally test Lambda functions packaged as container images we open-sourced a lightweight web-server, Lambda Runtime Interface Emulator (RIE), which allows your function packaged as a container image to accept HTTP requests. ...
    Downloads: 0 This Week
    Last Update:
    See Project
  • 7
    web2py

    web2py

    Free and open source full-stack enterprise framework

    Free open source full-stack framework for rapid development of fast, scalable, secure and portable database-driven web-based applications. Written and programmable in Python (version 3 and 2.7). Everything you need in one package including fast multi-threaded web server, SQL database and web-based interface. No third-party dependencies but works with third-party tools. Create, modify, deploy and manage applications from anywhere using your browser.
    Downloads: 4 This Week
    Last Update:
    See Project
  • 8
    I hate money

    I hate money

    A simple shared budget manager web application

    I hate money is a web application made to ease shared budget management. It keeps track of who bought what, when, and for whom; and helps to settle the bills. I hate money is written in python, using the flask framework. It’s developed with ease of use in mind and is trying to keep things simple. Hope you (will) like it! The code is distributed under a BSD beerware derivative: if you meet the people in person and you want to pay them a craft beer, you are highly encouraged to do so.
    Downloads: 6 This Week
    Last Update:
    See Project
  • 9
    Django REST framework

    Django REST framework

    Powerful and flexible toolkit for building Web APIs

    Django REST framework is a powerful and flexible toolkit for building Web APIs. Some reasons you might want to use REST framework: The Web browsable API is a huge usability win for your developers. Authentication policies including packages for OAuth1a and OAuth2. Serialization that supports both ORM and non-ORM data sources. Customizable all the way down - just use regular function-based views if you don't need the more powerful features. Extensive documentation, and great community...
    Downloads: 109 This Week
    Last Update:
    See Project
  • Turn traffic into pipeline and prospects into customers Icon
    Turn traffic into pipeline and prospects into customers

    For account executives and sales engineers looking for a solution to manage their insights and sales data

    Docket is an AI-powered sales enablement platform designed to unify go-to-market (GTM) data through its proprietary Sales Knowledge Lake™ and activate it with intelligent AI agents. The platform helps marketing teams increase pipeline generation by 15% by engaging website visitors in human-like conversations and qualifying leads. For sales teams, Docket improves seller efficiency by 33% by providing instant product knowledge, retrieving collateral, and creating personalized documents. Built for GTM teams, Docket integrates with over 100 tools across the revenue tech stack and offers enterprise-grade security with SOC 2 Type II, GDPR, and ISO 27001 compliance. Customers report improved win rates, shorter sales cycles, and dramatically reduced response times. Docket’s scalable, accurate, and fast AI agents deliver reliable answers with confidence scores, empowering teams to close deals faster.
    Learn More
  • 10
    Reflex Dev

    Reflex Dev

    Web apps in pure Python

    Reflex is a Python framework for building full-stack web apps entirely in Python—without writing JavaScript for the frontend. It provides fast live reloads, built-in state management, deployment tooling, and optional AI-powered scaffolding to accelerate development.
    Downloads: 4 This Week
    Last Update:
    See Project
  • 11
    DrissionPage

    DrissionPage

    Python based web automation tool. Powerful and elegant

    DrissionPage is a Python-based automation framework that blends the capabilities of Selenium for browser automation with Requests-HTML for fast, headless web data extraction. It enables seamless switching between browser-controlled and headless HTTP sessions within the same interface. Ideal for web scraping, testing, and automation, DrissionPage is lightweight and highly efficient, offering more flexibility than standard Selenium or Requests usage alone.
    Downloads: 5 This Week
    Last Update:
    See Project
  • 12
    CadQuery

    CadQuery

    A python parametric CAD scripting framework based on OCCT

    ...Provide a non-proprietary, plain text model format that can be edited and executed with only a web browser. The scripts use a standard programming language, Python, and thus can benefit from the associated infrastructure. This includes many standard libraries and IDEs. CadQuery's CAD kernel Open CASCADE Technology (OCCT) is much more powerful than the CGAL used by OpenSCAD. Features supported natively by OCCT include NURBS, splines, surface sewing, STL repair, STEP import/export, and other complex operations.
    Downloads: 69 This Week
    Last Update:
    See Project
  • 13
    Scrapy

    Scrapy

    A fast, high-level web crawling and web scraping framework

    Scrapy is a fast, open source, high-level framework for crawling websites and extracting structured data from these websites. Portable and written in Python, it can run on Windows, Linux, macOS and BSD. Scrapy is powerful, fast and simple, and also easily extensible. Simply write the rules to extract the data, and add new functionality if you wish without having to touch the core. Scrapy does the rest, and can be used in a number of applications. It can be used for data mining, monitoring...
    Downloads: 16 This Week
    Last Update:
    See Project
  • 14
    liquidpy

    liquidpy

    A port of liquid template engine for python

    A port of liquid template engine for Python, on the shoulder of jinja2.
    Downloads: 0 This Week
    Last Update:
    See Project
  • 15
    Bottle

    Bottle

    bottle.py is a fast and simple micro-framework for python applications

    Bottle is a minimalist web framework for building small web applications and APIs in Python. It is distributed as a single file with no external dependencies, making it perfect for rapid development, prototyping, or embedded use. Despite its small size, Bottle supports routing, templates, request handling, and plugin support, offering a full-featured toolkit in an extremely compact package.
    Downloads: 4 This Week
    Last Update:
    See Project
  • 16
    Eel

    Eel

    A Python library for making simple Electron-like HTML/JS GUI apps

    ...If you are familiar with Python and web development, probably just jump to this example which picks random file names out of the given folder (something that is impossible from a browser). There are several options for making GUI apps in Python, but if you want to use HTML/JS (in order to use jQueryUI or Bootstrap, for example) then you generally have to write a lot of boilerplate code to communicate from the Client (Javascript) side to the Server (Python) side.
    Downloads: 4 This Week
    Last Update:
    See Project
  • 17
    Django

    Django

    The Web framework for perfectionists with deadlines

    Django is a high-level, free and open-source Python web framework founded on the Model–Template–View (MTV) pattern, designed to facilitate rapid development of secure, maintainable, and scalable database-driven websites. First, read docs/intro/install.txt for instructions on installing Django. Next, work through the tutorials in order (docs/intro/tutorial01.txt, docs/intro/tutorial02.txt, etc.).
    Downloads: 22 This Week
    Last Update:
    See Project
  • 18
    PyMySQL

    PyMySQL

    MySQL client library for Python

    PyMySQL is a 100% Python implementation of the MySQL client protocol, allowing Python applications to connect to MySQL and MariaDB databases without requiring binary extensions. It supports standard DB‑API 2.0 features, such as cursors, transactions, and parameterized queries. PyMySQL is versatile for web applications, scripts, and tools, offering compatibility with ORMs like SQLAlchemy and frameworks like Django.
    Downloads: 6 This Week
    Last Update:
    See Project
  • 19
    Dash

    Dash

    Build beautiful web-based analytic apps, no JavaScript required

    Dash is a Python framework for building beautiful analytical web applications without any JavaScript. Built on top of Plotly.js, React and Flask, Dash easily achieves what an entire team of designers and engineers normally would. It ties modern UI controls and displays such as dropdown menus, sliders and graphs directly to your analytical Python code, and creates exceptional, interactive analytics apps.
    Downloads: 5 This Week
    Last Update:
    See Project
  • 20
    Sanic

    Sanic

    Async Python 3.6+ web server/framework

    Build fast, run fast with Sanic! Sanic is a Python 3.6+ web server and web framework designed to go fast. It provides a way to get a highly performant HTTP server up and running fast, while also making it easy to build, expand, and eventually scale. Sanic aspires to be as simple as possible while delivering the performance that you require. It allows the usage of the async/await syntax added in Python 3.5, so your code is guaranteed to be non-blocking and speedy. ...
    Downloads: 0 This Week
    Last Update:
    See Project
  • 21
    Rio

    Rio

    WebApps in pure Python. No JavaScript, HTML and CSS needed

    Rio is a Python framework designed to build web applications without the need for HTML, CSS, or JavaScript. Inspired by frameworks like Flutter and React, Rio offers a declarative interface and reusable components, enabling developers to create dynamic web apps entirely in Python. It streamlines the development process by managing both frontend and backend seamlessly.
    Downloads: 0 This Week
    Last Update:
    See Project
  • 22
    Parsera

    Parsera

    Lightweight library for scraping web-sites with LLMs

    Scrape data from any website with only a link and column descriptions. Parsera is a tool designed to scrape web content, specifically handling poorly structured or messy websites.
    Downloads: 7 This Week
    Last Update:
    See Project
  • 23
    miniblink49

    miniblink49

    Lighter, faster browser kernel of blink to integrate HTML UI in apps

    ...After turning off the cross-domain switch, you can use various cross-domain functions (support cross-domain). Headless mode, which greatly saves resources and is suitable for crawlers (headless mode, be suitable for Web Crawler).
    Downloads: 10 This Week
    Last Update:
    See Project
  • 24
    SiteOne Crawler (desktop app)

    SiteOne Crawler (desktop app)

    A free, feature-rich web analyzer and exporter/cloner you will love!

    A free in-depth website analyzer providing audits of security, performance, SEO, accessibility and other technical aspects. Available as a desktop application for Windows/macOS/Linux and as a CLI tool for advanced users and CI/CD processes. It also includes an offline web page exporter (website clone, mirror).
    Downloads: 3 This Week
    Last Update:
    See Project
  • 25
    Stlite

    Stlite

    n-browser Streamlit

    Stlite is a WebAssembly-powered framework that enables Streamlit applications to run entirely in the browser without requiring a Python backend server. It achieves this by using Pyodide, a WebAssembly-based Python runtime, to execute Python code directly within the browser environment. This allows developers to build interactive data applications using familiar Streamlit APIs while eliminating the need for server-side infrastructure. Stlite supports a wide range of Python libraries and enables real-time interactivity, making it suitable for dashboards, data visualization tools, and educational applications. ...
    Downloads: 4 This Week
    Last Update:
    See Project
MongoDB Logo MongoDB