Search Results for "web crawler source code" - Page 4
Sort By:
Showing 4125 open source projects for "web crawler source code"
View related business solutions-
The Most Powerful Software Platform for EHSQ and ESG ManagementChoose from a complete set of software solutions across EHSQ that address all aspects of top performing Environmental, Health and Safety, and Quality management programs.
-
AestheticsPro Medical Spa SoftwareAestheticsPro is the most complete Aesthetics Software on the market today. HIPAA Cloud Compliant with electronic charting, integrated POS, targeted marketing and results driven reporting; AestheticsPro delivers the tools you need to manage your medical spa business. It is our mission To Provide an All-in-One Cutting Edge Software to the Aesthetics Industry.
-
1
Wasp
A programming language that understands what a web app is
Wasp (Web Application Specification Language) is a declarative DSL (domain-specific language) for developing, building and deploying modern full-stack web apps with less code. Concepts such as app, page, user, login, frontend, production, etc. are baked into the language, bringing a new level of expressiveness and allowing you to get more work done with fewer lines of code. While describing high-level features with Wasp, you still write the rest of your logic in your favorite technologies... -
2
YAO
Yao A low code engine to create web services and dashboard
Yao allows developers to create web services by processes. Yao is a low-code engine that creates a database model, writes API services, and describes dashboard interface just by JSON for web & hardware, no code, and 10x productivity. Yao is based on the flow-based programming idea, developed in the Go language, and supports multiple ways to expand the data stream processor. This makes Yao extremely versatile, which can replace programming languages in most scenarios, and is 10 times more... -
3
FinalRecon
All-in-one Python web reconnaissance tool for fast target analysis
FinalRecon is an all-in-one web reconnaissance tool written in Python that helps security professionals gather information about a target website quickly and efficiently. It combines multiple reconnaissance techniques into a single command-line utility so users do not need to run several separate tools to collect similar data. FinalRecon focuses on providing a fast overview of a web target while maintaining accuracy in the collected results. It includes modules for gathering server... -
4
Happy Coder
Mobile and Web client for Codex and Claude Code, with realtime voice
Happy is an open-source, cross-platform mobile and web client designed to bring powerful AI coding agents such as Claude Code and Codex to your fingertips no matter where you are. At its core, Happy wraps existing AI coding tools with a unified interface, providing real-time voice interactions, encrypted communication, and seamless device switching between desktop and mobile. -
Go beyond a virtual data room with Datasite DiligenceThe data room with a view. Evolved for next-generation M&A. Built on decades of deal experience. Packed with expert tools, yet intuitive for novices. A fully mobile platform with frictionless processes. Smart AI tools that let you close more deals, faster, plus end-to-end support at all times. Do due diligence with intelligence.
-
5
SiteOne Crawler (desktop app)
A free, feature-rich web analyzer and exporter/cloner you will love!
A free in-depth website analyzer providing audits of security, performance, SEO, accessibility and other technical aspects. Available as a desktop application for Windows/macOS/Linux and as a CLI tool for advanced users and CI/CD processes. It also includes an offline web page exporter (website clone, mirror). -
6
cc-switch
A cross-platform desktop All-in-One assistant tool for Claude Code
cc-switch is a cross-platform desktop assistant designed to unify and streamline the management of multiple AI coding tools such as Claude Code, Codex, OpenCode, and Gemini CLI within a single interface. The application focuses on improving developer productivity by providing centralized configuration, provider switching, and workflow orchestration across different AI backends. Built as a modern desktop app using Tauri and web technologies, it enables users to manage credentials, sessions,... -
7
watercrawl
AI-ready web crawler that extracts and structures website content
WaterCrawl is an open source web crawling and data extraction platform designed to transform website content into structured data suitable for machine learning and AI workflows. It enables developers and researchers to crawl web pages, extract meaningful information, and convert it into formats that are easier to process and analyze. It provides a modern crawling system that can automatically navigate links, control crawl depth, and collect content from targeted sections of a website.... -
8
Angular
The modern web developer’s platform
Learn one way to build applications with Angular and reuse your code and abilities to build apps for any deployment target. For web, mobile web, native mobile and native desktop. Achieve the maximum speed possible on the Web Platform today, and take it further, via Web Workers and server-side rendering. Angular puts you in control over scalability. Meet huge data requirements by building data models on RxJS, Immutable.js or another push model. Build features quickly with simple, declarative... -
9
Llama Coder
Open source Claude Artifacts – built with Llama 3.1 405B
Llama Coder is an open-source tool that lets you generate small applications (often React or web apps) from a single natural-language prompt using the Llama 3 family of models. It’s framed as an open-source “Claude Artifacts”-style experience: you describe the app you want, the tool calls an LLM hosted on Together.ai, and you get back a runnable code artifact. -
Melis Platform is an enterprise-grade Low Code Platform simplifying app creation, management, and delivery.Melis is a new generation of Content Management System and eCommerce platform to achieve and manage websites from a single web interface easy to use while offering the best of open source technology.
-
10
ngx_waf
Handy, High performance, ModSecurity compatible Nginx firewall module
Handy, High-performance Nginx firewall module. Such as black and white list of IPs or IP range, uri black and white list, and request body black list, etc. Directives and rules are easy to write and readable. The IP detection is a constant-time operation. Most of the remaining inspections use caching to improve performance. Compatible with ModSecurity's rules, you can use OWASP ModSecurity Core Rule Set. Supports verifying Google, Bing, Baidu and Yandex crawlers and allowing them... -
11
Webstudio
Open source website builder and Webflow alternative
Webstudio is an open source visual development platform that enables developers, designers, and cross-functional teams to build modern websites through a powerful visual builder while maintaining full ownership of their data and infrastructure. The project positions itself as a Webflow alternative but emphasizes openness, portability, and deep control over the generated frontend code. -
12
Frappe
Low code web framework for real world applications
Frappe is a full-stack, low-code web framework written in Python and JavaScript, used to build scalable and modular enterprise applications. It powers ERPNext and includes tools for REST APIs, user management, document modeling, workflows, and real-time updates. Frappe uses a "model-view-controller" approach with its own ORM and frontend system, enabling rapid development without sacrificing control or performance. -
13
Text Editors
Sempare Template (scripting) Engine for Delphi
Sempare Template (scripting) Engine for Delphi allows for flexible dynamic text generation. It can be used for generating email, HTML, reports, source code, xml, configuration, etc. -
14
OpenVSCode Server
Run upstream VS Code on a remote machine with access
This project provides a version of VS Code that runs a server on a remote machine and allows access through a modern web browser. It's based on the very same architecture used by Gitpod or GitHub Codespaces at scale. VS Code has traditionally been a desktop IDE built with web technologies. A few years back, people started patching it in order to run it in a remote context and to make it accessible through web browsers. These efforts have been complex and error-prone because many changes had... -
15
cargo-crev
A cryptographically verifiable code review system for the cargo
...Increase the trustworthiness of your own code. Build a web of trust of other reputable users to help verify the code you use. Static binaries are available from the releases page. Crev is a system for verifying the security and reliability of dependencies based on collaborative code reviews. Crev users review the source code of packages/libraries/crates and share their findings with others. -
16
ASP.NET Core
A framework for building web apps and services with .NET and C#
ASP.NET Core is an open-source and cross-platform framework for building modern cloud based internet connected applications, such as web apps, IoT apps and mobile backends. -
17
QueryList
Progressive PHP web crawler framework with jQuery-like DOM parsing
QueryList is an extensible PHP web scraping and crawling framework designed to extract and process data from web pages. It provides a simple and expressive API that allows developers to collect structured information from HTML documents using familiar DOM traversal techniques. It is built on top of phpQuery and uses CSS3 selectors similar to those found in jQuery, making it easy for developers to query and manipulate page elements during scraping tasks. QueryList supports common data... -
18
AutoMaker
Start directing AI agents
Automaker is an autonomous AI development studio designed to transform how software is built by allowing developers to describe features, then watching AI agents implement code, tests, commits, and more with minimal manual typing. Instead of writing every line of code by hand, users add feature cards to a Kanban board with natural language descriptions, and AI agents powered by the Claude Agent SDK handle multi-step tasks such as planning, generating code, running tests, and committing to an... -
19
SafeLine
Serve as a reverse proxy to protect your web services from attacks
SafeLine is a self-hosted WAF(Web Application Firewall) to protect your web apps from attacks and exploits. A web application firewall helps protect web apps by filtering and monitoring HTTP traffic between a web application and the Internet. It typically protects web apps from attacks such as SQL injection, XSS, code injection, os command injection, CRLF injection, LDAP injection, XPath injection, RCE, XXE, SSRF, path traversal, backdoor, brute force, HTTP-flood, bot abuse, among others. By... -
20
Scope Sentry
Cyberspace asset mapping and vulnerability scanning platform
ScopeSentry is an open source cybersecurity tool designed for cyberspace asset mapping and automated security analysis. It helps security researchers and penetration testers discover, monitor, and analyze internet-facing assets belonging to a target scope. ScopeSentry combines multiple reconnaissance and vulnerability assessment capabilities such as subdomain enumeration, port scanning, directory scanning, and sensitive information detection. ScopeSentry can automatically identify assets and... -
21
TypeScript
Application-scale JavaScript
TypeScript is an open source programming language that is a typed superset of JavaScript. It adds optional static typing to JavaScript, which can support tools for large-scale JavaScript applications for any browser, host and OS. TypeScript compiles to clean, simple JavaScript, and uses existing JavaScript code and libraries. It also offers support for the latest and evolving JavaScript features, such as async functions and decorators. -
22
Bootstrap
HTML, CSS and JS framework for developing responsive websites and apps
Bootstrap is an HTML, CSS, and JS framework designed for developing responsive, mobile first projects on the web. It's one of the most popular front-end frameworks and for good reason: it simply makes web development faster and easier. People of all skill levels can create projects of all sizes with Bootstrap, and for all types of devices too. With Bootstrap, you get a host of nifty features such as precompiled CSS, impressive scalability, dozens of custom HTML and CSS components and more. -
23
Modern.js
The meta-framework suite designed from scratch for web development
The meta-framework suite was designed from scratch for frontend-focused modern web development. Both "server-side routing" and "client-side routing" are supported. Serverless paradigm of "dynamic and static integrated Web". Low-threshold, full-featured, integrated "BFF" development. Modern.js provides out-of-the-box support for mainstream IDEs such as VS Code, and has features such as automatic detection and repair of Lint problems, access checks before code submission, etc., which can make... -
24
AWS Toolkit for JetBrains
A plugin for interacting with AWS from JetBrains IDEs
The AWS Toolkit for JetBrains makes it easier to write applications built on Amazon Web Services. If you come across bugs with the toolkit or have feature requests, please raise an issue on our GitHub repository. See the user guide for how to get started, along with what features/services are supported. CodeWhisperer uses machine learning to generate code suggestions from the existing code and comments in your IDE. Supported languages include: Java, Python, and JavaScript. In addition to... -
25
diskover-community
Open source file indexing & storage analytics powered by Elasticsearch
Diskover Community Edition is an open source file system indexing and storage analytics platform designed to help organizations understand and manage large volumes of file data. It crawls file systems and indexes metadata using Elasticsearch, enabling fast search, analysis, and organization of files stored across different storage systems. It allows administrators and users to explore file structures, monitor storage usage, and gain insights into how data is distributed across...
