Search Results for "web crawler source code"
Sort By:
Showing 232 open source projects for "web crawler source code"
View related business solutions-
Simplify Purchasing For Your BusinessSimplify every aspect of buying for your business in Order.co. From sourcing products to scaling purchasing across locations to automating your AP and approvals workstreams, Order.co is the platform of choice for growing businesses.
-
Award-Winning Medical Office Software Designed for Your SpecialtyRXNT is an ambulatory healthcare technology pioneer that empowers medical practices and healthcare organizations to succeed and scale through innovative, data-backed, AI-powered software.
-
1
GPT Crawler
Crawl a site to generate knowledge files to create your own custom GPT
GPT Crawler is an open-source tool designed to automatically crawl websites and generate structured knowledge that can be used to build AI assistants and retrieval systems. It focuses on extracting high-quality textual content from web pages and preparing it in formats suitable for embedding, indexing, or fine-tuning workflows. The project is especially useful for teams that want to turn documentation sites or knowledge bases into conversational AI backends without building custom scrapers from scratch. ... -
2
Roo Code
Roo Code gives you a whole dev team of AI agents in your code editor
Roo Code is an AI-powered software engineering platform that works interactively in your IDE and autonomously in the cloud to help teams ship faster. It combines a powerful VS Code extension with cloud-based agents that can take on real development tasks across GitHub, Slack, and the web. Designed to work on your terms, Roo Code gives you full control locally while enabling delegation and parallel execution at scale. -
3
Every Code
Local AI coding agent CLI with multi-agent orchestration tools
Every Code (often referred to simply as Code) is a fast, local AI-powered coding agent designed to run directly in the terminal environment. It is a community-driven fork of the Codex CLI, with a strong emphasis on improving real-world developer ergonomics and workflows. Every Code enhances the traditional coding assistant model by introducing multi-agent orchestration, allowing multiple AI agents to collaborate, compare solutions, and refine outputs in parallel. It supports integration with... -
4
Visual Studio Code
Modern IDE and code editor from Microsoft for Mac, Windows, and Linux
Visual Studio Code combines the simplicity of a code editor with what developers need for their core edit-build-debug cycle. It provides comprehensive code editing, navigation, and understanding support along with lightweight debugging, a rich extensibility model, and lightweight integration with existing tools. Visual Studio Code is a distribution of the Code - OSS repository with Microsoft-specific customizations released under a traditional Microsoft product license. Visual Studio Code is... -
Skillfully - The future of skills based hiringSkillfully transforms hiring through AI-powered skill simulations that show you how candidates actually perform before you hire them. Our platform helps companies cut through AI-generated resumes and rehearsed interviews by validating real capabilities in action. Through dynamic job specific simulations and skill-based assessments, companies like Bloomberg and McKinsey have cut screening time by 50% while dramatically improving hire quality.
-
5
WebLLM
Bringing large-language models and chat to web browsers
WebLLM is a modular, customizable javascript package that directly brings language model chats directly onto web browsers with hardware acceleration. Everything runs inside the browser with no server support and is accelerated with WebGPU. We can bring a lot of fun opportunities to build AI assistants for everyone and enable privacy while enjoying GPU acceleration. WebLLM offers a minimalist and modular interface to access the chatbot in the browser. The WebLLM package itself does not come... -
6
Telegram Web A
Telegram Web A, GPL v3
Telegram Web A is a full‐featured web client implementation for Telegram, built from scratch as a lightweight, modern single-page application. It uses a custom front-end framework (called “Teact”) that re-implements React-style paradigms and pairs them with a custom version of the MTProto library (based on GramJS) to interact with Telegram’s backend infrastructure. The project achieved recognition (winning first prize in the Telegram Lightweight Client Contest) and serves as the code base... -
7
Proton Web Clients
Monorepo hosting the proton web clients
Proton Web Clients is a monorepo hosting the web applications for Proton’s suite of privacy-focused services, including the core Proton Mail webmail interface and related web apps like Proton Calendar, Proton Drive, Proton Account, Proton VPN, Proton Pass, and other connected tools. It consolidates all web client code, shared modules, dependencies, and development tooling into a single repository, enabling unified maintenance, consistency of design patterns, and efficient evolution of... -
8
douyin
Open source Douyin crawler for collecting and downloading public data
DouyinCrawler is an open source data collection tool designed to gather publicly available information from the Douyin platform. It demonstrates how to build a Python-based web crawler combined with a graphical interface and command line functionality. It allows users to collect data from various types of Douyin content, including user profiles, videos, hashtags, and music pages. -
9
X-Crawl
Flexible Node.js AI-assisted crawler library
A high-performance web crawling and scraping framework for Node.js, designed for large-scale data extraction. -
AestheticsPro Medical Spa SoftwareAestheticsPro is the most complete Aesthetics Software on the market today. HIPAA Cloud Compliant with electronic charting, integrated POS, targeted marketing and results driven reporting; AestheticsPro delivers the tools you need to manage your medical spa business. It is our mission To Provide an All-in-One Cutting Edge Software to the Aesthetics Industry.
-
10
AWS Toolkit for Visual Studio Code
Local Lambda debug, CodeWhisperer, SAM/CFN syntax, etc.
The AWS Toolkit extension for Visual Studio Code enables you to interact with Amazon Web Services (AWS). Try the AWS Code Sample Catalog to start coding with the AWS SDK. The AWS Explorer provides access to the AWS services that you can work with when using the Toolkit. To see the AWS Explorer, choose the AWS icon in the Activity bar. The Developer Tools panel is a section for developer-focused tooling curated for working in an IDE. The Developer Tools panel can be found underneath the AWS... -
11
Fluent UI Web
Collection of utilities andcomponents for building web applications
A collection of UX frameworks for creating beautiful, cross-platform apps that share code, design, and interaction behavior. Build for one platform or for all. Everything you need is here. Build your own apps using the same open source components we do, with accessibility, internationalization, and performance included. From tutorials to a fun collection of API references, find what you need to design and develop your own Fluent experience. -
12
watercrawl
AI-ready web crawler that extracts and structures website content
WaterCrawl is an open source web crawling and data extraction platform designed to transform website content into structured data suitable for machine learning and AI workflows. It enables developers and researchers to crawl web pages, extract meaningful information, and convert it into formats that are easier to process and analyze. It provides a modern crawling system that can automatically navigate links, control crawl depth, and collect content from targeted sections of a website.... -
13
OpenVSCode Server
Run upstream VS Code on a remote machine with access
This project provides a version of VS Code that runs a server on a remote machine and allows access through a modern web browser. It's based on the very same architecture used by Gitpod or GitHub Codespaces at scale. VS Code has traditionally been a desktop IDE built with web technologies. A few years back, people started patching it in order to run it in a remote context and to make it accessible through web browsers. These efforts have been complex and error-prone because many changes had... -
14
Webstudio
Open source website builder and Webflow alternative
Webstudio is an open source visual development platform that enables developers, designers, and cross-functional teams to build modern websites through a powerful visual builder while maintaining full ownership of their data and infrastructure. The project positions itself as a Webflow alternative but emphasizes openness, portability, and deep control over the generated frontend code. -
15
Modern.js
The meta-framework suite designed from scratch for web development
The meta-framework suite was designed from scratch for frontend-focused modern web development. Both "server-side routing" and "client-side routing" are supported. Serverless paradigm of "dynamic and static integrated Web". Low-threshold, full-featured, integrated "BFF" development. Modern.js provides out-of-the-box support for mainstream IDEs such as VS Code, and has features such as automatic detection and repair of Lint problems, access checks before code submission, etc., which can make... -
16
Happy Coder
Mobile and Web client for Codex and Claude Code, with realtime voice
Happy is an open-source, cross-platform mobile and web client designed to bring powerful AI coding agents such as Claude Code and Codex to your fingertips no matter where you are. At its core, Happy wraps existing AI coding tools with a unified interface, providing real-time voice interactions, encrypted communication, and seamless device switching between desktop and mobile. -
17
Fragments by E2B
Open source template for AI-powered code generation apps w/ sandboxes
Fragments is an open source template designed for building applications where artificial intelligence generates and executes code directly from natural language prompts. It provides a ready-to-use web interface where users can chat with AI models to create code snippets, applications, or scripts in various programming environments. Generated code can be executed safely in isolated sandbox environments using the E2B SDK, which helps protect the host system while running AI-produced programs. ... -
18
Llama Coder
Open source Claude Artifacts – built with Llama 3.1 405B
Llama Coder is an open-source tool that lets you generate small applications (often React or web apps) from a single natural-language prompt using the Llama 3 family of models. It’s framed as an open-source “Claude Artifacts”-style experience: you describe the app you want, the tool calls an LLM hosted on Together.ai, and you get back a runnable code artifact. -
19
Angular
The modern web developer’s platform
Learn one way to build applications with Angular and reuse your code and abilities to build apps for any deployment target. For web, mobile web, native mobile and native desktop. Achieve the maximum speed possible on the Web Platform today, and take it further, via Web Workers and server-side rendering. Angular puts you in control over scalability. Meet huge data requirements by building data models on RxJS, Immutable.js or another push model. Build features quickly with simple, declarative... -
20
AutoMaker
Start directing AI agents
Automaker is an autonomous AI development studio designed to transform how software is built by allowing developers to describe features, then watching AI agents implement code, tests, commits, and more with minimal manual typing. Instead of writing every line of code by hand, users add feature cards to a Kanban board with natural language descriptions, and AI agents powered by the Claude Agent SDK handle multi-step tasks such as planning, generating code, running tests, and committing to an... -
21
JavaScript Obfuscator
A powerful obfuscator for JavaScript and Node.js
JavaScript Obfuscator is a Node.js library and CLI that transforms readable JavaScript into hardened, difficult-to-reverse code. It applies techniques such as identifier mangling, string array extraction/encoding, control-flow flattening, dead-code injection, and numeric literal transformations to disguise intent. Advanced options include self-defending code, domain locking, debug/console protection, and property key transformation, allowing you to tailor defenses to your threat model. The... -
22
Brokk
Brokk brings code intelligence to AI
Brokk is a code intelligence assistant framework designed to let large language models (LLMs) understand code semantically (not just as raw text) so that they can work effectively on large codebases that don’t fit wholly in a prompt context. It helps bridge the gap between LLMs and real-world engineering code by offering tooling to index, analyze, query, and augment code context, so that AI can meaningfully reason about existing code, suggest edits, and navigate across projects. Modular... -
23
Pascal Editor
A 3D building editor built with React Three Fiber and WebGPU
Editor is a lightweight, browser-based code editing environment designed to provide a fast and minimal interface for writing and testing code directly in the browser. The project focuses on simplicity and performance, offering a distraction-free workspace that allows developers to quickly prototype ideas without the overhead of a full development environment. It leverages modern web technologies to deliver a responsive and interactive experience, making it suitable for quick experiments,... -
24
Vercel
Optimal workflow for frontend teams
Vercel combines the best developer experience with an obsessive focus on end-user performance. Our platform enables frontend teams to do their best work. Developers love Next.js, the open source React framework Vercel built together with Google and Facebook. Next.js powers the biggest websites like Twilio, for use cases in e-commerce, travel, news, and marketing. Vercel is the best place to deploy any frontend app. Start by deploying with zero configuration to our global edge network. Scale... -
25
Srcbook
TypeScript-centric app development platform
Srcbook is an open-source, local-first development environment that blends a TypeScript notebook with an AI-assisted app builder, giving developers a flexible space to prototype, explore, and build full web applications with AI support and real execution. It runs locally through a CLI tool backed by Node.js, launching a web interface where users can write and execute TypeScript code interactively, visualize results, and iterate quickly on ideas without leaving the browser. ...
