Search Results for "web crawler source code" - Page 4
Sort By:
Showing 373 open source projects for "web crawler source code"
View related business solutions-
Skillfully - The future of skills based hiringSkillfully transforms hiring through AI-powered skill simulations that show you how candidates actually perform before you hire them. Our platform helps companies cut through AI-generated resumes and rehearsed interviews by validating real capabilities in action. Through dynamic job specific simulations and skill-based assessments, companies like Bloomberg and McKinsey have cut screening time by 50% while dramatically improving hire quality.
-
Network Discovery Software | JDisc DiscoveryJDisc Discovery is a comprehensive network inventory and IT asset management solution designed to help organizations gain clear, up-to-date visibility into their IT environment. It automatically scans and maps devices across the network, including servers, workstations, virtual machines, and network hardware, to create a detailed inventory of all connected assets. This includes critical information such as hardware configurations, software installations, patch levels, and relationshipots between devices.
-
1
AWX
A web-based user interface built on top of Ansible
AWX provides a web-based user interface, REST API, and task engine built on top of Ansible. It is one of the upstream projects for Red Hat Ansible Automation Platform. Starting in version 18.0, the AWX Operator is the preferred way to install AWX. AWX can also alternatively be installed and run in Docker, but this install path is only recommended for development/test-oriented deployments, and has no official published release. Uses naming and structure consistent with the AWX HTTP API.... -
2
Spark TTS
Spark-TTS Inference Code
Spark TTS is an open-source, PyTorch-based text-to-speech inference system that leverages large language models to produce highly natural, intelligible speech from text input. It uses an efficient single-stream architecture where speech tokens are directly reconstructed from the predictions of an LLM, removing the need for external acoustic models or complex vocoders and making the generation pipeline cleaner and faster. The project supports zero-shot voice cloning, meaning it can imitate a... -
3
gpt-oss
gpt-oss-120b and gpt-oss-20b are two open-weight language models
gpt-oss is OpenAI’s open-weight family of large language models designed for powerful reasoning, agentic workflows, and versatile developer use cases. The series includes two main models: gpt-oss-120b, a 117-billion parameter model optimized for general-purpose, high-reasoning tasks that can run on a single H100 GPU, and gpt-oss-20b, a lighter 21-billion parameter model ideal for low-latency or specialized applications on smaller hardware. Both models use a native MXFP4 quantization for... -
4
AWS Agent Skills
AWS Skills for Agents
AWS Agent Skills is a repository that curates AWS-focused agent skills — capability modules that give AI assistants like Claude Code and Codex deep, practical knowledge across key Amazon Web Services domains. Instead of streaming giant documentation sets or relying on episodic web search, this project compresses AWS best practices, usage patterns, edge cases, and real-world engineering guides into pre-structured skill definitions that are token-efficient and tailored for reasoning. The... -
Intelligent Automation Solutions Built for Modern Finance TeamsDigitally transform your business with workflow automation and integrated payment solutions. Digitally store and secure your data with advanced search and accessibility features that keeps your documents at the tip of your team’s fingers.
-
5
UI UX Pro Max
AI SKILL that provide design intelligence
UI UX Pro Max is an open-source AI skill designed to provide intelligent design assistance for professional user interfaces and user experiences across web, mobile, and cross-platform frameworks. It uses an AI reasoning engine to generate complete design systems tailored to project requirements, recommending layouts, typography, colors, spacing, and component structures automatically based on natural language prompts. -
6
DeepWiki Open
AI-Powered Wiki Generator for GitHub/Gitlab/Bitbucket Repositories
DeepWiki Open is an open-source, AI-powered wiki generator that automatically creates fully navigable, richly structured wiki documentation for GitHub, GitLab, or Bitbucket repositories by combining code analysis, vector embeddings, retrieval-augmented generation (RAG), and visualization tools. Users can enter a repository URL and the system will clone the project, build semantic embeddings of its codebase, extract architecture and relationships, generate human-readable documentation, and produce visual diagrams to help explain complex code structure. ... -
7
HASS Configurator
Configuration UI for Home Assistant
The HASS Configurator is a small web app (you access it via a web browser) that provides a filesystem browser and text-editor to modify files on the machine the configurator is running on. It has been created to allow easy configuration of Home Assistant. It is powered by Ace editor, which supports syntax highlighting for various code/markup languages. YAML files (the default language for Home Assistant configuration files) will be automatically checked for syntax errors while editing. The... -
8
LinkChecker
Check links in web documents or full websites
LinkChecker is a free, GPL licensed website validator. LinkChecker checks links in web documents or full websites. It runs on Python 3 systems, requiring Python 3.8 or later. The version in the pip repository may be old, to find out how to get the latest code, plus platform-specific information and other advice see doc/install.txt in the source code archive. If you do not want to install any additional libraries/dependencies you can use the Docker image which is published on GitHub Packages. -
9
TurboGears
Python web framework with full-stack layer
TurboGears is a hybrid web framework able to act both as a Full Stack framework or as a Microframework. TurboGears helps you get going fast and gets out of your way when you want it! TurboGears can be used both as a full stack framework or as a microframework in single-file mode. TurboGears 2 is built on top of the experience of several next-generation web frameworks including TurboGears 1 (of course), Django, and Rails. All of these frameworks had limitations that frustrated us, and TG2 was... -
Share your screen instantly while on a phone call with CrankWheel for an engaging presentation.Our 10x simpler screen sharing tool is designed for you if you spend your days on the phone with clients, and need to add a visual presentation to close sales. No more scheduling a follow-up meeting, or teaching them to use a complex tool. Send them a text message or email, and they see your screen in seconds.
-
10
dynaconf
Configuration Management for Python
Inspired by the 12-factor application guide. Settings management (default values, validation, parsing, templating). Protection of sensitive information (passwords/tokens). Multiple file formats toml|yaml|json|ini|py and also customizable loaders. Full support for environment variables to override existing settings (dotenv support included). Optional layered system for multi environments [default, development, testing, production] (also called multi profiles). Built-in support for Hashicorp... -
11
REST APIs with Flask and Python
Projects and e-book for our course, REST APIs with Flask and Python
A full course to teach you how to use Flask and Python to make REST APIs using multiple Flask extensions and PostgreSQL. Learn Flask, Docker, PostgreSQL, and more. Build professional-grade REST APIs with Python. No more outdated tutorials. Use Python 3.10+ and the latest versions of every Flask extension and library. Run your apps in Docker, host your code with Git, write documentation with Swagger, and test your APIs while developing. Learn how to perform user authentication using JWTs and... -
12
changedetection.io
The best free open source website change detection and restock service
Loved by smart shoppers, data journalists, research engineers, data scientists, security researchers, and more. From simply monitoring website pages that have a change (such as watching prices, and restocking notifications), to deep inspection such as PDF text support, JSON and XML monitoring, and extensive text triggers. Monitor out-of-stock products and get alerts when those products are back in stock, get restock alerts via Discord, Slack, email, and many other platforms. Using the... -
13
OpenHands
Open-source autonomous AI software engineer
...So we're building all our agents in the open on GitHub, under the MIT license. Our agents can do anything a human developer can: they write code, run commands, and use the web. We're partnering with AI safety experts like Invariant Labs to balance innovation with security. -
14
AgenticSeek
Fully Local Manus AI. No APIs, No $200 monthly bills
AgenticSeek is a fully local autonomous AI assistant designed as a privacy-focused alternative to cloud-based agent platforms. It runs entirely on the user’s hardware and can autonomously browse the web, write code, and plan multi-step tasks without sending data to external services. The system is optimized for local reasoning models and emphasizes zero cloud dependency to maintain full user control. AgenticSeek includes intelligent agent selection, allowing it to determine the best internal... -
15
Agent Stack
Deploy and share agents with open infrastructure
Agent Stack is an open infrastructure platform designed to take AI agents from prototype to production, no matter how they were built. It includes a runtime environment, multi-tenant web UI, catalog of agents, and deployment flow that seeks to remove vendor lock-in and provide greater autonomy. Under the hood it’s built on the “Agent2Agent” (A2A) protocol, enabling interoperability between different agent ecosystems, runtime services, and frameworks. The platform supports agents built in... -
16
notebooklm-py
Unofficial Python API and agentic skill for Google NotebookLM
notebooklm-py is an unofficial Python API and agent-ready integration layer for Google NotebookLM that exposes NotebookLM functionality through code, the command line, and AI agent workflows. Its goal is to provide programmatic access not just to standard notebook operations, but also to many capabilities that are either limited or unavailable in the web interface, making it especially useful for automation and custom pipelines. The project covers notebook management, source ingestion, conversational querying, research workflows, and sharing controls, while also enabling the generation of a wide range of study and media artifacts. ... -
17
Eigent
The Open Source Cowork Desktop to Unlock Your Exceptional Productivity
Eigent is an open-source cowork desktop application designed to help you build, manage, and deploy a custom AI workforce. It enables multiple specialized AI agents to collaborate in parallel, turning complex workflows into automated, end-to-end tasks. Built on the CAMEL-AI multi-agent framework, Eigent emphasizes productivity, flexibility, and transparent system design. You can run Eigent fully locally for maximum privacy and data control, or choose a cloud-connected experience for quick... -
18
ChatGPT Clone
ChatGPT interface with better UI
ChatGPT Clone demonstrates a ChatGPT-style conversational interface wired to large-language-model backends, packaged so developers can self-host and extend. The goal is to replicate the core chat UX—message history, streaming tokens, code blocks, and system prompts—while letting you plug in different provider APIs or local models. It showcases a clean separation between the web client and the message orchestration layer so you can experiment with prompts, roles, and memory strategies. The... -
19
PythonPark
Python open source project "The Road to Self-Study Programming"
PythonPark is a large, curated “learning playground” for Python — essentially a comprehensive self-study meta-repository aimed at helping learners progress in Python programming, data science, machine learning, web scraping, and software engineering practices. It aggregates tutorials, learning guides, project examples, and resources across topics: from Python basics and data structures to machine learning, web scraping, and even interview preparation and “programmer life” guidance. Because... -
20
Integuru v0
The first AI agent that builds permissionless integrations
Integuru is an open-source AI agent designed to automatically create integrations between software platforms by reverse-engineering their internal APIs. Instead of relying on official developer documentation or publicly available APIs, the system analyzes network traffic generated by user interactions within a web application. Developers capture browser requests and authentication data, which the agent then uses to infer the structure of the platform’s internal API endpoints. ... -
21
You-Get
Dumb downloader that scrapes the web
...It’s also a good choice for when you want to get rid of any closed-source technology or proprietary JavaScript code from running on your computer. -
22
ChatGLM-6B
ChatGLM-6B: An Open Bilingual Dialogue Language Model
ChatGLM-6B is an open bilingual (Chinese + English) conversational language model based on the GLM architecture, with approximately 6.2 billion parameters. The project provides inference code, demos (command line, web, API), quantization support for lower memory deployment, and tools for finetuning (e.g., via P-Tuning v2). It is optimized for dialogue and question answering with a balance between performance and deployability in consumer hardware settings. Support for quantized inference... -
23
CodeChecker
CodeChecker is an analyzer tooling, defect database
CodeChecker is a static analysis infrastructure built on the LLVM/Clang Static Analyzer toolchain, replacing scan-build in a Linux or macOS (OS X) development environment. Executes Clang-Tidy and Clang Static Analyzer with Cross-Translation Unit analysis, Statistical Analysis (when checkers are available). Creates the JSON compilation database by wiretapping any build process (e.g., CodeChecker log -b "make"). Automatically analyzes GCC cross-compiled projects: detecting GCC or Clang... -
24
Slack Machine
A simple, yet powerful and extendable Slack bot
Slack Machine is a simple, yet powerful and extendable Slack bot framework. More than just a bot, Slack Machine is a framework that helps you develop your Slack workspace into a ChatOps powerhouse. Slack Machine is built with an intuitive plugin system that lets you build bots quickly but also allows for easy code organization. A plugin can look as simple as this: -
25
A.I.G
Full-stack AI Red Teaming platform
AI-Infra-Guard is a powerful open-source security platform from Tencent’s Zhuque Lab designed to assess the safety and resilience of AI infrastructures, codebases, and components through automated scanning and evaluation tools. It brings together AI infrastructure vulnerability scanning, MCP server risk analysis, and jailbreak evaluation into a unified workflow so that enterprises and individuals can identify critical security issues without relying on external services. Users can deploy it...
