Search Results for "web crawler source code"
Sort By:
Showing 980 open source projects for "web crawler source code"
View related business solutions-
Skillfully - The future of skills based hiringSkillfully transforms hiring through AI-powered skill simulations that show you how candidates actually perform before you hire them. Our platform helps companies cut through AI-generated resumes and rehearsed interviews by validating real capabilities in action. Through dynamic job specific simulations and skill-based assessments, companies like Bloomberg and McKinsey have cut screening time by 50% while dramatically improving hire quality.
-
BrandMail Email Signatures for OutlookBrandMail®, developed by BrandQuantum, is a software solution that seamlessly integrates with Microsoft Outlook to empower every employee in the organisation to automatically create consistently branded emails via a single toolbar that provides access to brand standards and the latest pre-approved content.
-
1
crawler
Collection of JS reverse engineering examples for web scraping study
crawler is a collection of web scraping and JavaScript reverse engineering examples designed for learning how modern websites protect their data and how those protections can be analyzed. It contains many case studies that demonstrate how to analyze and replicate request parameters, cookies, and encryption logic used by real websites. Each directory in the project focuses on a specific target service or scenario, showing how browser network requests and JavaScript code can be studied to reproduce API calls programmatically. ... -
2
EasySpider
A visual no-code/code-free web crawler/spider
A visual code-free/no-code web crawler/spider, supporting both Chinese and English. -
3
Phoenix Code Editor
Phoenix is a modern open-source Code Editor for the web
Phoenix is a modern open-source and free software code editor for the web, built for the browser. -
4
Every Code
Local AI coding agent CLI with multi-agent orchestration tools
Every Code (often referred to simply as Code) is a fast, local AI-powered coding agent designed to run directly in the terminal environment. It is a community-driven fork of the Codex CLI, with a strong emphasis on improving real-world developer ergonomics and workflows. Every Code enhances the traditional coding assistant model by introducing multi-agent orchestration, allowing multiple AI agents to collaborate, compare solutions, and refine outputs in parallel. It supports integration with... -
Instant Remote Support Software. Unattended Remote Access Software.Zoho Assist is cloud-based remote support and remote access software that helps you support customers from a distance through web-based, on-demand remote support sessions. Set up unattended remote access and manage remote PCs, laptops, mobile devices, and servers effortlessly. A few seconds is all you need to establish secure connections to offer your customers remote support solutions.
-
5
Web-Maker
A blazing fast & offline frontend playground
Web-Maker is an offline playground for your web experiments. Something like CodePen or JSFiddle, but much more faster and works offline because it runs completely on your system. Supports Preprocessors: HTML (Pug & Markdown), CSS (SCSS, LESS & Stylus, Atomic CSS) & JavaScript (ES6, TypeScript & CoffeeScript). Hi! I am Kushagra Gour. Web Maker is a free and open-source project. To keep me motivated for working on such open-source and free side projects, I have launched a Patreon campaign.... -
6
Laravel Web Tinker
Tinker in your browser
Artisan's tinker command is a great way to tinker with your application in the terminal. Unfortunately running a few lines of code, making edits, and copy/pasting code can be bothersome. Wouldn't it be great to tinker in the browser? This package will add a route to your application where you can tinker to your heart's content. In case light hurts your eyes, there's a dark mode too. -
7
whatsapp-web.js
WhatsApp library for NodeJS that connects through the browser app
A WhatsApp client library for NodeJS that connects through the WhatsApp Web browser app. Programmatically control WhatsApp whether you're running user or business accounts. It uses Puppeteer to run a real instance of Whatsapp Web to avoid getting blocked. Programmatically control WhatsApp whether you're running user or business accounts. Whatsapp-web.js connects to an official version of WhatsApp Web under the hood, reducing ban risks. The object-oriented approach makes it easy to get... -
8
Web Experience Toolkit
Open source code library for building innovative websites
Web Experience Toolkit (WET): Open source code library for building innovative websites that are accessible, usable, interoperable, mobile-friendly and multilingual. This collaborative open source project is led by the Government of Canada. A collection of flexible and themeable templates and reusable components. A collaborative open source project led by the Government of Canada. -
9
katana
Fast CLI web crawler for discovering endpoints in modern web apps
Katana is an open source command-line web crawling and spidering framework developed by ProjectDiscovery. It is designed to efficiently crawl websites and web applications in order to discover endpoints, resources, and other useful information that may not be easily visible through manual browsing. Katana focuses on speed and automation, making it suitable for use in security reconnaissance workflows and automated pipelines. Katana supports both standard HTTP crawling and headless browser... -
Teradata VantageCloud Enterprise is a data analytics platform for performing advanced analytics on AWS, Azure, and Google Cloud.VantageCloud is the complete cloud analytics and data platform, delivering harmonized data and Trusted AI for all. Built for performance, flexibility, and openness, VantageCloud enables organizations to unify diverse data sources, run complex analytics, and deploy AI models—all within a single, scalable platform.
-
10
Lightning Web Components
A Blazing Fast, Enterprise-Grade Web Components Foundation
Lean, lightweight runtime optimized for performance, with minimal boilerplate code. Lightning Web Components uses standard HTML, modern JavaScript (ES6+), and the best of native Web Components. Leave abstractions behind and build custom elements with HTML, JavaScript, and CSS. Lightning web components are custom elements built using HTML and modern JavaScript. Ready to learn more? Check out the Developer Guide and write code in the Playground. You’ll be up and running in no time. Create... -
11
fess
Open source enterprise search server for websites, files, and data
...Fess includes a built-in crawler that can collect content from sources such as databases, CSV files, and shared storage, making it suitable for centralized knowledge discovery. It supports indexing and searching across many document formats including office documents, PDFs, and compressed archives. It also provides a web-based administrative interface that allows administrators to configure crawling targets, manage indexing tasks, and adjust search settings from a graphical dashboard. -
12
Lottie-Web
Render After Effects animations natively
Lottie-Web renders vector animations exported from Adobe After Effects as lightweight JSON, playing them in the browser via SVG or Canvas. Animations remain crisp at any resolution, load quickly compared to videos or GIFs, and can be controlled programmatically for interactivity. The runtime supports playback options like loop, autoplay, segment control, and event hooks, making it easy to sync motion with UI state. Because animations are data rather than pixels, designers and developers can... -
13
IPFS Web UI
A frontend for an IPFS node
A web interface to IPFS, shipped with Kubo, and ipfs-desktop. Check on your node stats, explore the IPLD powered merkle forest, see peers around the world and manage your files, without needing to touch the CLI. When working on the code, run an ipfs daemon, the local dev server, the unit tests, and the storybook component viewer and see the results of your changes as you save files. If you need to run IPFS in a Docker container, you can still have Web UI available by exposing both the... -
14
douyin
Open source Douyin crawler for collecting and downloading public data
DouyinCrawler is an open source data collection tool designed to gather publicly available information from the Douyin platform. It demonstrates how to build a Python-based web crawler combined with a graphical interface and command line functionality. It allows users to collect data from various types of Douyin content, including user profiles, videos, hashtags, and music pages. -
15
Playwright Skill for Claude Code
Claude Code Skill for browser automation with Playwright
...The system supports a wide range of use cases, including testing web applications, validating user interfaces, automating workflows, and extracting data from websites. One of its key advantages is its ability to generate custom Playwright code tailored to each request, allowing flexible and context-aware automation. -
16
web-extension-starter
Web Extension starter to build "Write Once Run on Any Browser"
Web Extension starter to build "Write Once Run on Any Browser" extension. Cross Browser Support (Web-Extensions API). Browser Tailored Manifest generation. Automatic build on code changes. Auto packs browser-specific build files. SASS styling. TypeScript by default. ES6 modules support. React UI Library by default. Smart reload. Used by extensions in production that has over 100,000+ users. -
17
Jupyter Notebook
Jupyter Interactive Notebook
The notebook extends the console-based approach to interactive computing in a qualitatively new direction, providing a web-based application suitable for capturing the whole computation process: developing, documenting, and executing code, as well as communicating the results. The Jupyter notebook combines two components. A web application, which is a browser-based tool for interactive authoring of documents which combine explanatory text, mathematics, computations and their rich media... -
18
Web Dev for Beginners
About 24 Lessons, 12 Weeks, Get Started as a Web Developer
Web-Dev-For-Beginners is Microsoft’s open source, project-based curriculum for learning web development from scratch. Designed as a 12-week, 24-lesson course, it covers HTML, CSS, and JavaScript fundamentals through hands-on projects like terrariums, browser extensions, and space games. Each lesson includes a mix of pre-lecture quizzes, written content, assignments, challenges, and post-lecture quizzes to reinforce learning. -
19
spider_collection
Collection of Python web scraping scripts for data extraction tasks
spider_collection is a collection of Python web crawler scripts created primarily for experimentation, learning, and practical scraping tasks. spider_collection gathers multiple independent spiders designed to collect data from different platforms and services, demonstrating a variety of scraping techniques and workflows. These crawlers make use of common Python scraping tools such as requests, parsel, BeautifulSoup, and the Scrapy framework to extract structured information from web pages.... -
20
Bot Framework Web Chat
A highly-customizable web-based client for Azure Bot Services
This repository contains code for the Bot Framework Web Chat component. The Bot Framework Web Chat component is a highly-customizable web-based client for the Bot Framework V4 SDK. The Bot Framework SDK v4 enables developers to model conversation and build sophisticated bot applications. This repo is part of the Microsoft Bot Framework, a comprehensive framework for building enterprise-grade conversational AI experiences. -
21
ADAMANT Messenger Progressive Web App
ADAMANT decentralized Messenger, progressive web application
A messaging application client for ADAMANT Blockchain. ADAMANT is a decentralized anonymous messenger based on the blockchain system. It’s independent of any governments or corporations, and even developers due to the distributed network infrastructure that contains an open-source code. The ADAMANT blockchain system belongs to its users. Nobody can control, block, deactivate, restrict or censor accounts. Users take full responsibility for their content, messages, media, and goals and... -
22
Agregore Browser
A minimal browser for the distributed web (Desktop version)
A minimal web browser for the distributed web. Web Extension support. Built-in Markdown/Gemini/JSON rendering extension. Built-in QR code scanner and generator extension. Generate a QR code for the current page. Scan a QR code from the browser action window. Right-click a link or image to generate a QR code for it. Built-in ad blocker (ublock origin). Built-in support for creating web archives via ArchiveWeb.page. -
23
Next.js
The React Framework
Next.js is the React framework for lightweight apps, static websites, pre-rendered apps and more. It solves the most common problems associated with building a complete web application with React, such as those involving code bundling and transforming, production automizations, page rendering and having to write server-side code. Next.js offers a best in class “Developer Experience” through such capabilities as pre-rendering, single command static exporting, automatic code-splitting, hot... -
24
mslearn-tailspin-spacegame-web
Code used in Microsoft Learn modules to support Azure DevOps
The Tailspin Space Game Web project is a sample application created by Microsoft as part of its learning resources. It’s a web-based game application used in Microsoft Learn modules and documentation to demonstrate concepts such as Azure App Services, continuous integration and delivery (CI/CD) pipelines, and DevOps practices with GitHub Actions and Azure Pipelines. The project is intentionally lightweight and easy to deploy so learners can quickly experiment with cloud deployment, testing,... -
25
Monaco Editor
A browser based code editor
Monaco Editor is the rich, browser-based code editor that powers Visual Studio Code, providing advanced editing capabilities as a standalone embeddable library for web applications. Models are at the heart of Monaco editor. It's what you interact with when managing content. A model represents a file that has been opened. This could represent a file that exists on a file system, but it doesn't have to. For example, the model holds the text content, determines the language of the content, and...
