Search Results for "web crawler source code" - Page 8

Showing 374 open source projects for "web crawler source code"

View related business solutions
  • viewneo - Smart software for digital advertising boards Icon
    viewneo - Smart software for digital advertising boards

    Smart digital signage for 1 to 1000+ screens.

    viewneo is a user-friendly, cloud-based solution that allows companies of all sizes to set up digital signage
    Learn More
  • Digital business card + lead capture + contact enrichment Icon
    Digital business card + lead capture + contact enrichment

    Your complete in-person marketing platform

    Share digital business cards, capture leads, and enrich validated contact info - at events, in the field, and beyond. Powered by AI and our proprietary data engine, Popl drives growth for companies around the world, turning every handshake into an opportunity.
    Learn More
  • 1
    Komodo Edit

    Komodo Edit

    Komodo Edit is a fast and free multi-language code editor

    Code faster with the IDE for today’s web languages – Python, PHP, Perl, Golang, Ruby, and more. Powerful editor with syntax highlighting, autocomplete, code refactoring, and more. Visual debugger and more to debug, inspect, and test your code. Support for Git, Mercurial, Subversion, CVS, Perforce and Bazaar. Tons of add-ons for customizing and extending features. Great tools for pair programming and collaboration. Python, PHP, Perl, Go, Ruby, Node.js, JavaScript, and more. Define your own...
    Downloads: 42 This Week
    Last Update:
    See Project
  • 2
    OpsManage

    OpsManage

    Automated operation and maintenance platform

    Automated operation and maintenance platform: code and application deployment CI/CD, asset management CMDB, scheduled task management platform, SQL audit | rollback, task scheduling, on-site WIKI. A code deployment, application deployment, scheduled tasks, and equipment asset management platform. Welcome to star or fork my open source project. If you need to quote the project code in your own project, please declare the agreement and copyright information in the project. ...
    Downloads: 0 This Week
    Last Update:
    See Project
  • 3
    DecryptLogin

    DecryptLogin

    Python library providing APIs for automated website login workflows

    DecryptLogin is a Python library designed to simplify automated login processes for many popular websites by providing ready-to-use APIs that simulate authentication behavior. It focuses on implementing login mechanisms through HTTP requests, allowing developers to programmatically authenticate with supported services without manually replicating complex login flows. It includes modules that handle different authentication modes such as PC login, mobile login, and QR code login depending on...
    Downloads: 0 This Week
    Last Update:
    See Project
  • 4
    FormaVid

    FormaVid

    Small Business Appliance

    The FormaVid Small Business Appliance https://formavid.org is designed to integrate a content management system (CMS), an issues tracker and an invoicing application into a single, well constructed, offering. It is an excellent starting point for any developer(s) wishing to support the CMS or any of the other components, including the appliance itself. All components are stable, open source and well supported. The appliance is built using scripts so no hidden "monkey business" and you can...
    Downloads: 0 This Week
    Last Update:
    See Project
  • DataImpulse - Ethical Provider of Residential, Mobile, and Data Center IPs Icon
    DataImpulse - Ethical Provider of Residential, Mobile, and Data Center IPs

    For anyone looking for residential proxies, mobile proxies, and datacenter proxies

    DataImpulse (dataimpulse.com) is a proxy provider offering a pool of over 90 million ethically sourced residential, mobile, and data center IP addresses from 195 countries. Pricing for residential proxies starts at $1 per GB with a pay-as-you-go model; there are no subscriptions or traffic expiration dates.
    Learn More
  • 5
    Moriarty Project

    Moriarty Project

    Web-based OSINT tool for investigating phone number information

    Moriarty Project is an open source web-based investigation tool designed to gather publicly available information about phone numbers. It allows users to input a phone number and analyze various details related to that number through multiple investigation features. It performs information gathering by scraping data from online sources to retrieve insights such as owner information, spam risk, and related web references.
    Downloads: 8 This Week
    Last Update:
    See Project
  • 6
    grab-site

    grab-site

    Web crawler for archiving and backing up sites into WARC archives

    grab-site is an open source web crawling tool designed to archive and back up websites by recursively downloading their content. It works by taking a starting URL and systematically following links across the site, capturing pages and resources and saving them into WARC archive files for long-term preservation. Internally, the crawler uses a fork of the wpull engine to fetch and process web pages efficiently during large-scale crawls. grab-site includes a built-in dashboard that displays real-time crawl activity, including which URLs are currently being processed and how many remain in the queue. ...
    Downloads: 0 This Week
    Last Update:
    See Project
  • 7
    ostRAT

    ostRAT

    OpenSourceTelegramRAT - Remote PC access via Telegram Bot.

    ostRAT is free and open source. GPLv3 Сomputer remote control software. Works via telegram bot. A lot of functions, for example: - Screenshot: sends a screenshot - Off: turns off the computer - Url: opens entered link - Write: sends your text to the computer - Move: changes mouse location with x and y - and more! WARNING: Using the bot is recommended only on your device. Failure to comply with the recommendation may result in criminal liability.
    Downloads: 0 This Week
    Last Update:
    See Project
  • 8
    pspider

    pspider

    Simple Python framework for building multithreaded web crawlers

    PSpider is a lightweight web crawling framework written in Python designed to simplify the development of custom web spiders. It focuses on providing an easy-to-understand architecture while still supporting concurrent crawling for improved performance. It uses a multithreaded model that separates the crawling workflow into several components responsible for fetching, parsing, and saving data. Tasks are managed through queues, allowing different parts of the crawler to process work...
    Downloads: 1 This Week
    Last Update:
    See Project
  • 9
    Recursive Sans & Mono

    Recursive Sans & Mono

    Recursive Mono & Sans is a variable font family for code & UI

    Recursive Sans & Mono is a variable type family built for better code & UI. It is inspired by casual script signpainting, but designed primarily to meet the needs of programming environments and application interfaces. In programming, “recursion” is when a function calls itself, using its own output as an input to yield powerful results. Recursive Mono was used as a tool to help build itself: it was used to write Python scripts to automate type production work and to generate specimen...
    Downloads: 0 This Week
    Last Update:
    See Project
  • Ditto Edge Server is a lightweight standalone server for resource-constrained edge environments, based on the core Ditto Edge SDK. Icon
    Ditto Edge Server is a lightweight standalone server for resource-constrained edge environments, based on the core Ditto Edge SDK.

    With Ditto Edge Server, you can join devices as small as a Raspberry Pi to a local mesh network and synchronize data across edge environments.

    Ditto's Edge SDK is the only thing your edge devices need to ensure your application is operational in any environment, regardless of network conditions.
    Learn More
  • 10
    Google Cloud Vision API examples

    Google Cloud Vision API examples

    Sample code for Google Cloud Vision

    The cloud-vision repository is a sample code collection for the Google Cloud Vision API that shows developers how to implement image analysis tasks across a wide range of languages and platforms. It contains examples organized by language and environment, including Go, Java, Node.js, PHP, Python, Ruby, .NET, Android, iOS, and even a Chrome extension, which makes it especially valuable as a cross-platform learning resource. The repository demonstrates concrete image understanding use cases,...
    Downloads: 0 This Week
    Last Update:
    See Project
  • 11
    CIPS-3D

    CIPS-3D

    3D-aware GANs based on NeRF (arXiv)

    3D-aware GANs based on NeRF (arXiv). This repository contains the code of the paper, CIPS-3D: A 3D-Aware Generator of GANs Based on Conditionally-Independent Pixel Synthesis. The problem of mirror symmetry refers to the sudden change of the direction of the bangs near the yaw angle of pi/2. We propose to use an auxiliary discriminator to solve this problem. Note that in the initial stage of training, the auxiliary discriminator must dominate the generator more than the main discriminator...
    Downloads: 0 This Week
    Last Update:
    See Project
  • 12
    Code Cookbook

    Code Cookbook

    Reusable code patterns which you can use as reference or copy

    Reusable code patterns which you can use as reference or copy to your project. Achieve small or large tasks using recipes that contain steps, scripts, and config files.
    Downloads: 0 This Week
    Last Update:
    See Project
  • 13
    e-Dokyumento

    e-Dokyumento

    e-Dokyumento is web-based Document Management System (DMS)

    e-Dokyumento is opensource web-based Document Management System (DMS) A Document Management which automates the basic office document workflow such as receiving, filing, routing, and approving through capturing (scanning), digitizing (OCR Reading), storing, tagging, and electronically routing and approving (e-signature) of electronic documents. # Demo : https://e-dokyumento.herokuapp.com/ https://edokyu.seillig.com/ (refer to Readme.md for the...
    Downloads: 1 This Week
    Last Update:
    See Project
  • 14
    Spyne

    Spyne

    A transport agnostic sync/async RPC library

    Spyne is a Python RPC toolkit that makes it easy to expose online services that have a well-defined API using multiple protocols and transports. It integrates with popular Python web frameworks as well as libraries like SQLAlchemy to keep your code as DRY as possible. Spyne aims to save the protocol implementers the hassle of implementing their own remote procedure call api and the application programmers the hassle of jumping through hoops just to expose their services using multiple...
    Downloads: 0 This Week
    Last Update:
    See Project
  • 15
    instagram-profilecrawl

    instagram-profilecrawl

    Instagram profile crawler that extracts posts, tags, and stats

    instagram-profilecrawl is a Python-based automation script designed to collect publicly available information from Instagram profiles. It crawls profile data such as follower counts, post information, hashtags, and other engagement-related metadata. It operates by automating a web browser using Selenium and performing requests to gather structured information from the platform. instagram-profilecrawl can analyze multiple usernames in a single run and store the extracted information locally...
    Downloads: 3 This Week
    Last Update:
    See Project
  • 16
    DracOS GNU/Linux Remastered
    What is DracOS GNU/Linux Remastered ? DracOS GNU/Linux Remastered ( https://github.com/dracos-linux ) is the Linux operating system from Indonesia , open source is built based on Debian live project under the protection of the GNU General Public License v3.0. This operating system is one variant of Linux distributions, which is used to perform security testing (penetration testing). Dracos linux in Arm by hundreds hydraulic pentest, forensics and reverse engineering. Use a GUI-based...
    Downloads: 26 This Week
    Last Update:
    See Project
  • 17
    Aquila X

    Aquila X

    Easy build your personal search engine with Aquila Network

    Easy build your personal search engine with Aquila Network. Aquila X is the gateway to Aquila Network and it's applications. AquilaX is a smart bookmarking tool. You can keep your bookmarks and search through it's contents. Choose to keep all your data in a local server or in the cloud. This is an open source software and thus is auditable.
    Downloads: 0 This Week
    Last Update:
    See Project
  • 18
    ReconSpider

    ReconSpider

    Most Advanced Open Source Intelligence (OSINT) Framework

    ...Reconnaissance is a mission to obtain information by various detection methods, about the activities and resources of an enemy or potential enemy, or geographic characteristics of a particular area. A Web crawler, sometimes called a spider or spiderbot and often shortened to crawler, is an Internet bot that systematically browses the World Wide Web, typically for the purpose of Web indexing (web spidering).
    Downloads: 8 This Week
    Last Update:
    See Project
  • 19
    restless

    restless

    A lightweight REST miniframework for Python

    ...If you can import a module to work with the data & can represent it as JSON, Restless can work with it. Restless is small & easy to keep in your head. Common usages involve overriding just a few easily remembered method names. Total source code is an under a thousand lines of code.
    Downloads: 3 This Week
    Last Update:
    See Project
  • 20
    ML workspace

    ML workspace

    All-in-one web-based IDE specialized for machine learning

    All-in-one web-based development environment for machine learning. The ML workspace is an all-in-one web-based IDE specialized for machine learning and data science. It is simple to deploy and gets you started within minutes to productively built ML solutions on your own machines. This workspace is the ultimate tool for developers preloaded with a variety of popular data science libraries (e.g., Tensorflow, PyTorch, Keras, Sklearn) and dev tools (e.g., Jupyter, VS Code, Tensorboard)...
    Downloads: 0 This Week
    Last Update:
    See Project
  • 21
    aseryla

    aseryla

    Aseryla code repositories

    This project describes a model of how the semantic human memory represents the information relevant to the objects of the world in text format. It provides a system and a GUI application capable of extracting and managing concepts and relations from English texts. https://aseryla2.sourceforge.io/
    Downloads: 0 This Week
    Last Update:
    See Project
  • 22
    Pytago

    Pytago

    A source-to-source transpiler for Python to Go translation

    pytago is a source-to-source transpiler that converts some Python into human-readable Go. It enables developers to translate Python codebases into Go, facilitating migration or interoperability between the two languages. ​
    Downloads: 2 This Week
    Last Update:
    See Project
  • 23
    Glazier

    Glazier

    A tool for automating the installation of Windows OS

    ...It streamlines the entire Windows imaging process by booting systems into the Windows Preinstallation Environment (WinPE), retrieving installation instructions from a web server, and automatically applying operating systems, software, and configurations. The tool is fully text-based and code-driven, with configurations written in YAML, allowing teams to leverage source control for versioning, collaboration, and quality assurance. By distributing installation data via HTTPS, Glazier ensures scalability and flexibility, supporting both simple local servers and large-scale cloud-based deployments. ...
    Downloads: 0 This Week
    Last Update:
    See Project
  • 24

    HomeTabs

    HomeTabs project helps you to organize bookmarks for web browsers

    HomeTabs project helps you to organize bookmarks for web browsers (like a standart browser's home page, but cooler and more comfortable). Design of HomeTabs was inspiried by Mozilla Firefox startpage, i think this is the best way to organise bookmarks, but history of browsing saved on homepage - is bad idea. GitHub: https://github.com/grildroid/HomeTabs Discord: https://discord.gg/6ZGDgFjDVm
    Downloads: 0 This Week
    Last Update:
    See Project
  • 25
    I Heart LA

    I Heart LA

    Compilable markdown for linear algebra

    I Heart LA is a compilable markdown for math. It can generate working code in your favorite language (C++, Python, MATLAB, more to come) and LaTeX from snippets.
    Downloads: 0 This Week
    Last Update:
    See Project
MongoDB Logo MongoDB