Search Results for "python web crawler"

ArchiveBox is a powerful, self-hosted internet archiving solution to collect, save, and view websites offline. Without active preservation effort, everything on the internet eventually disappears or degrades. Archive.org does a great job as a centralized service, but saved URLs have to be public, and they can't save every type of content. ArchiveBox is an open source tool that lets organizations & individuals archive both public & private web content while retaining control over their data....

...These Rock-ons, combined with advanced NAS features, turn Rockstor into a private cloud storage solution accessible from anywhere, giving users complete control of cost, ownership, privacy and data security. Rockstor UI is written in Javascript, making it simple to manage everything from your Web browser. The backend is written in Python and exposes RESTful APIs to easily extend functionality!

A cloud backup solution that employs the "ChaCha20 + Serpent-256 CBC + HMAC-SHA3-512" authenticated encryption scheme for data encryption and ML-KEM-1024 for quantum-resistant key exchange. Check it out at https://plum-cave.netlify.app/ GitHub page: https://github.com/Northstrix/plum-cave

This project provides an open-source setup for IPFS Kubo (public and private pinning) combined with IPFS Cluster replication, designed for reliable, production-ready decentralized storage. It is suitable for developers, infrastructure operators, and Web3 projects that need predictable IPFS availability without maintaining complex node infrastructure. The system supports multiple retention plans for IPFS Kubo pinning, allowing users to choose how long data remains pinned, with transparent...

bitfarm-Archiv is a powerful Document Management (DMS), Enterprise Content Management (ECM) and Knowledge Management System (KMS) with Workflow Components. Help us! As we live in the internet age, the best thing, you can help, is to write a short statement about your scenario and your use of the DMS, along with your experiences and put it on your own website or in a blog or forum. It would help us best, if you can also add a hyperlink to our site http://www.bitfarm-archiv.com. By this...

Using this project you can make backup and track changes of configuration of network devices like switches, routers, etc.

A set of tools (command line and GUI) to provide a complete digital photo workflow for Unixes. EXIF headers are used as the central information repository, so users may change their software at any time without loosing any data.

mediaTUM is free software written in Python for archiving and retrieval of images, documents and other research data. It was originally developed in the framework of the DFG project IntegraTUM and is continuously expanded with new functionalities as required.

Cloud Export is a tool to automatically extract your data from web applications and save it to your local file system for backup purposes, but more extensive than Google Takeout. Plans are based on http://www.dataliberation.org.

The archive-crawler project is building Heritrix: a flexible, extensible, robust, and scalable web crawler capable of fetching, archiving, and analyzing the full diversity and breadth of internet-accesible content.

Backup and restore of files to web mail systems, ftp, sftp. Uses free storage of gmail/hotmail etc. Archives files, splits large files, encrypts and uploads. Requires python (tested with python 2.5)

Mutualized distant storage space management tool (using a distributed system).

Arrowbase is a collection of tools for backup persoses. Together they combine a backup system that can be used on more then one Operating system. This makes the project not only widely spread but portable as wel.

A configurable knowledge management framework. It works out of the box, but it's meant mainly as a framework to build complex information retrieval and analysis systems. The 3 major components: Crawler, Analyzer and Indexer can also be used separately.

XSDB XML is to DATA as HTML is to DOCUMENT. Publish and combine data as easily as HTML format and web browsers publish and view documents. Implementations in Python, javascript, java, C#/.NET.

CAIRN is a modular copy and restore program for the imaging of a computer. It copies every file on a computer and figures out how to recreate it from scratch. It is primarily network oriented but is also flexible enough to boot from any possible method.

Agile Author is a framework for developing networked repositories of digital information such as digital libraries and content management systems.

An Open Source fork of the Redhat up2date client and the NRH-up2date server.

Cat-photo makes administration and web pages with photos easy.

idyuts is \"I Dare You to Use This Shell\"; a pre-hibernate approach to replacing an ORM written with jython functors into a pure-Java language command pattern. The \"pipeline codegen artifacts\" are simple IoC templates, and trivial to adapt

View, track, filter, archive, alert, group, rotate logs through a GUI, CLI, or WebUI.

Rescuezilla is an easy-to-use disk cloning and imaging application that's fully compatible with Clonezilla — the industry-standard trusted by tens of millions. Yes, Rescuezilla is the Clonezilla GUI (graphical user interface) that you might have been looking for. **See: https://rescuezilla.com/ for download links** **NEW** Weekly rolling release downloads: https://github.com/rescuezilla/rescuezilla/releases Rescuezilla is a fork of Redo Backup and Recovery (now called Redo...