RainforestCluster

RainforestCluster is an Amazon EC2 python program that manages and load-balances dynamic clusters to allow for maximum workflow flexibility and speed at minimal cost. It enables one to quickly and cheaply create dynamic compute clusters in the cloud, which can then run computational pipelines generically. It is also able to optimize the use of spot instances - idle computers in Amazon's cloud that are available at drastically reduced cost (5x-10x cheaper) - but can be terminated at any moment if capacity drops or the bid price rises. It also provides pre-installed features such as GlusterFS distributed filesystems, ThunderstormDistributor queuing system, RAID 0 /scratch, password-less ssh, and automatic cluster management, for ease of use and maximum processing speed for computational tools. Originally it was developed as a different version for the Wall Lab at Harvard CBMI.

Features

Create Amazon EC2 custom clusters dynamically
Add, remove, and loadbalance nodes
Create large shared gluster volumes with spanning across multiple masters
Distribute jobs efficiently using ThunderstormDistributor
View machine, job, and cluster statistics on a web-based interface

Project Activity

See All Activity >

License

MIT License

Follow RainforestCluster

RainforestCluster Web Site

Other Useful Business Software

The Most Powerful Software Platform for EHSQ and ESG Management

Addresses the needs of small businesses and large global organizations with thousands of users in multiple locations.

Choose from a complete set of software solutions across EHSQ that address all aspects of top performing Environmental, Health and Safety, and Quality management programs.

Learn More

Rate This Project

User Reviews

Be the first to post a review of RainforestCluster!

Additional Project Details

Operating Systems

Cygwin, Linux, Mac

Intended Audience

Information Technology, Science/Research

User Interface

Console/Terminal

Programming Language

Python

Related Categories

Python Systems Administration Software, Python Clustering Software, Python Distributed Computing Software

Registered

2014-03-09

Similar Business Software

Google Compute Engine

Compute Engine is Google's infrastructure as a service (IaaS) platform for organizations to create and run cloud-based virtual machines. Computing infrastructure in predefined or custom machine sizes to accelerate your cloud transformation. General purpose (E2, N1, N2, N2D) machines provide a...

See Software
Comet Backup

Start running backups and restores in less than 15 minutes! Fast, secure backup software for businesses and IT providers. Comet is a flexible, all-in-one backup platform available in 13 languages. You choose your backup destination, server location, configuration and setup. Backup to your...

See Software
SureSync

SureSync Pro is a file replication and synchronization application that provides one-way and multi-way processing in both scheduled and real-time modes. The Communications Agent provides real-time monitors, delta copies via Remote Differential Compression, TCP communications, compression, and...

See Software
Files.com

Files.com is a cloud-native Managed File Transfer (MFT) platform that unifies file transfers, sharing, and automation across any cloud, protocol, or partner. It connects 50+ storage systems — including Amazon S3, Azure, Google Drive, SharePoint, Dropbox, and Box — presenting them as a single...

See Software
Google Cloud Platform

Google Cloud is a cloud-based service that allows you to create anything from simple websites to complex applications for businesses of all sizes. New customers get $300 in free credits to run, test, and deploy workloads. All customers can use 25+ products for free, up to monthly usage...

See Software
Kasm Workspaces

Kasm Workspaces streams your workplace environment directly to your web browser…on any device and from any location. Kasm uses our high-performance streaming and secure isolation technology to provide web-native Desktop as a Service (DaaS), application streaming, and secure/private web...

See Software