Download
Aviary is an LLM serving solution that makes it easy to deploy and manage a variety of open source LLMs. Providing an extensive suite of pre-configured open source LLMs, with defaults that work out of the box. Supporting Transformer models hosted on Hugging Face Hub or present on local disk. Aviary has native support for autoscaling and multi-node deployments thanks to Ray and Ray Serve. Aviary can scale to zero and create new model replicas (each composed of multiple GPU workers) in response to demand. Ray ensures that the orchestration and resource management is handled automatically. Aviary is able to support hundreds of replicas and clusters of hundreds of nodes, deployed either in the cloud or on-prem.
Features
- Fully supporting multi-GPU & multi-node model deployments
- Providing a REST API that is similar to OpenAI's to make it easy to migrate and cross test them
- Offering high performance features like continuous batching, quantization and streaming
- Simplifying the addition of new LLMs to within minutes in most cases
- Simplifying the deployment of multiple LLMs within a single unified framework
- Providing an extensive suite of pre-configured open source LLMs, with defaults that work out of the box
Project Samples
Categories
Large Language Models (LLM)License
Apache License V2.0Follow Aviary
Other Useful Business Software
Power through agendas and documents, make more informed decisions and conduct board meetings faster.
iBabs not only captures the entire decision-making process – it takes all the paperwork out of meetings. iBabs empowers everyone who has ever organized or attended, a meeting. With a seemingly simple app that offers complete control and a comprehensive overview of all those fiddly details. With about 3000 organizations and over 300,000 users, iBabs gives you peace of mind. So you can quickly organize effective meetings, and good decisions can be made with confidence. iBabs didn’t just happen overnight. We started analyzing and simplifying board meeting processes many years ago. We understand all the work that goes into meetings, and how to streamline everything so it all flows smoothly. On any device, confidentially, securely and automatically. Make good decisions with confidence.
Rate This Project
Login To Rate This Project
User Reviews
Be the first to post a review of Aviary!
