AICI: Prompts as (Wasm) Programs
Port of Facebook's LLaMA model in C/C++
A high-throughput and memory-efficient inference and serving engine
Port of OpenAI's Whisper model in C/C++
C++ implementation of ChatGLM-6B & ChatGLM2-6B & ChatGLM3 & GLM4(V)
High-performance neural network inference framework for mobile
OpenVINO™ Toolkit repository
Run Local LLMs on Any Device. Open-source
ONNX Runtime: cross-platform, high performance ML inferencing
The official Python client for the Huggingface Hub
Bayesian inference with probabilistic programming
Unofficial (Golang) Go bindings for the Hugging Face Inference API
C#/.NET binding of llama.cpp, including LLaMa/GPT model inference
Everything you need to build state-of-the-art foundation models
Ready-to-use OCR with 80+ supported languages
A unified framework for scalable computing
LLMs as Copilots for Theorem Proving in Lean
The free, Open Source alternative to OpenAI, Claude and others
Build Production-ready Agentic Workflow with Natural Language
Deep Learning API and Server in C++14 support for Caffe, PyTorch
C++ library for high performance inference on NVIDIA GPUs
FlashInfer: Kernel Library for LLM Serving
Phi-3.5 for Mac: Locally-run Vision and Language Models
Uncover insights, surface problems, monitor, and fine tune your LLM
PArallel Distributed Deep LEarning: Machine Learning Framework