Port of Facebook's LLaMA model in C/C++
Run Local LLMs on Any Device. Open-source
CodeGeeX: An Open Multilingual Code Generation Model (KDD 2023)
Distribute and run LLMs with a single file
TT-NN operator library, and TT-Metalium low level kernel programming
Mooncake is the serving platform for Kimi
Alibaba's high-performance LLM inference engine for diverse apps
Emscripten: An LLVM-to-WebAssembly Compiler
C++ implementation of ChatGLM-6B & ChatGLM2-6B & ChatGLM3 & GLM4(V)
An Easy-to-Use and High-Performance AI Deployment Framework
Fast Multimodal LLM on Mobile Devices
Production ready toolkit to run AI locally
A @ClickHouse fork that supports high-performance vector search
INT4/INT5/INT8 and FP16 inference on CPU for RWKV language model
UCCL is an efficient communication library for GPUs
High-speed Large Language Model Serving for Local Deployment
Locally run an Instruction-Tuned Chat-Style LLM
Implements a reference architecture for creating information systems