Open Source C Large Language Models (LLM)

Run, create, and share large language models (LLMs). Get up and running with large language models, locally. Run Llama 2 and other models on macOS. Customize and create your own.

The llama.cpp project enables the inference of Meta's LLaMA model (and other models) in pure C/C++ without requiring a Python runtime. It is designed for efficient and fast model execution, offering easy integration for applications needing LLM-based capabilities. The repository focuses on providing a highly optimized and portable implementation for running large language models directly within C/C++ environments.

TuyaOpen is an open-source AI-enabled Internet of Things development framework designed to simplify the creation and deployment of smart connected devices. The platform provides a cross-platform C and C++ software development kit that supports a wide range of hardware platforms including Tuya microcontrollers, ESP32 boards, Raspberry Pi devices, and other embedded systems. It offers a unified development environment where developers can build devices capable of communicating with IoT cloud services while integrating AI capabilities and intelligent automation features. The system includes built-in networking support for communication protocols such as Wi-Fi, Bluetooth, and Ethernet, allowing devices to connect securely to remote services and applications. TuyaOpen also integrates with Tuya’s broader cloud ecosystem, enabling developers to manage device authentication, firmware updates, device activation, and remote monitoring from centralized services.

Run a fast ChatGPT-like model locally on your device. This combines the LLaMA foundation model with an open reproduction of Stanford Alpaca a fine-tuning of the base model to obey instructions (akin to the RLHF used to train ChatGPT) and a set of modifications to llama.cpp to add a chat interface. Download the zip file corresponding to your operating system from the latest release. The weights are based on the published fine-tunes from alpaca-lora, converted back into a PyTorch checkpoint with a modified script and then quantized with llama.cpp the regular way.

llama2.c is a minimalist implementation of the Llama 2 language model architecture designed to run entirely in pure C. Created by Andrej Karpathy, this project offers an educational and lightweight framework for performing inference on small Llama 2 models without external dependencies. It provides a full training and inference pipeline: models can be trained in PyTorch and later executed using a concise 700-line C program (run.c). While it can technically load Meta’s official Llama 2 models, current support is limited to fp32 precision, meaning practical use is capped at models up to around 7B parameters. The goal of llama2.c is to demonstrate how a compact and transparent implementation can perform meaningful inference even with small models, emphasizing simplicity, clarity, and accessibility. The project builds upon lessons from nanoGPT and takes inspiration from llama.cpp, focusing instead on minimalism and educational value over large-scale performance.

Llama 2 Everywhere (L2E) is an open-source implementation of the LLaMA-2 large language model architecture designed to demonstrate how transformer-based language models can be executed with extremely minimal code. The project focuses on simplicity and educational clarity by implementing inference for LLaMA-style models in a compact C program rather than relying on large machine learning frameworks. Developers can train models using a Python training pipeline and then run inference using a lightweight C implementation that requires very few dependencies. The architecture mirrors the structure of the LLaMA-2 model family, allowing compatible model checkpoints to be converted and executed within the simplified runtime environment. Because the implementation is intentionally minimal, it serves as a teaching tool for understanding how transformer architectures operate at a low level.

Python bindings for the Transformer models implemented in C/C++ using GGML library.

PicoLM is an open-source inference framework designed to run large language models on extremely constrained hardware environments such as inexpensive single-board computers and embedded systems. The project focuses on enabling efficient local inference by optimizing memory usage, computation, and system dependencies so that relatively large models can operate on devices with minimal RAM. It is written primarily in C and designed with a minimalist architecture that removes unnecessary dependencies and external libraries. The runtime is capable of running language models with billions of parameters on devices with only a few hundred megabytes of memory, which is significantly lower than typical LLM infrastructure requirements. This makes PicoLM particularly suitable for edge computing, offline AI applications, and embedded AI devices that cannot rely on cloud resources.