Local LLM runnersMITC++

llama.cpp

LLM inference in C/C++ — the engine behind most local AI apps.

Stars: 131.2K
Forks: 24.4K
Open issues: 2.6K
Last commit: 2026-10-15

Live from the GitHub API · updated 2026-10-15

What is llama.cpp?

llama.cpp runs GGUF-quantized models efficiently on CPUs and GPUs (CUDA, Metal, Vulkan, ROCm, SYCL). It includes a CLI, an OpenAI-compatible server and tools for converting and quantizing models.

Key features

GGUF quantization from 1.5 to 8 bits
OpenAI-compatible llama-server with a web UI
Wide GPU backend support
Download models directly from Hugging Face with -hf

System requirements

Platforms: Windows · macOS · Linux · Docker
RAM: Depends on model size and quantization

How to install llama.cpp

Summarized from the project’s README. Projects change quickly — check the GitHub page for the latest instructions.

Package managers

1
Install with Homebrew (macOS/Linux) or winget (Windows).
```
brew install llama.cpp
# or
winget install llama.cpp
```

Chat with a model from Hugging Face.

llama-cli -hf ggml-org/gemma-3-1b-it-GGUF

3
Or start the OpenAI-compatible server on http://localhost:8080.
```
llama-server -hf ggml-org/gemma-3-1b-it-GGUF
```

Build from source

Clone and build (add -DGGML_CUDA=ON for NVIDIA).

git clone https://github.com/ggml-org/llama.cpp
cd llama.cpp
cmake -B build
cmake --build build --config Release

Strengths

Fastest CPU inference
Runs everywhere

Limitations

Lower-level than Ollama or LM Studio

llama.cpp alternatives compared

Software	Stars	License	VRAM
llama.cpp LLM inference in C/C++ — the engine behind most local AI apps.	131.2K	MIT	—
Ollama Get up and running with large language models locally.	183.2K	MIT	—
KoboldCpp Single-file llama.cpp-based runner for GGUF models with a story-writing UI.	12K	AGPL-3.0	—
llamafile Distribute and run an LLM as a single executable file.	26.3K	Apache-2.0	—
vLLM High-throughput LLM serving engine with PagedAttention.	93.8K	Apache-2.0	—
Jan Open-source ChatGPT-style desktop app that runs 100% offline.	45K	AGPL-3.0	—

FAQ

Is llama.cpp free?

Yes — llama.cpp is open-source software released under the MIT license. Check the license (and any model weight licenses) before commercial use.

Does llama.cpp run on Windows, Mac and Linux?

llama.cpp supports: Windows, macOS, Linux, Docker.

What hardware do I need for llama.cpp?

RAM: Depends on model size and quantization

How do I install llama.cpp?

1. Install with Homebrew (macOS/Linux) or winget (Windows). 2. Chat with a model from Hugging Face. 3. Or start the OpenAI-compatible server on http://localhost:8080.

What are the best llama.cpp alternatives?

Popular alternatives include Ollama, KoboldCpp, llamafile, vLLM. See the comparison table on this page.

Tools to run LLMs locally