本地大模型运行MITC++

llama.cpp

LLM inference in C/C++ — the engine behind most local AI apps.

Star 数: 131.2K
Fork 数: 24.4K
未关闭 Issue: 2.6K
最近提交: 2026-10-15

数据来自 GitHub API · 更新于 2026-10-15

llama.cpp 是什么？

llama.cpp runs GGUF-quantized models efficiently on CPUs and GPUs (CUDA, Metal, Vulkan, ROCm, SYCL). It includes a CLI, an OpenAI-compatible server and tools for converting and quantizing models.

主要功能

GGUF quantization from 1.5 to 8 bits
OpenAI-compatible llama-server with a web UI
Wide GPU backend support
Download models directly from Hugging Face with -hf

系统要求

平台: Windows · macOS · Linux · Docker
内存: Depends on model size and quantization

如何安装 llama.cpp

根据项目 README 整理。开源项目更新很快，请以 GitHub 页面上的最新说明为准。

Package managers

1
Install with Homebrew (macOS/Linux) or winget (Windows).
```
brew install llama.cpp
# or
winget install llama.cpp
```

Chat with a model from Hugging Face.

llama-cli -hf ggml-org/gemma-3-1b-it-GGUF

3
Or start the OpenAI-compatible server on http://localhost:8080.
```
llama-server -hf ggml-org/gemma-3-1b-it-GGUF
```

Build from source

Clone and build (add -DGGML_CUDA=ON for NVIDIA).

git clone https://github.com/ggml-org/llama.cpp
cd llama.cpp
cmake -B build
cmake --build build --config Release

优势

Fastest CPU inference
Runs everywhere

不足

Lower-level than Ollama or LM Studio

llama.cpp 替代品对比

软件	Star 数	许可证	显存
llama.cpp LLM inference in C/C++ — the engine behind most local AI apps.	131.2K	MIT	—
Ollama Get up and running with large language models locally.	183.2K	MIT	—
KoboldCpp Single-file llama.cpp-based runner for GGUF models with a story-writing UI.	12K	AGPL-3.0	—
llamafile Distribute and run an LLM as a single executable file.	26.3K	Apache-2.0	—
vLLM High-throughput LLM serving engine with PagedAttention.	93.8K	Apache-2.0	—
Jan Open-source ChatGPT-style desktop app that runs 100% offline.	45K	AGPL-3.0	—

常见问题

llama.cpp 免费吗？

是的——llama.cpp 是开源软件，采用 MIT 许可证。商用前请确认软件及模型权重的许可条款。

llama.cpp 支持 Windows、Mac 和 Linux 吗？

llama.cpp 支持：Windows, macOS, Linux, Docker。

运行 llama.cpp 需要什么配置？

RAM: Depends on model size and quantization

如何安装 llama.cpp？

1. Install with Homebrew (macOS/Linux) or winget (Windows). 2. Chat with a model from Hugging Face. 3. Or start the OpenAI-compatible server on http://localhost:8080.

llama.cpp 有哪些好的替代品？

常见替代品包括 Ollama, KoboldCpp, llamafile, vLLM，详见本页对比表。

在本地运行大语言模型的工具