本地大模型运行Apache-2.0Python

vLLM

High-throughput LLM serving engine with PagedAttention.

Star 数: 93.8K
Fork 数: 23.4K
未关闭 Issue: 8.9K
最近提交: 2026-10-15

数据来自 GitHub API · 更新于 2026-10-15

vLLM 是什么？

vLLM is the most widely used open-source engine for serving LLMs in production. PagedAttention and continuous batching deliver high throughput, with an OpenAI-compatible server and support for quantization, tensor parallelism and many model architectures.

主要功能

PagedAttention and continuous batching
OpenAI-compatible API server
Tensor/pipeline parallelism across GPUs
Quantization (AWQ, GPTQ, FP8)

系统要求

平台: Linux · Docker
显卡: NVIDIA (CUDA) primarily; AMD ROCm, TPU and others supported
Python: 3.9–3.12

如何安装 vLLM

根据项目 README 整理。开源项目更新很快，请以 GitHub 页面上的最新说明为准。

pip

1
Install.
```
pip install vllm
```
2
Serve a model on http://localhost:8000.
```
vllm serve Qwen/Qwen2.5-1.5B-Instruct
```

Docker

Run the OpenAI-compatible server.

docker run --gpus all -p 8000:8000 --ipc=host vllm/vllm-openai:latest --model Qwen/Qwen2.5-1.5B-Instruct

优势

Best throughput for serving
Production-proven

不足

Linux and data-center GPUs focus
Overkill for single users

vLLM 替代品对比

软件	Star 数	许可证	显存
vLLM High-throughput LLM serving engine with PagedAttention.	93.8K	Apache-2.0	—
SGLang Fast serving framework for LLMs and vision-language models.	37.2K	Apache-2.0	—
llama.cpp LLM inference in C/C++ — the engine behind most local AI apps.	131.2K	MIT	—
LocalAI Self-hosted, OpenAI-compatible API for text, images, audio and more.	49.5K	MIT	—
Ollama Get up and running with large language models locally.	183.2K	MIT	—
Jan Open-source ChatGPT-style desktop app that runs 100% offline.	45K	AGPL-3.0	—

常见问题

vLLM 免费吗？

是的——vLLM 是开源软件，采用 Apache-2.0 许可证。商用前请确认软件及模型权重的许可条款。

vLLM 支持 Windows、Mac 和 Linux 吗？

vLLM 支持：Linux, Docker。

运行 vLLM 需要什么配置？

NVIDIA (CUDA) primarily; AMD ROCm, TPU and others supported

如何安装 vLLM？

1. Install. 2. Serve a model on http://localhost:8000.

vLLM 有哪些好的替代品？

常见替代品包括 SGLang, llama.cpp, LocalAI, Ollama，详见本页对比表。

在本地运行大语言模型的工具