0
Local LLM runnersApache-2.0Python

vLLM

High-throughput LLM serving engine with PagedAttention.

GitHubDocs
Stars
93.8K
Forks
23.4K
Open issues
8.9K
Last commit
2026-10-15

Live from the GitHub API · updated 2026-10-15

What is vLLM?

vLLM is the most widely used open-source engine for serving LLMs in production. PagedAttention and continuous batching deliver high throughput, with an OpenAI-compatible server and support for quantization, tensor parallelism and many model architectures.

Key features

  • PagedAttention and continuous batching
  • OpenAI-compatible API server
  • Tensor/pipeline parallelism across GPUs
  • Quantization (AWQ, GPTQ, FP8)

System requirements

Platforms
Linux · Docker
GPU
NVIDIA (CUDA) primarily; AMD ROCm, TPU and others supported
Python
3.9–3.12

How to install vLLM

Summarized from the project’s README. Projects change quickly — check the GitHub page for the latest instructions.

pip

  1. 1
    Install.
    pip install vllm
  2. 2
    Serve a model on http://localhost:8000.
    vllm serve Qwen/Qwen2.5-1.5B-Instruct

Docker

  1. 1
    Run the OpenAI-compatible server.
    docker run --gpus all -p 8000:8000 --ipc=host vllm/vllm-openai:latest --model Qwen/Qwen2.5-1.5B-Instruct

Strengths

  • Best throughput for serving
  • Production-proven

Limitations

  • Linux and data-center GPUs focus
  • Overkill for single users

vLLM alternatives compared

SoftwareStarsLicensePlatformsVRAM
vLLM

High-throughput LLM serving engine with PagedAttention.

93.8KApache-2.0—
SGLang

Fast serving framework for LLMs and vision-language models.

37.2KApache-2.0—
llama.cpp

LLM inference in C/C++ — the engine behind most local AI apps.

131.2KMIT—
LocalAI

Self-hosted, OpenAI-compatible API for text, images, audio and more.

49.5KMIT—
Ollama

Get up and running with large language models locally.

183.2KMIT—
Jan

Open-source ChatGPT-style desktop app that runs 100% offline.

45KAGPL-3.0—

FAQ

Is vLLM free?

Yes — vLLM is open-source software released under the Apache-2.0 license. Check the license (and any model weight licenses) before commercial use.

Does vLLM run on Windows, Mac and Linux?

vLLM supports: Linux, Docker.

What hardware do I need for vLLM?

NVIDIA (CUDA) primarily; AMD ROCm, TPU and others supported

How do I install vLLM?

1. Install. 2. Serve a model on http://localhost:8000.

What are the best vLLM alternatives?

Popular alternatives include SGLang, llama.cpp, LocalAI, Ollama. See the comparison table on this page.

Tools to run LLMs locally