11 tools
Tools to run LLMs locally
Inference engines and desktop apps that run Llama, Qwen, Mistral and other open models on your machine.
OllamaGet up and running with large language models locally.MITllama.cppLLM inference in C/C++ — the engine behind most local AI apps.MITJanOpen-source ChatGPT-style desktop app that runs 100% offline.AGPL-3.0GPT4AllPrivate, on-device chat with local document Q&A.MITLocalAISelf-hosted, OpenAI-compatible API for text, images, audio and more.MITvLLMHigh-throughput LLM serving engine with PagedAttention.Apache-2.0SGLangFast serving framework for LLMs and vision-language models.Apache-2.0Text Generation WebUIThe “AUTOMATIC1111 of text generation” — a Gradio UI for local LLMs.AGPL-3.0KoboldCppSingle-file llama.cpp-based runner for GGUF models with a story-writing UI.AGPL-3.0llamafileDistribute and run an LLM as a single executable file.Apache-2.0MLX LMRun and fine-tune LLMs on Apple Silicon with Apple’s MLX framework.MIT