11 个工具
在本地运行大语言模型的工具
在你自己的电脑上运行 Llama、Qwen、Mistral 等开源模型的推理引擎和桌面应用。
OllamaGet up and running with large language models locally.MITllama.cppLLM inference in C/C++ — the engine behind most local AI apps.MITJanOpen-source ChatGPT-style desktop app that runs 100% offline.AGPL-3.0GPT4AllPrivate, on-device chat with local document Q&A.MITLocalAISelf-hosted, OpenAI-compatible API for text, images, audio and more.MITvLLMHigh-throughput LLM serving engine with PagedAttention.Apache-2.0SGLangFast serving framework for LLMs and vision-language models.Apache-2.0Text Generation WebUIThe “AUTOMATIC1111 of text generation” — a Gradio UI for local LLMs.AGPL-3.0KoboldCppSingle-file llama.cpp-based runner for GGUF models with a story-writing UI.AGPL-3.0llamafileDistribute and run an LLM as a single executable file.Apache-2.0MLX LMRun and fine-tune LLMs on Apple Silicon with Apple’s MLX framework.MIT