TRL
Hugging Face’s library for post-training LLMs: SFT, DPO, GRPO.
- Star 数
- 19.5K
- Fork 数
- 3.1K
- 未关闭 Issue
- 267
- 最近提交
- 2026-10-15
数据来自 GitHub API · 更新于 2026-10-15
TRL 是什么？
TRL (Transformer Reinforcement Learning) provides trainers for supervised fine-tuning, DPO, GRPO, reward modeling and more, integrated with Transformers, PEFT and Accelerate.
主要功能
- SFT, DPO, GRPO, PPO trainers
- PEFT/LoRA integration
- CLI for quick runs
系统要求
- 平台
- Linux · Windows · macOS
- Python
- 3.9+
如何安装 TRL
根据项目 README 整理。开源项目更新很快，请以 GitHub 页面上的最新说明为准。
pip
- 1Install.
pip install trl - 2Run supervised fine-tuning from the CLI.
trl sft --model_name_or_path Qwen/Qwen2.5-0.5B --dataset_name trl-lib/Capybara --output_dir Qwen2.5-0.5B-SFT
优势
- Reference implementations
- Tight Hugging Face integration
不足
- Code-first
TRL 替代品对比
| 软件 | Star 数 | 许可证 | 平台 | 显存 |
|---|---|---|---|---|
| TRL Hugging Face’s library for post-training LLMs: SFT, DPO, GRPO. | 19.5K | Apache-2.0 | — | |
| Unsloth Fine-tune LLMs 2× faster with up to 70% less memory. | 78K | Apache-2.0 | 3 GB+ depending on model size | |
| Axolotl Config-driven LLM fine-tuning for many models and techniques. | 12.6K | Apache-2.0 | — | |
| LLaMA-Factory Unified fine-tuning for 100+ LLMs and VLMs with a no-code web UI. | 75.5K | Apache-2.0 | — | |
| Kohya’s GUI GUI for training LoRA, DreamBooth and fine-tunes for SD, SDXL and FLUX. | 12.6K | Apache-2.0 | 8 GB+ for SD1.5 LoRA; 12–24 GB for SDXL/FLUX | |
| OneTrainer One-stop trainer for diffusion models with a desktop UI. | 3.2K | AGPL-3.0 | 8 GB+ |
常见问题
TRL 免费吗？
是的——TRL 是开源软件，采用 Apache-2.0 许可证。商用前请确认软件及模型权重的许可条款。
TRL 支持 Windows、Mac 和 Linux 吗？
TRL 支持：Linux, Windows, macOS。
如何安装 TRL？
1. Install. 2. Run supervised fine-tuning from the CLI.
TRL 有哪些好的替代品？
常见替代品包括 Unsloth, Axolotl, LLaMA-Factory, Kohya’s GUI，详见本页对比表。