TRL
Hugging Face’s library for post-training LLMs: SFT, DPO, GRPO.
- Stars
- 19.5K
- Forks
- 3.1K
- Open issues
- 267
- Last commit
- 2026-10-15
Live from the GitHub API · updated 2026-10-15
What is TRL?
TRL (Transformer Reinforcement Learning) provides trainers for supervised fine-tuning, DPO, GRPO, reward modeling and more, integrated with Transformers, PEFT and Accelerate.
Key features
- SFT, DPO, GRPO, PPO trainers
- PEFT/LoRA integration
- CLI for quick runs
System requirements
- Platforms
- Linux · Windows · macOS
- Python
- 3.9+
How to install TRL
Summarized from the project’s README. Projects change quickly — check the GitHub page for the latest instructions.
pip
- 1Install.
pip install trl - 2Run supervised fine-tuning from the CLI.
trl sft --model_name_or_path Qwen/Qwen2.5-0.5B --dataset_name trl-lib/Capybara --output_dir Qwen2.5-0.5B-SFT
Strengths
- Reference implementations
- Tight Hugging Face integration
Limitations
- Code-first
TRL alternatives compared
| Software | Stars | License | Platforms | VRAM |
|---|---|---|---|---|
| TRL Hugging Face’s library for post-training LLMs: SFT, DPO, GRPO. | 19.5K | Apache-2.0 | — | |
| Unsloth Fine-tune LLMs 2× faster with up to 70% less memory. | 78K | Apache-2.0 | 3 GB+ depending on model size | |
| Axolotl Config-driven LLM fine-tuning for many models and techniques. | 12.6K | Apache-2.0 | — | |
| LLaMA-Factory Unified fine-tuning for 100+ LLMs and VLMs with a no-code web UI. | 75.5K | Apache-2.0 | — | |
| Kohya’s GUI GUI for training LoRA, DreamBooth and fine-tunes for SD, SDXL and FLUX. | 12.6K | Apache-2.0 | 8 GB+ for SD1.5 LoRA; 12–24 GB for SDXL/FLUX | |
| OneTrainer One-stop trainer for diffusion models with a desktop UI. | 3.2K | AGPL-3.0 | 8 GB+ |
FAQ
Is TRL free?
Yes — TRL is open-source software released under the Apache-2.0 license. Check the license (and any model weight licenses) before commercial use.
Does TRL run on Windows, Mac and Linux?
TRL supports: Linux, Windows, macOS.
How do I install TRL?
1. Install. 2. Run supervised fine-tuning from the CLI.
What are the best TRL alternatives?
Popular alternatives include Unsloth, Axolotl, LLaMA-Factory, Kohya’s GUI. See the comparison table on this page.