hugging-face-model-trainer

TRL Training on Hugging Face Jobs

Detailed Guide

Read [the detailed guide](references/detailed-guide.md) before executing this skill. It retains the complete procedure and reference material. Treat its safety, prerequisites, and validation requirements as mandatory. For focused work, load the relevant sections; for end-to-end work, read the guide completely.

When to Use This Skill

Use this skill when users want to:

• Fine-tune language models on cloud GPUs without local infrastructure
• Train with TRL methods (SFT, DPO, GRPO, etc.)
• Run training jobs on Hugging Face Jobs infrastructure
• Convert trained models to GGUF for local deployment (Ollama, LM Studio, llama.cpp)
• Ensure trained models are permanently saved to the Hub
• Use modern workflows with optimized defaults

When to Use Unsloth

Use Unsloth (references/unsloth.md) instead of standard TRL when:

• Limited GPU memory - Unsloth uses ~60% less VRAM
• Speed matters - Unsloth is ~2x faster
• Training large models (>13B) - memory efficiency is critical
• Training Vision-Language Models (VLMs) - Unsloth has FastVisionModel support

See references/unsloth.md for complete Unsloth documentation and scripts/unsloth_sft_example.py for a production-ready training script.

Prerequisites Checklist

Before starting any training job, verify:

✅ Account & Authentication

• Hugging Face Account with Pro, Team, or Enterprise plan (Jobs require paid plan)
• Authenticated login: Check with hf_whoami()
• HF_TOKEN for Hub Push ⚠️ CRITICAL - Training environment is ephemeral, must push to Hub or ALL training results are lost
• Token must have write permissions
• MUST pass secrets={"HF_TOKEN": "$HF_TOKEN"} in job config to make token available (the $HF_TOKEN syntax

references your actual token value)

✅ Dataset Requirements

• Dataset must exist on Hub or be loadable via datasets.load_dataset()
• Format must match training method (SFT: "messages"/text/prompt-completion; DPO: chosen/rejected; GRPO: prompt-only)
• ALWAYS validate unknown datasets before GPU training to prevent format failures (see Dataset Validation section below)
• Size appropriate for hardware (Demo: 50-100 examples on t4-small; Production: 1K-10K+ on a10g-large/a100-large)

⚠️ Critical Settings

• Timeout must exceed expected training time - Default 30min is TOO SHORT for most training. Minimum recommended: 1-2 hours. Job fails and loses all progress if timeout is exceeded.
• Hub push must be enabled - Config: push_to_hub=True, hub_model_id="username/model-name"; Job: secrets={"HF_TOKEN": "$HF_TOKEN"}

Example Training Scripts

Production-ready templates with all best practices:

Load these scripts for correctly:

• scripts/train_sft_example.py - Complete SFT training with Trackio, LoRA, checkpoints
• scripts/train_dpo_example.py - DPO training for preference learning
• scripts/train_grpo_example.py - GRPO training for online RL

These scripts demonstrate proper Hub saving, Trackio integration, checkpoint management, and optimized parameters. Pass their content inline to hf_jobs() or use as templates for custom scripts.

Limitations

• Use this skill only when the task clearly matches its upstream product or API scope.
• Verify commands, API behavior, pricing, quotas, credentials, and deployment effects against current official documentation before making changes.
• Do not treat generated examples as a substitute for environment-specific tests, security review, or user approval for destructive or costly actions.