hugging-face-jobs

Running Workloads on Hugging Face Jobs

Detailed Guide

Read [the detailed guide](references/detailed-guide.md) before executing this skill. It retains the complete procedure and reference material. Treat its safety, prerequisites, and validation requirements as mandatory. For focused work, load the relevant sections; for end-to-end work, read the guide completely.

When to Use This Skill

Use this skill when users want to:

• Run Python workloads on cloud infrastructure
• Execute jobs without local GPU/TPU setup
• Process data at scale
• Run batch inference or experiments
• Schedule recurring tasks
• Use GPUs/TPUs for any workload
• Persist results to the Hugging Face Hub

Prerequisites Checklist

Before starting any job, verify:

✅ Account & Authentication

• Hugging Face Account with Pro, Team, or Enterprise plan (Jobs require paid plan)
• Authenticated login: Check with hf_whoami()
• HF_TOKEN for Hub Access ⚠️ CRITICAL - Required for any Hub operations (push models/datasets, download private repos, etc.)
• Token must have appropriate permissions (read for downloads, write for uploads)

✅ Token Usage (See Token Usage section for details)

When tokens are required:

• Pushing models/datasets to Hub
• Accessing private repositories
• Using Hub APIs in scripts
• Any authenticated Hub operations

How to provide tokens:

# hf_jobs MCP tool — $HF_TOKEN is auto-replaced with real token:
{"secrets": {"HF_TOKEN": "$HF_TOKEN"}}

# HfApi().run_uv_job() — MUST pass actual token:
from huggingface_hub import get_token
secrets={"HF_TOKEN": get_token()}

⚠️ CRITICAL: The $HF_TOKEN placeholder is ONLY auto-replaced by the hf_jobs MCP tool. When using HfApi().run_uv_job(), you MUST pass the real token via get_token(). Passing the literal string "$HF_TOKEN" results in a 9-character invalid token and 401 errors.

Token Usage Guide

Understanding Tokens

What are HF Tokens?

• Authentication credentials for Hugging Face Hub
• Required for authenticated operations (push, private repos, API access)
• Stored securely on your machine after hf auth login

Token Types:

• Read Token - Can download models/datasets, read private repos
• Write Token - Can push models/datasets, create repos, modify content
• Organization Token - Can act on behalf of an organization

When Tokens Are Required

Always Required:

• Pushing models/datasets to Hub
• Accessing private repositories
• Creating new repositories
• Modifying existing repositories
• Using Hub APIs programmatically

Not Required:

• Downloading public models/datasets
• Running jobs that don't interact with Hub
• Reading public repository information

How to Provide Tokens to Jobs

Method 1: Automatic Token (Recommended)

hf_jobs("uv", {
    "script": "your_script.py",
    "secrets": {"HF_TOKEN": "$HF_TOKEN"}  # ✅ Automatic replacement
})

How it works:

• $HF_TOKEN is a placeholder that gets replaced with your actual token
• Uses the token from your logged-in session (hf auth login)
• Most secure and convenient method
• Token is encrypted server-side when passed as a secret

Benefits:

• No token exposure in code
• Uses your current login session
• Automatically updated if you re-login
• Works seamlessly with MCP tools

Method 2: Explicit Token (Not Recommended)

hf_jobs("uv", {
    "script": "your_script.py",
    "secrets": {"HF_TOKEN": "hf_abc123..."}  # ⚠️ Hardcoded token
})

When to use:

• Only if automatic token doesn't work
• Testing with a specific token
• Organization tokens (use with caution)

Security concerns:

• Token visible in code/logs
• Must manually update if token rotates
• Risk of token exposure

Method 3: Environment Variable (Less Secure)

hf_jobs("uv", {
    "script": "your_script.py",
    "env": {"HF_TOKEN": "hf_abc123..."}  # ⚠️ Less secure than secrets
})

Difference from secrets:

• env variables are visible in job logs
• secrets are encrypted server-side
• Always prefer secrets for tokens

Using Tokens in Scripts

In your Python script, tokens are available as environment variables:

# /// script
# dependencies = ["huggingface-hub"]
# ///

import os
from huggingface_hub import HfApi

# Token is automatically available if passed via secrets
token = os.environ.get("HF_TOKEN")

# Use with Hub API
api = HfApi(token=token)

# Or let huggingface_hub auto-detect
api = HfApi()  # Automatically uses HF_TOKEN env var

Best practices:

• Don't hardcode tokens in scripts
• Use os.environ.get("HF_TOKEN") to access
• Let huggingface_hub auto-detect when possible
• Verify token exists before Hub operations

Token Verification

Check if you're logged in:

from huggingface_hub import whoami
user_info = whoami()  # Returns your username if authenticated

Verify token in job:

import os
assert "HF_TOKEN" in os.environ, "HF_TOKEN not found!"
token = os.environ["HF_TOKEN"]
print(f"Token starts with: {token[:7]}...")  # Should start with "hf_"

Common Token Issues

Error: 401 Unauthorized

• Cause: Token missing or invalid
• Fix: Add secrets={"HF_TOKEN": "$HF_TOKEN"} to job config
• Verify: Check hf_whoami() works locally

Error: 403 Forbidden

• Cause: Token lacks required permissions
• Fix: Ensure token has write permissions for push operations
• Check: Token type at https://huggingface.co/settings/tokens

Error: Token not found in environment

• Cause: secrets not passed or wrong key name
• Fix: Use secrets={"HF_TOKEN": "$HF_TOKEN"} (not env)
• Verify: Script checks os.environ.get("HF_TOKEN")

Error: Repository access denied

• Cause: Token doesn't have access to private repo
• Fix: Use token from account with access
• Check: Verify repo visibility and your permissions

Token Security Best Practices

1. Never commit tokens - Use $HF_TOKEN placeholder or environment variables
2. Use secrets, not env - Secrets are encrypted server-side
3. Rotate tokens regularly - Generate new tokens periodically
4. Use minimal permissions - Create tokens with only needed permissions
5. Don't share tokens - Each user should use their own token
6. Monitor token usage - Check token activity in Hub settings

Complete Token Example

# Example: Push results to Hub
hf_jobs("uv", {
    "script": """
# /// script
# dependencies = ["huggingface-hub", "datasets"]
# ///

import os
from huggingface_hub import HfApi
from datasets import Dataset

# Verify token is available
assert "HF_TOKEN" in os.environ, "HF_TOKEN required!"

# Use token for Hub operations
api = HfApi(token=os.environ["HF_TOKEN"])

# Create and push dataset
data = {"text": ["Hello", "World"]}
dataset = Dataset.from_dict(data)
dataset.push_to_hub("username/my-dataset", token=os.environ["HF_TOKEN"])

print("✅ Dataset pushed successfully!")
""",
    "flavor": "cpu-basic",
    "timeout": "30m",
    "secrets": {"HF_TOKEN": "$HF_TOKEN"}  # ✅ Token provided securely
})

Limitations

• Use this skill only when the task clearly matches the scope described above.
• Do not treat the output as a substitute for environment-specific validation, testing, or expert review.
• Stop and ask for clarification if required inputs, permissions, safety boundaries, or success criteria are missing.