Train Hugging Face models on cloud GPUs

VaultLayer runs Hugging Face training — Transformers Trainer, TRL, Accelerate, and PEFT/QLoRA — on cloud GPUs without code changes. Checkpoints auto-save, and single-node jobs resume from the last step automatically after an interruption — no code changes.

Run Trainer or TRL unchanged

The default training image ships Transformers, Accelerate, PEFT, and TRL, so your script runs as-is:

vl run train.py

No SDK, no decorators, no container conventions to learn — VaultLayer wraps the command you already run.

Resume is automatic

The Hugging Face Trainer already writes checkpoints, and on single-node runs VaultLayer resumes from the last one automatically when a job is interrupted — no code changes:

vl run train.py

VaultLayer syncs those checkpoints to durable storage and re-provisions on failure — see how checkpoint & resume works. Multi-node jobs pass resume_from_checkpoint=os.environ.get("VAULTLAYER_RESUME_CHECKPOINT") explicitly.

LoRA, QLoRA, and full fine-tunes

Run any fine-tuning mode with --train-mode qlora|lora|full, sized to your model. For choosing a GPU by model size, see GPU types for training; for the broader workflow, fine-tune LLMs on your own cloud.

Frequently asked questions