Fine-tune video and image models at scale with NVIDIA NeMo Automodel and 🤗 Diffusers
Special thanks to Sayak Paul from Hugging Face for their contributions to the integration work and for co-authoring this blog. Diffusion models power some of the most exciting open-source releases of the last two years - such as FLUX. 1-dev for text-to-image and Wan 2.
Key Takeaways
- 1 and HunyuanVideo for text-to-video.
The 🤗 Diffusers library has become the de facto home for these models, giving researchers and builders a single, consistent interface for inference, adaptation, and pipeline composition.
- Supported diffusion models What this collaboration unlocks A look at the fine-tuning workflow 1.
- Checkpoints round-trip cleanly back into the Diffusers ecosystem.
The recipes and training scripts can be easily modified to suit training at any scale.
- Model Hugging Face ID Task Parameters LoRA recipe Wan 2.
- Your fine-tuned checkpoint loads directly into a for inference, or back to the Hub for sharing.
1 and HunyuanVideo for text-to-video. The 🤗 Diffusers library has become the de facto home for these models, giving researchers and builders a single, consistent interface for inference, adaptation, and pipeline composition. In addition, training and fine-tuning diffusion models are also on the rise, requiring utilities that offer memory-efficient sharding, latent caching, multiresolution bucketing, and configurations that scale gracefully from one GPU to hundreds.
To cater to these technical demands, we offer the NVIDIA NeMo Automodel open-source library. Today, we're highlighting the collaboration between NVIDIA and Hugging Face that brings production-grade, distributed diffusion training to any Diffusers-format model on the Hugging Face Hub - with no checkpoint conversion and no model rewrites for any new model. The integration is documented in the Diffusers training guide and is fully open source under Apache 2.
Generate from the fine-tuned checkpoint 4. Performance Other Finetuned/LoRA examples Try it today Coming next: Pythonic recipe APIs Resources What is NeMo Automodel? NeMo Automodel is an open-source PyTorch DTensor-native training library, part of the NVIDIA NeMo framework, built around two design principles that matter for the Diffusers ecosystem: Hugging Face native.
For more details please read the original article at Hugging Face.
Continue Learning
Comments
Sign in to join the conversation