Native-speed vLLM transformers modeling backend
We're on a journey to advance and democratize artificial intelligence through open source and open science. Back to Articles a]:hidden"> Native-speed vLLM transformers modeling backend Published July 8, 2026 Update on GitHub Upvote 11 +5 Harry Mellor hmellor Follow Lysandre lysandre Follow TL;DR : The transformers vLLM backend is now as fast (or faster) than custom vLLM implementations for many LLM architectures. Model authors can automatically leverage their transformers implementations to get ultra fast vLLM inference, for free.
Key Takeaways
- The transformers library has become the reference modeling library for Machine Learning.
It supports 450+ architectures through consistent APIs, and is designed with the main goal that model implementations are self contained and easy to understand .
- This has been allowing model authors to run transformers models (LLMs and VLMs alike) inside vLLM, without having to port anything.
Transformers provides the modeling code, and vLLM provides extremely optimized inference techniques such as continuous batching and custom attention kernels.
- Custom models where the code lives in a Hub repo are unlikely to work as they will not have been written compliantly.
- Parallelization across GPUs, compilation, fused kernels, and many more, all contribute to leveraging your hardware to achieve ultra-fast inference.
A new model used to be integrated once for transformers, and once for vLLM with custom optimizations When model authors wanted the absolute best performance, they were still writing custom vLLM implementations.
- After any patterns have been identified, it uses ast (abstract syntax tree) to manipulate the source code and rewrite some of the operations in place.
Stats & Key Facts
- #It supports 450+ architectures through consistent APIs, and is designed with the main goal that model implementations are self contained and easy to understand .
The transformers library has become the reference modeling library for Machine Learning. It supports 450+ architectures through consistent APIs, and is designed with the main goal that model implementations are self contained and easy to understand . Going through transformers code makes it easy for contributors to learn how an architecture works, and then port it to other frameworks such as vLLM, SGLang, MLX, llama.
We have fully embraced this role in the ecosystem and are investing a lot of effort to make it easier. A big step in this direction was the integration last year of transformers as a modeling backend in vLLM. This has been allowing model authors to run transformers models (LLMs and VLMs alike) inside vLLM, without having to port anything.
Transformers provides the modeling code, and vLLM provides extremely optimized inference techniques such as continuous batching and custom attention kernels. This integration gets better now 🚀! Running any* Hugging Face model through the transformers modeling backend is a single flag - .
For more details please read the original article at Hugging Face.
Continue Learning
Comments
Sign in to join the conversation