Google open-sources speedy DiffusionGemma text diffusion model

Overview

Google released DiffusionGemma, an open-source large language model based on text diffusion, an emerging machine learning approach. The company says it generates text four times faster than traditional LLMs while using less RAM, allowing it to run on high-end consumer graphics cards. The model produces 256 tokens at once instead of one token at a time, which drives its speed. DiffusionGemma is available on Hugging Face under an open-source license.

Key Takeaways

DiffusionGemma is an open-source LLM based on text diffusion that Google says runs four times faster than traditional LLMs.
Its memory efficiency lets it run on high-end consumer graphics cards that usually struggle with LLMs.
The model generates 256 tokens at once rather than one at a time, which drives the speed gain.
It uses a mixture-of-experts design with 26 billion parameters but activates only 3.8 billion per prompt.
DiffusionGemma is available on Hugging Face under an open-source license.

Stats & Key Facts

#Generates text four times faster than traditional LLMs
#Produces 256 tokens at once
#More than 1,000 tokens per second on a single H100
#Over 700 tokens per second on a GeForce RTX 5090
#26 billion total parameters
#3.8 billion parameters activated per prompt

Google open-sources speedy DiffusionGemma text diffusion model

What DiffusionGemma is

Google released a text diffusion language model.

›DiffusionGemma is a large language model based on text diffusion, an emerging machine learning approach.
›Google says it generates text four times faster than traditional LLMs and uses less RAM.
›Its memory efficiency lets it run on high-end consumer graphics cards that usually struggle to support LLMs.

How text diffusion works

The architecture is derived from image generation methods.

›Image generation begins with a blurry photo containing Gaussian noise that a model gradually removes.
›When DiffusionGemma receives a prompt, it generates a placeholder response of random words.
›It then replaces a subset of the random text with real words, reviews the edits, and repeats until the response is ready.

Why it is faster

Parallel token generation drives the speed.

›Standard LLMs usually generate responses one token at a time.
›DiffusionGemma produces 256 tokens at once, and that parallelization makes it faster.
›Google says it can generate more than 1,000 tokens per second on a single H100 and over 700 tokens per second on a GeForce RTX 5090.

The H100 is a server-grade GPU that Nvidia launched in 2022, while the RTX 5090 is a desktop-grade chip.

Memory efficiency

Design choices lower RAM consumption.

›The model uses a mixture-of-experts architecture with 26 billion parameters but activates only 3.8 billion to answer a prompt.
›Activating fewer parameters lowers memory usage.
›It further lowers RAM consumption by keeping information in a lightweight data format called NVFP4.

Base model and changes

DiffusionGemma adapts an existing Gemma model.

›It is based on an LLM called Gemma 4 26B A4B that Google released in April.
›To enable text diffusion, Google replaced the model's attention mechanism.
›The original attention mechanism inferred meaning from preceding text, while the new module also reviews the text that follows a word.

Google research scientists Brendan O'Donoghue and Sebastian Flennerhag wrote that applying diffusion to large models has remained a challenge, and DiffusionGemma changes this by shifting how models use hardware. The model is available on Hugging Face under an open-source license.

Frequently Asked Questions

What is DiffusionGemma?

It is an open-source large language model from Google based on text diffusion, which Google says generates text four times faster than traditional LLMs while using less RAM.

Why is it faster than standard LLMs?

It produces 256 tokens at once instead of one token at a time, and that parallelization drives the speed gain.

What performance numbers did Google report?

Google says it generates more than 1,000 tokens per second on a single H100 and over 700 tokens per second on a GeForce RTX 5090.

How does it lower memory usage?

It uses a mixture-of-experts design that activates only 3.8 billion of its 26 billion parameters per prompt and stores information in a lightweight format called NVFP4.

Where can people get DiffusionGemma?

DiffusionGemma is available on Hugging Face under an open-source license.

DiffusionGemma applies text diffusion to bring faster, more memory-efficient generation to consumer hardware.