How Trustpilot built a real-time architecture for data enrichment using Gemma

Overview

Google Cloud describes how Trustpilot built a real-time architecture to enrich millions of user reviews using fine-tuned Gemma models. Rather than relying on a closed frontier model, Trustpilot fine-tuned open-weight google/gemma-2-9b into a suite of specialized models for tasks like named entity recognition, topic classification, and sentiment extraction. The streaming pipeline runs on Dataflow and Agent Platform endpoints, decoupling business logic from raw LLM inference.

Key Takeaways

Trustpilot processes millions of user reviews in real time under strict latency and cost constraints.
It fine-tuned open-weight Gemma models instead of relying on a closed frontier model like Gemini.
Owning the models gives Trustpilot total model independence, predictable economics, expanded MLOps capabilities, and architectural continuity.
The team built a suite of specialized models on the lightweight google/gemma-2-9b base.
Training data came from consensus annotation using Gemini 2.0 and 2.5 Pro and Flash teacher models over a stratified sample of the review corpus.
The custom models outperformed the legacy solution and came within a couple percentage points of the teacher models' consensus accuracy.

Stats & Key Facts

#Millions of user reviews processed in real time
#Base model: google/gemma-2-9b
#Teacher models from the Gemini 2.0/2.5 Pro/Flash family
#Accuracy a couple percentage points below the teacher consensus
#A2 VMs using A100 GPUs

How Trustpilot built a real-time architecture for data enrichment using Gemma

The problem Trustpilot set out to solve

›Trustpilot's business relies on delivering deep, actionable review intelligence.
›It must safeguard data integrity and extract every drop of metadata from incoming reviews.
›LLMs parse messy human-written text to run named entity recognition, categorize business domains, score sentiment, and pinpoint customer intent.

The article notes that prompting an LLM for a few reviews is easy, but processing millions in real time without inflating costs is a major engineering hurdle.

Why fine-tune an open model

For a pipeline this critical, the post argues closed models are rarely the best option.

›Total model independence: Trustpilot controls the retraining lifecycle, free from a vendor's update schedule or sudden API changes.
›Predictable economics: fixed infrastructure costs replace variable per-token pricing, making millions of predictions viable.
›Expanding MLOps capabilities: building in-house lets Trustpilot bake in its review-intelligence secret sauce.
›Architectural continuity: standardizing on an open-weight lineage preserves the ability to use future base-model iterations.

Building the model suite

›Trustpilot built a suite of specialized models rather than one massive model.
›The base was the lightweight google/gemma-2-9b.
›Consensus annotation over a stratified sample used teacher models from the Gemini 2.0/2.5 Pro/Flash family.
›This produced high-quality datasets for topic classification, NER, and sentiment extraction.

The fine-tuned custom models considerably outperformed the legacy solution and delivered accuracy just a couple percentage points lower than the teacher models' consensus.

System architecture

The architecture was built on Dataflow and Gemini Enterprise Agent Platform Endpoints.

›The pairing works well because of the out-of-the-box VertexAIModelHandlerJSON.
›The classifier is a FastAPI-based endpoint handling pre and post-processing, prompt templating, and chaining.
›The LLM is a separate Agent Platform endpoint dedicated to serving the Gemma model via vLLM.
›Decoupling keeps the Dataflow job clean and lets the two endpoints scale independently with traffic.

Performance tuning

›Trustpilot focused on squeezing performance from the whole pipeline.
›It emphasized A2 VMs using A100 GPUs.
›It used the customized, optimized version of vLLM maintained by the Gemini Enterprise Agent Platform.

Frequently Asked Questions

Why did Trustpilot fine-tune open models instead of using a closed frontier model?

For total model independence, predictable economics, expanded MLOps capabilities, and architectural continuity, which the post says closed models rarely provide for a business-critical pipeline.

Which base model did Trustpilot use?

It used the lightweight open-weight google/gemma-2-9b as the base for a suite of specialized models.

How was the training data created?

Through consensus annotation over a stratified sample of the review corpus, using teacher models from the Gemini 2.0/2.5 Pro and Flash family.

How accurate are the custom models?

They considerably outperformed the legacy solution and came within a couple percentage points of the teacher models' consensus accuracy.

What does the system architecture run on?

It runs on Dataflow and Gemini Enterprise Agent Platform Endpoints, with a FastAPI classifier endpoint separate from a Gemma-serving LLM endpoint using vLLM.

Trustpilot's pipeline shows how fine-tuned open-weight Gemma models, served on Dataflow and Agent Platform endpoints, can enrich millions of reviews in real time at predictable cost.

Why It Matters for Business

Real business deployments are the most reliable signal of where AI is generating measurable ROI. Watching which sectors operationalize AI, what they pay for it, and how it changes their P&L tells you more than any vendor demo. These case studies are what serious buyers and investors triangulate on.