How Trustpilot built a real-time architecture for data enrichment using Gemma

Overview

Trustpilot has developed a real-time data enrichment architecture using fine-tuned Gemma models to process millions of user reviews efficiently while controlling costs and latency. This transition to generative AI allows Trustpilot to enhance its review intelligence capabilities, ensuring data integrity and maximizing value from incoming reviews.

Key Takeaways

Trustpilot processes millions of reviews in real-time, utilizing fine-tuned Gemma models for enhanced data enrichment.
By fine-tuning open-weight models, Trustpilot gains total independence over its AI strategy and avoids reliance on third-party vendors.
The architecture leverages Dataflow and Gemini Enterprise Agent Platform for efficient model deployment and scalability.
Trustpilot's approach includes building specialized models that outperform legacy solutions while maintaining cost-effectiveness.
Performance tuning focuses on optimizing the pipeline, particularly using A100 GPUs for maximum efficiency.

Stats & Key Facts

#Trustpilot processes millions of user reviews in real-time.
#The accuracy of the custom models is just a couple percentage points lower than the teacher models' consensus.

How Trustpilot built a real-time architecture for data enrichment using Gemma

The Challenge of Real-Time Review Processing

Processing user reviews efficiently is crucial for Trustpilot's business model.

›Trustpilot handles millions of reviews while maintaining strict latency and cost constraints.
›The need for deep review intelligence drives the architecture's design.

In an era where user feedback is paramount, Trustpilot's ability to process reviews in real-time is essential. The company has established itself as a leader in transparency and genuine feedback, necessitating a robust system that can handle vast amounts of data without compromising quality.

Why Fine-Tune Open Models?

Trustpilot's decision to fine-tune models stems from the need for control and cost-effectiveness.

›Fine-tuning open models allows for total independence from third-party updates.
›Predictable infrastructure costs make it financially viable to run millions of predictions.
›In-house model development enhances Trustpilot's MLOps capabilities.

By opting for fine-tuned open-weight models like Gemma, Trustpilot ensures that it retains full control over its AI strategy. This independence is critical for a business that relies heavily on the accuracy and reliability of its review processing systems.

Building a Specialized Model Suite

Trustpilot's architecture includes a suite of specialized models tailored for various tasks.

›The company uses a lightweight Gemma model as a base for its custom solutions.
›High-quality training datasets are created through consensus annotation of the review corpus.
›Specialized tasks include topic classification, named entity recognition (NER), and sentiment extraction.

Instead of deploying a single large model, Trustpilot has developed a suite of specialized models that excel in specific tasks. This approach not only improves performance but also allows for greater flexibility and scalability in handling user reviews.

System Architecture Overview

The architecture is designed for efficiency and scalability.

›Built on Dataflow and Gemini Enterprise Agent Platform Endpoints.
›Decoupled business logic from raw LLM inference for cleaner processing.
›Separate endpoints for classifiers and LLMs enhance operational efficiency.

Trustpilot's system architecture is strategically designed to maximize efficiency. By separating business logic from the raw inference processes, the company can maintain a clean and effective pipeline that scales according to traffic demands.

Performance Tuning and Optimization

Maximizing performance is a key focus for Trustpilot's architecture.

›Utilizes A2 VMs with A100 GPUs for enhanced processing power.
›Optimized versions of vLLM are leveraged for better performance.
›Continuous tuning ensures the pipeline operates at peak efficiency.

To ensure that the architecture operates at its best, Trustpilot is committed to performance tuning. By utilizing powerful hardware and optimized software solutions, the company can deliver real-time results without sacrificing quality.

Frequently Asked Questions

What is Trustpilot's main business focus?

Trustpilot focuses on delivering deep, actionable review intelligence to promote transparency and genuine feedback.

Why did Trustpilot choose to fine-tune open models instead of using off-the-shelf models?

Fine-tuning open models allows Trustpilot to maintain total independence, control costs, and enhance its MLOps capabilities.

How does Trustpilot ensure the accuracy of its models?

Trustpilot employs a consensus annotation process over a stratified sample of its review corpus to create high-quality training datasets.

What technologies support Trustpilot's architecture?

The architecture is built on Dataflow and Gemini Enterprise Agent Platform, utilizing separate endpoints for classifiers and LLMs.

What kind of performance optimizations does Trustpilot implement?

Trustpilot focuses on leveraging A100 GPUs and optimized software versions to maximize efficiency and performance across its pipeline.

Trustpilot's innovative approach sets a new standard in real-time data processing.

Why It Matters for Business

Real business deployments are the most reliable signal of where AI is generating measurable ROI. Watching which sectors operationalize AI, what they pay for it, and how it changes their P&L tells you more than any vendor demo. These case studies are what serious buyers and investors triangulate on.