Skip to main content
AI Learning Resources

Quick Overview

This video is a behind-the-scenes documentary case study produced by Google Cloud Tech. It follows a team of Google Developer Experts at Sonoma Raceway as they design, deploy, and refine a real-time AI race coach directly in the field using Google Antigravity and on-device models.

Key Points

  • 1.Google Developer Experts collaborated at Sonoma Raceway to build an on-device AI race coach capable of delivering low-latency real-time guidance.
  • 2.Because cloud connectivity was unavailable on the racetrack, the team deployed an open-weight Gemma model running directly on an onboard device Tensor Processing Unit.
  • 3.Vehicle telemetry including GPS, accelerometer, pedal position, and steering angle was routed through a CAN to USB converter into the AI model.
  • 4.The engineering team used Google Antigravity to rapidly build, debug, and iterate on the application directly trackside between 20-minute driving sessions.
  • 5.After initial issues with excessive chatter and mismatched track positioning were resolved, the AI coach successfully provided timely throttle prompts that increased the driver lap speed.

Summary

At Sonoma Raceway, a technical circuit characterized by tight turns and substantial elevation changes, a group of Google Developer Experts gathered to build an interactive AI racecar coach. Driver and Google Developer Experts Americas Lead AJ Mirwani framed the central challenge: creating an AI coaching assistant that a driver could safely rely on at speeds over 100 miles per hour. The team viewed motorsport as an extreme proving ground for real-time AI trustworthiness, offering insights that could transfer to critical fields like medicine and environmental science.

Developing for a moving racecar introduced strict technical constraints. Split-second latency was crucial, and the absence of trackside Wi-Fi ruled out calling cloud endpoints like Gemini. To resolve this, the engineers fine-tuned Google's open-weight Gemma model to run entirely on-device using the Tensor Processing Unit of an onboard Pixel device. To feed the model real-time context, engineer Brian Luc and the team used a CAN to USB converter to extract telemetry from the vehicle, including GPS coordinates, accelerometer readings, pedal positions, and steering wheel angles. This telemetry was merged with coach data in a structured context file to perform on-device inference for pre-race briefings, real-time prompts, and post-session evaluations.

During track testing, team member Simon Margolis noted that Google Antigravity helped bridge domain knowledge gaps by guiding the integration of vehicle data streams, allowing developers to focus on application logic. The initial track run presented practical hurdles: the AI coach spoke continuously without pause, overwhelming the driver with chatter, and misidentified the driver's track location by calling out Turn 3 advice while the car was still navigating Turn 1. Developer Vikram Tiwari used the twenty-minute intervals between sessions to investigate track coordinates, correct corner mapping, and deploy updated builds directly trackside.

Over numerous iterative cycles, the software evolved from testing to reliable execution on the track. The refined Gemma model quieted down, timing its guidance to precise cornering moments. When the system prompted Mirwani to apply throttle at a specific exit point, the recommendation proved effective, leading to faster sector times. The project demonstrated that using Antigravity and local edge models allows developers to rapidly engineer trustworthy, high-performance AI applications under challenging physical constraints.

The Challenge of Real-Time Trackside AI

Google Developer Experts set out to test whether an artificial intelligence system could be built in a short time frame and trusted by a driver moving at over 100 miles per hour. Operating on a demanding course like Sonoma Raceway required real-time coaching where latency of even a split second could impact safety, demonstrating capabilities beyond standard chatbot interfaces.

Edge Architecture and Telemetry Integration

Due to the absence of reliable Wi-Fi on the racetrack, the team bypassed cloud-dependent models like Gemini and fine-tuned Google's open-weight Gemma model to execute locally on a Pixel device using its onboard TPU. Vehicle sensors capturing steering angle, pedal positions, accelerometer readings, and GPS coordinates fed telemetry into the model via a CAN to USB converter alongside expert coach context files.

Rapid Iteration and Track Testing with Antigravity

Using Google Antigravity, developers rapidly iterated on the software directly at the track. Early test runs revealed problems with excessive voice chatter and out-of-sync corner tracking, which the team diagnosed and patched within twenty-minute intervals between runs. As the model adapted to track data, it delivered concise, high-confidence audio cues that helped driver AJ Mirwani improve lap speed.

The Bottom Line

The project establishes that lightweight, open-weight models such as Gemma can run locally on mobile hardware to provide accurate, sub-second guidance in high-stakes environments without cloud connectivity. By using Google Antigravity, the engineering team demonstrated how rapid trackside development and telemetry integration can overcome real-time communication challenges. The experiment proves the viability of edge AI coaching and provides a foundation for extending low-latency AI frameworks into other mission-critical domains.

FAQ

What is Google Antigravity and what was its role in building the AI racecar coach?

Google Antigravity is a development tool used by the Google Developer Experts to rapidly build, debug, and iterate on the AI coaching application trackside, assisting with complex data integration tasks like extracting vehicle telemetry.

Why did the engineering team choose to fine-tune Gemma instead of using Gemini?

Gemini requires cloud connectivity and internet access, which was not available on the racetrack. The team fine-tuned Google's open-weight Gemma model so it could run locally on a device's onboard TPU without network latency or reliance on Wi-Fi.

How was vehicle telemetry collected from the racecar for the AI model to analyze?

The team connected a CAN to USB converter to the car's system, allowing the application to capture sensor data including GPS, accelerometer figures, pedal positions, and steering wheel angles.

What issues occurred during the initial on-track test of the AI racecar coach?

The initial version spoke continuously and proved distracting to the driver, while also misidentifying track locations by giving turn guidance for Turn 3 when the car was only at Turn 1.

At which racing venue did the Google Developer Experts conduct their field test?

The team conducted their development and field testing at Sonoma Raceway, chosen for its technical turns and significant elevation changes.

Worth watching for

Software developers, machine learning engineers, and automotive tech enthusiasts interested in edge AI deployment, local model fine-tuning, and real-time telemetry processing.

  • google-antigravity
  • gemma
  • edge-ai
  • racing-telemetry
  • google-developer-experts
Watch on YouTubeMore videos from Google Cloud Tech