Skip to main content

Quick Overview

Presented at OpenAI DevDay, this technical talk introduces GPT-Live, OpenAI's real-time full-duplex voice model. Speakers Alex Gamble and Erika Kettleson walk through live system demos with a conversational AI co-host to demonstrate voice steering, tool delegation, and backend integration patterns.

Key Points

  • 1.GPT-Live is a full-duplex voice model capable of streaming audio in and out simultaneously without requiring a separate voice activity detection model.
  • 2.Conversational characteristics such as tone, pacing, accents, interruption tolerance, and backchannel reactions are fully steerable through natural language prompting.
  • 3.Running secondary GPT-Live sessions enables multi-speaker interactions and provides a realistic environment for automated voice evaluations.
  • 4.GPT-Live supports two delegation modes: client delegation, which passes tasks to an existing backend or agent harness, and responses delegation, which lets OpenAI manage execution.
  • 5.Backend systems communicate updates to GPT-Live using three distinct channels: silent thinking context, speakable commentary text, and real-time system instructions.
  • 6.Because turn-taking is integrated directly into the model rather than relying on silence heuristics, GPT-Live can maintain conversational flow in noisy environments.

Summary

Alex Gamble and Erika Kettleson open the presentation at OpenAI DevDay by introducing GPT-Live 1, an API model designed for real-time, interactive voice experiences. They are joined by Marin, a live voice agent running on GPT-Live that co-hosts the talk without relying on manual keyboard input. Alex explains that GPT-Live is a full-duplex model, meaning it continuously streams audio in and out simultaneously. This allows natural conversational dynamics, including interruptions and overlaps, while eliminating the need for standalone voice activity detection components to determine when a user has finished speaking.

To showcase personality control and multi-agent interaction, Marin invites Colin, a second GPT-Live session configured with a dry British persona, onto the stage. The hosts and the two agents play a storytelling game alternating between positive and negative developments. Erika explains that running a secondary voice session is not only valuable for multi-character experiences, but also serves as an automated evaluation strategy. Developers can configure simulated user sessions with varied personalities and goals to benchmark how voice applications handle challenging conversations.

Erika and Alex next address delegation and application architecture through an interactive drawing game. While Marin keeps up a live conversation with the audience, background tasks run asynchronously using tool calling and code execution to sketch objects on screen. When audience members guess items such as a kite, a giraffe, and a carousel, the backend registers state updates without halting the ongoing dialogue. The presenters detail three channels used to update the voice model during delegation: the thinking stream for silent context updates, the commentary stream for text the model can read aloud or paraphrase, and the instruction stream for trusted system guardrails.

The talk examines two implementation pathways for delegation: client delegation, which forwards events to a developer's proprietary backend or existing agent harness, and responses delegation, where OpenAI hosts and manages model responses. In the final demonstration, the presenters test GPT-Live in an environment with high ambient sound by playing background audio through the speakers. Marin successfully interprets instructions, adjusts audio volume, and alters the music style without losing track of speech input or confusing the soundtrack for user speech. The presentation concludes with Marin triggering OpenAI's latest image generation model to produce a twelve-image farewell collage while sharing documentation and quickstart resources.

Full-Duplex Speech and Steerable Personality

GPT-Live operates as a full-duplex voice model, allowing continuous bidirectional audio streaming so users and agents can speak simultaneously, interrupt, and interject naturally. Rather than relying on rigid turn-taking heuristics or external voice activity detection models, GPT-Live manages conversational flow internally. Developers can prompt the model to adjust persona traits, including accent, enthusiasm, interruption tolerance, and backchannel cues like laughing or acknowledging statements.

Multi-Session Architecture and Automated Evals

The system architecture allows applications to spin up concurrent GPT-Live sessions with distinct system prompts and vocal identities. During the presentation, the primary voice agent brought in a secondary agent to participate in a collaborative word game. Beyond interactive entertainment, deploying parallel sessions with differing personas enables developers to simulate production environments and evaluate agent performance without requiring human conversation partners.

Backend Delegation and Context Update Streams

To handle complex tasks while maintaining an active voice conversation, GPT-Live delegates work to external systems using either client delegation or OpenAI-hosted responses delegation. While external tools execute tasks in the background, such as generating drawings or running code, backend services can feed information back into the active voice session across three channels: silent thinking context, speakable commentary text, or directive system instructions.

Turnless Interaction and Environmental Noise Resilience

Because GPT-Live is built without turn-based silence thresholds, it distinguishes spoken intent from ambient noise and background audio playback. The demonstration illustrated this by having the model maintain fluent conversation and adjust media controls while loud music played over the speakers. The presentation concluded with the model invoking an image generation tool to produce a twelve-panel grid of event artwork while simultaneously bidding the audience farewell.

The Bottom Line

The presentation demonstrates that GPT-Live combines full-duplex voice streaming, background delegation, and ambient noise resistance into a coherent developer architecture. By decoupling heavy computational tasks from conversational audio streaming, the model allows applications to run complex tools without introducing conversational lag or awkward silence. While the talk establishes practical integration patterns and evaluation methods, it leaves open specific API latency benchmarks and production pricing tiers.

FAQ

What is GPT-Live and how does its full-duplex voice architecture work?

GPT-Live is a real-time multimodal voice model that streams incoming and outgoing audio simultaneously. Its full-duplex architecture handles turn-taking internally without relying on separate voice activity detection models or silence heuristics.

How can developers evaluate voice applications using secondary GPT-Live sessions?

Developers can spin up a second GPT-Live session configured with custom instructions, accents, and behavioral goals to act as a user. This allows automated testing of conversational pacing, interruption handling, and overall agent performance.

What is the difference between client delegation and responses delegation in GPT-Live?

Client delegation sends task events back to the developer's application to execute within custom backends or existing agent harnesses. Responses delegation delegates task execution directly to OpenAI-hosted models and tools.

Which three channels can backend systems use to update GPT-Live during tasks?

Backends can update the model using the thinking stream for silent context, the commentary stream for speakable text, and the instruction stream for guiding behavioral guardrails and system rules.

Why does GPT-Live continue to function effectively in noisy acoustic environments?

Because GPT-Live does not rely on silence-based voice activity detection to decide when to speak, it can distinguish direct user conversation from continuous ambient noise or background music playback.

Worth watching for

Software engineers and product teams designing real-time multimodal applications, voice assistants, and interactive AI agent workflows.

  • gpt-live
  • voice-agents
  • multimodal
  • openai
  • developer-tools