Hands Free, AIs Forward: NVIDIA XR AI Brings Agents to AR Glasses

However, building agentic systems that combine models, skills, harnesses, tools and an agentic runtime to help people perform hands-on work is challenging. To operate effectively in dynamic, real-world environments, these agents must do more than generate responses. Like human workers, they need knowledge, tools and specialized skills to perceive and understand the world through video, audio and sensor data, interpret fast-changing conditions and spatial context, retrieve information from enterprise systems, reason about the next best action and use software tools to complete tasks.

All of this must happen with low latency and in a way that supports the user without creating distraction. NVIDIA XR AI is a developer library that helps developers build these agentic applications. By connecting inputs from AR glasses and XR devices with AI models, enterprise data, tools and accelerated computing, NVIDIA XR AI enables agents that can perceive, reason and act in the flow of work.

It provides a foundation for developers to build or connect skills and tools for enterprise XR applications, and simplifies the integration of multimodal perception, enterprise retrieval, reasoning models and agent orchestration. Together, these capabilities make it easier to build spatially aware, multimodal AI agents that deliver low-latency, context-aware assistance in AR and XR experiences. NVIDIA NeMo Agent Toolkit enables tool use, reasoning workflows and multi-agent coordination, while NVIDIA accelerated computing platforms - including NVIDIA DGX Spark, NVIDIA DGX Station and NVIDIA RTX PRO systems - provide the infrastructure to run inference across cloud, data center and edge environments.

Together, these capabilities enable AI agents that can understand their surroundings, access enterprise knowledge, reason about complex tasks and deliver contextual assistance in real time. Across manufacturing, science, healthcare, design and immersive learning, developers and enterprises are already tapping NVIDIA XR AI - embedding AI agents where the work happens. Siemens is exploring in a research context how NVIDIA XR AI and NVIDIA DGX Spark can help factory engineers find maintenance information, troubleshoot issues, verify work and capture what happened on the shop floor.

With this system, an engineer wearing lightweight glasses can ask an AI agent about a programmable logic controller issue and receive real-time guidance, connecting industrial systems, digital twins and automation workflows. In the research lab, Rana , an AutoBio company building AI systems for scientific research, is introducing its LabOS system on NVIDIA XR AI to bring spatial intelligence directly into scientific workflows. LabOS provides real-time, hands-free guidance for complex experimental workflows, starting with stem cell therapy and gene-editing research at the Cong Lab at Stanford University School of Medicine and the Wang Lab at Princeton University .

Built on the XR AI architecture, the LabOS co-scientist perceives, understands and acts within the lab environment, helping researchers identify the right sample and CRISPR gene editor, guiding each experimental step and capturing a structured, reproducible record as humans, robots and AI systems collaborate at the bench. Physically aware AI agents, delivered through AR glasses and powered by NVIDIA GPUs, serve as a next-generation interface for AI-assisted science - keeping researchers focused on complex procedures while receiving contextual guidance in real time.