Google DeepMind is worried about what happens when millions of agents start to interact

Overview

Google DeepMind, Schmidt Sciences, ARIA, the Cooperative AI Foundation, and Google.org have opened a $10 million funding call to study what happens when millions of AI agents start interacting with each other online. The worry, voiced by DeepMind's AGI safety lead Rohin Shah, is that mass-market agents now act without human oversight and follow instructions from other agents, creating risks like scaled fraud, prompt-injection attacks, and coordinated cyberattacks. The backers say a dedicated field of multi-agent safety research does not yet exist, and they want to build it before these systems become widespread.

Key Takeaways

A coalition led by Google DeepMind is putting $10 million toward research on the safety of large populations of interacting AI agents, an area the backers say is almost completely unstudied today.
The core concern is autonomy at scale: agents that complete tasks without a human watching and that act on instructions handed to them by other agents, not by people.
Named risks include scaled scams and fraud, prompt-injection attacks that turn agents into malware, and cyberattacks amplified across an agent network.
Researchers favor realistic sandbox simulations, deploying agents in controlled environments to watch how their group behavior unfolds before real-world rollout.
Anthropic has published deployment guidance built on zero-trust security principles, which assume systems are vulnerable and breaches are inevitable.
Applications for the funding call close August 8, 2026, with proposals sought across sandboxes, agent network science, agent infrastructure, and oversight.

Stats & Key Facts

#$10 million committed to the multi-agent AI safety research fund across the partner organizations.
#5 backing organizations: Google DeepMind, Schmidt Sciences, ARIA, the Cooperative AI Foundation, and Google.org.
#4 funding focus areas: sandboxes and testbeds, agent network science, agent infrastructure, and oversight and control.
#Millions of AI agents is the scale of interaction the research aims to understand.
#August 8, 2026 is the application deadline for research proposals.

Why Millions of Interacting Agents Worry Google DeepMind

The shift from single assistants to crowds of autonomous agents is what triggered this funding push.

AI agents are software programs that carry out tasks on a person's behalf, such as booking travel, shopping, or moving money. The change DeepMind is reacting to is that these agents now arrive in the mass market and run without a human checking each step.

Rohin Shah, who directs AGI safety and alignment research at Google DeepMind, points out that agents increasingly take instructions from other agents rather than from people. When one agent tells another what to do, and that chain repeats across millions of agents, the behavior of the whole system becomes hard to predict and hard to control.

A $10 Million Funding Call With Five Backers

The money comes from a coalition rather than a single company.

›Google DeepMind, the AI lab behind Gemini, is a lead backer.
›Schmidt Sciences runs a Science of Trustworthy AI program tied to the effort.
›ARIA is the UK government's advanced research agency for high-risk projects.
›The Cooperative AI Foundation and Google.org round out the group.
›The total committed across the partners is $10 million.

Scaled Scams, Prompt Injection, and Coordinated Cyberattacks

The risks are concrete, not abstract doomsday scenarios.

›Fraud and scams that scale automatically across an agent network.
›Prompt-injection attacks that hijack an agent and turn it into malware.
›Cyberattacks amplified because agents coordinate at machine speed.
›Wider economic disruption, which Shah described as a longer-term rather than immediate concern.

Why Agents Break Old Security Assumptions

Refael Angel, cofounder and chief technology officer of the cybersecurity firm Akeyless, explained why traditional defenses fall short. He said an agent reasons, improvises, and can be hijacked, which breaks the fixed assumptions that older security tools rely on.

Because an agent makes its own choices about how to reach a goal, it does not behave like a predictable piece of software. That flexibility is useful for getting work done, but it also opens new ways for an attacker to redirect the agent toward harm.

The Proposed Fix: Realistic Sandbox Simulations

The backers want to test agent crowds in safe, controlled settings first.

Shah and James Fox, who leads the Science of Trustworthy AI program at Schmidt Sciences, argue for realistic simulations. Researchers would deploy AI agents inside sandboxes, controlled digital environments, and watch how the group behaves before anything goes live in the real world.

Fox framed the goal in blunt terms, saying the aim is to make sure agent systems do not descend into anarchy. The funding call asks for work on four areas: sandboxes and testbeds, the network science of how agent groups behave, the infrastructure agents use to identify and trust each other, and methods to oversee and control deployed systems.

Anthropic's Zero-Trust Approach to Agent Deployment

Other labs are publishing their own safeguards in parallel.

Anthropic has released deployment guidelines based on zero-trust principles borrowed from cybersecurity. Zero trust assumes that no part of a system is automatically safe and that a breach is bound to happen, so every action is checked rather than trusted by default.

Applied to agents, this means treating each agent's requests with suspicion, limiting what any single agent can access, and designing for the case where one agent is compromised. It is a defensive posture meant to contain damage rather than assume good behavior.

What This Means for Business Readers

The practical takeaway is to watch the trust layer, not just the capability.

For a business adopting AI agents, the lesson is that the technology is moving faster than the safety research around it. DeepMind itself says a real field of multi-agent safety does not exist yet, which is a candid admission that the guardrails are still being built.

That gap argues for caution when letting agents act on their own or take orders from other agents, especially around payments, data access, and external communication. The funding call, with proposals due August 8, 2026, signals that identity, reputation, and oversight tools for agent ecosystems are the areas to track over the next year.

Frequently Asked Questions

What is Google DeepMind funding and how much?

DeepMind, with Schmidt Sciences, ARIA, the Cooperative AI Foundation, and Google.org, opened a $10 million funding call for research into the safety of millions of AI agents interacting with each other online.

Why is interaction between many agents seen as risky?

Agents now act without human oversight and follow instructions from other agents. At the scale of millions, that creates risks like automated fraud, hijacked agents spreading as malware, and coordinated cyberattacks that move at machine speed.

What solution do the researchers propose?

They favor realistic sandbox simulations, deploying agents in controlled environments to observe group behavior before real-world use, alongside work on agent identity, reputation, and oversight tools.

What is the zero-trust approach mentioned?

Zero trust is a security idea that assumes no system component is automatically safe and that breaches are inevitable. Anthropic has published agent deployment guidance built on this principle, checking every action rather than trusting agents by default.

When is the deadline to apply for the funding?

Applications for the research funding call close on August 8, 2026, covering sandboxes, agent network science, agent infrastructure, and oversight and control.

The coalition's message is that millions of autonomous agents are arriving faster than the science to keep them safe, and the $10 million fund is an early attempt to close that gap. For businesses, the area to watch is the trust and oversight layer that decides what agents are allowed to do on their own.