Safety on Scaling Trust Communityhttps://scalingtrust.org.uk/tags/safety/Recent content in Safety on Scaling Trust CommunityHugoen-usThu, 11 Jun 2026 00:00:00 +0100Joining Forces with Google DeepMind, the Cooperative AI Foundation and Schmidt Scienceshttps://scalingtrust.org.uk/blog/joining-forces-with-schmidt-sciences-google-deepmind-and-the-cooperative-ai-foundation/Thu, 11 Jun 2026 00:00:00 +0100https://scalingtrust.org.uk/blog/joining-forces-with-schmidt-sciences-google-deepmind-and-the-cooperative-ai-foundation/We're co-launching a $10M [funding call](https://aria.org.uk/opportunity-spaces/trust-everything-everywhere/scaling-trust/funding) on the safety and security of multi-agent, multi-principal systems.We’ve gone from prompting models, to giving them tools, to deploying agents that spawn sub-agents of their own. They now interact in the wild, with humans and with each other1, and increasingly in the physical world, from factory robots to autonomous labs2.

How will this ecosystem organise itself? How will it reshape society, how much value will it create, and how trustworthy can it be made? We don’t know yet. What we can already see: agents are collapsing “transaction costs” between humans3, strategic equilibria are appearing that classical game theory never had to price4, and verification is often becoming the bottleneck as the cost of creating things with AI falls toward zero5.

Scaling Trust takes on a slice of these questions: the trust infrastructure — new security primitives, from cryptography and secure hardware to new kinds of sensors — that lets agents enter into contracts securely, programmatically, and at scale, across the digital and physical worlds. Behind that slice sits a bigger vision we believe in: technology that augments human flourishing6 and preserves plurality.

To that end, we’re excited to be joining forces with Google DeepMind , the Cooperative AI Foundation and Schmidt Sciences who all share this vision, folding our shared thinking into a $10M funding call.

The call grew out of our interlocking work looking at different angles of the same picture: Google DeepMind’s Distributional AGI Safety argues that highly capable AI may arrive as networks of specialised agents rather than a single system; the Cooperative AI Foundation’s Multi-Agent Risks from Advanced AI maps the failure modes that only exist between agents; Schmidt Sciences’ AI Agents and Science of Trustworthy AI programmes study how coordination between agents emerges and breaks; and our own programme thesis argues that secure contracts between agents can preserve pluralism and unlock new forms of coordination. Read together, they make one argument: if capable AI is a network of agents, then its risks live between them, its coordination needs a science, and its rails need building.

The call

Open to researchers worldwide — individuals, teams, institutions — for foundational work the market won’t fund. Awards up to $1M, proposals due August 8.

The focus throughout is multi-agent, multi-principal systems: not one company’s fleet of agents, but ecosystems of agents built and deployed by different actors with different interests. It’s split into four categories:

✶

Sandboxes & testbeds scalable, high-fidelity, reproducible places to study agent populations safely

Science of agent networks when does a group of agents become a collective agent, with goals of its own?

Agent infrastructure identity, reputation, commitments — for entities that can be copied, modified, simulated, or deleted at scale

Oversight & control detecting collusion, attributing failures, steering populations under partial observability

Apply here.

Related links

  1. See Moltbook , a social network for agents, and OpenClaw , an open-source agent framework — both wildly popular, both with serious security issues. ↩︎

  2. Gemini now drives humanoid robots on factory floors (Wired); on the lab side, see Steering Towards Safe Self-Driving Laboratories (Nature). ↩︎

  3. Coasean Bargaining at Scale — Seb Krier; and The Coasean Singularity? — Shahidi et al. (NBER). ↩︎

  4. Conditional Recall — Schlegel and Sun, on equilibria unlocked by provable forgetting; and Learning Collusion in Episodic, Inventory-Constrained Markets — Friedrich et al., on collusion emerging among pricing agents. ↩︎

  5. When AI Writes the World’s Software — Leo de Moura; and How to Solve Secure Program Synthesis — von Hippel et al. ↩︎

  6. Positive Alignment: Artificial Intelligence for Human Flourishing — Laukkonen, Krier, Bakalar et al. ↩︎

]]>