scaling trustcommunity

A curated collection of links for the Scaling Trust community to get you started diving deeper in the topics we care about.

This list is alive — we add, change, and cut it all the time, and we welcome the community’s input: suggest links on Discord , in #link-sharing.

Scaling Trust Programme

Key links related to ARIA’s Scaling Trust programme.

• Programme page — the Scaling Trust source of truth for official comms.
• Programme thesis (PDF) — the case for the programme.
• Funding efforts — open and upcoming funding calls.
• Pre-programme discovery — outputs from our first cohort of 14 short projects, spanning multi-agent, cryptographic, and cyber-physical trust.
• Discovery workshop recordings — talks from our October 2025 workshop, from Consumable quantum data to Securing ultra-large-scale cyber-physical infrastructure.
• Opportunity space: Trust Everything, Everywhere — the landing page, and the full document (PDF) .
• Scaling Trust Discord — community discussion, questions, and updates. Suggest a link to add in here via the #link-sharing channel!

Related communities and efforts

Kindred communities and friends, suggest your own!

• Cooperative AI Foundation Slack — the CAIF community.
• Schmidt Sciences: AI Agents — funding research on how multiple intelligent agents communicate and coordinate.
• Cosmos Institute — training “philosopher-builders” so AI serves human flourishing.
• Secure Program Synthesis — the community forming around provably secure software generation.
• Also funding work in this space: CAIF grants · Foresight Institute · Coefficient Giving · Survival and Flourishing Fund · UK AISI · SAIR

Vision

Some of the pieces that have inspired the programme.

• Is Information the Key? — Gilles Brassard. Quantum theory’s deepest laws may be about information itself.
• The Moral Character of Cryptographic Work — Phil Rogaway. Cryptography is political, and the field has moral obligations.
• The Quantum Thief — Hannu Rajaniemi. A heist novel set in a society that runs on cryptographic privacy.
• Black-Hole Radiation Decoding is Quantum Cryptography — Zvika Brakerski. Physics problems recast as cryptographic hardness.
• Coasean Bargaining at Scale — Seb Krier (Google DeepMind). AI agents collapse transaction costs and make hyper-local bargaining viable.
• The Coasean Singularity? — Shahidi et al. (NBER). How agent markets reshape demand, supply, and market design.
• Conditional Recall — Christoph Schlegel and Xinyuan Sun. Commitments to provably forget information, and what they unlock.
• Open Challenges in Multi-Agent Security — Christian Schroeder de Witt et al. The agenda-setting map of the field this programme funds.
• Multi-Agent Risks from Advanced AI — Hammond et al. The field survey: collusion, conflict, destabilising dynamics — and why they don’t reduce to single-agent safety.
• Positive Alignment — Laukkonen, Krier, Bakalar et al. Alignment as actively fostering flourishing, not just preventing harm.
• Distributional AGI Safety — Tomašev et al. (Google DeepMind). AGI may arrive as networks of agents, so safety becomes an ecosystem property.
• Programmable Cryptography — 0xParc. The case for general-purpose, composable cryptography.
• The Tragedy of the Agentic Commons — Strange Loop Canon. Why agent ecosystems strain shared resources without new governance.

Multi-agent testbeds & experiments

New, more realistic/open-ended environments (not just simple games) that are starting to be used to test LLMs in multi-agent settings.

• Emergence World — persistent simulation for long-horizon autonomy, coalitions, governance, and behavioural drift.
• The Agent Village at Edge Esmeralda — a 500-person popup town gave everyone personal agents for coordination and governance.
• Project Deal — Anthropic. Claude buying, selling, and negotiating in a real office marketplace.
• Vending-Bench Arena — Andon Labs. Competing agents run vending businesses over months.
• Project Iceberg — MIT Media Lab. National-scale human–AI labour-market simulation.
• AI Digest Village — what we learned from a year of agents living together.
• TERMS-Bench — a diagnostic benchmark for LLM negotiation: surplus capture, calibration, opponent modelling.
• CRUX — Kapoor et al. Open-world evals on long-horizon tasks that resist clean grading.
• The Agent Company — a benchmark running an entire fake software company on agents.
• Butter-Bench — Andon Labs. Can an LLM-controlled robot pass the butter?
• Digital Red Queen — Sakana. Adversarial program evolution in Core War.

Agentic game theory

How strategic behaviour changes when the players are AIs.

• Cooperative AI: machines must learn to find common ground — Dafoe et al. (Nature). The commentary that named the field.
• Foundations of Cooperative AI — Conitzer and Oesterheld. Why machine cooperation needs its own research field.
• Program Equilibrium — Moshe Tennenholtz. The foundational result: agents that can read each other’s code reach equilibria humans can’t.
• Secret Collusion among AI Agents — Motwani et al. Agents hiding coordination in plain sight via steganography.
• Learning Collusion in Episodic, Inventory-Constrained Markets — Friedrich et al. Pricing agents learn to collude under realistic constraints.
• Secure and Secret Cooperation in Robotic Swarms — Ferrer et al. Swarms that cooperate without exposing their members.
• Language Models Can Reduce Asymmetry in Information Markets — Rahaman et al. Agents brokering information they can inspect but not leak.
• Mechanism Design for Large Language Models — Dütting et al. Auction design when the bidders are language models.
• Virtual Agent Economies — Tomašev et al. (Google DeepMind). How to deliberately design the agent markets ahead of us.

Agent infrastructure

Identity, reputation, and commitments — the rails agents transact on.

• Infrastructure for AI Agents — Chan et al. The base layer agents need: identity, channels, oversight hooks.
• Inter-Agent Trust Models — a comparative study of trust across A2A, AP2, and ERC-8004.
• NDAI Agreements — Stephenson et al. Contracts between AIs without trusted third-party enforcement.

Autonomous protocol generation

Towards agents that generate provably secure protocols on demand.

• How to Solve Secure Program Synthesis — Max von Hippel et al. The four open challenges on the way to provably secure software generation.
• ArkLib — a Lean 4 library formally verifying SNARK components (Sum-Check, Spartan, FRI).
• Proof Assistants in the Age of AI — Leo de Moura. Lean’s creator on what AI changes for formal proof.
• When AI Writes the World’s Software — Leo de Moura. Machine-written code needs machine-checked guarantees.
• Owl — cryptographic protocols with formal, machine-checked security guarantees.

Secure requirement capture

How agents learn what we want — without leaking it.

• Privacy as Contextual Integrity — Helen Nissenbaum. The canonical frame: privacy is appropriate information flow, not secrecy.
• Privacy Reasoning in Ambiguous Contexts — Yi et al. Can models judge what’s appropriate to share, in context?

Formal AI security

Provable guarantees about learned systems.

• Provably Safe Systems — Tegmark and Omohundro. The maximalist case: safety guarantees should be mathematical proofs.
• Models That Prove Their Own Correctness — Amit, Goldwasser, Paradise, and Rothblum. Models that emit proofs their answers are right.
• Defeating Prompt Injections by Design — Debenedetti et al. System designs that make injection structurally impossible, not just unlikely.

Physical verification & secure hardware

Proving things about the physical world — chips, cameras, labs.

• Remotely Detectable Robot Policy Watermarking — Amir, Flageat, and Prorok. Spectral signals hidden in robot motion, detectable from video alone.
• IRIS (Infra-Red, in situ) Project Updates — Bunnie Studios. Non-destructive chip inspection with infrared imaging.
• SecureDNA — Baum et al. Cryptographic screening of the world’s DNA synthesis orders.
• Matta — camera verification of manufacturing.
• PCR7500 — trusted bioreactors: PCR data signed in a TEE at the point of capture.

Nature cryptography?

Security primitives based on the laws of nature.

• Physical One-Way Functions — Pappu et al. The original physically unclonable function: optical tokens as one-way functions.
• A New Approach to Nuclear Warhead Verification Using a Zero-Knowledge Protocol — Glaser et al. Prove a warhead is real without revealing its design.
• Quantum Cryptography: Uncertainty in the Service of Privacy — Charles Bennett. Quantum uncertainty itself as a privacy primitive.
• Consumable Data via Quantum Communication — Gilboa et al. Data that can only be used a bounded number of times.
• Conjugate Coding — Stephen Wiesner. The 1970s manuscript that invented quantum money.
• Unclonable Polymers and Their Cryptographic Applications — Almashaqbeh et al. Secret keys stored in molecules that can’t be copied.
• Building Unclonable Cryptography: A Tale of Two No-cloning Paradigms — Almashaqbeh et al. The field map of unclonability, quantum and physical.
• An Introduction to Protein Cryptography — Tirmazi et al. Encoding data in amino-acid sequences, tamper- and copy-resistant.
• Cryptography in the DNA of Living Cells — Volf et al. Multi-site base editing as message encryption inside living cells.
• Hidden Messages in DNA Could Reduce Biosecurity Risks — Danielle Gerhard. DNA watermarks for biosecurity.
• Neuroscience Needs Network Science — Barabási et al. The brain as a network-science problem.
• A New Age of Computing and the Brain — Golland et al. A research agenda where computing and neuroscience meet.
• Mosquito-derived Ingested DNA as a Tool for Monitoring Terrestrial Vertebrates — Chivas et al. Mosquito blood meals as a wildlife-monitoring sensor network.
• Molecules that Generate Fingerprints — Motiei et al. Fluorescent sensors as chemical fingerprints for authentication.
• TTEE: Marrying Cryptography and Physics — Quintus Kilbourn. A talk on what physics can do for cryptographic trust.
• Signature for Objects — Hayashi et al. Signing physical objects, with unforgeability against physically-enhanced adversaries.
• Cryptographic Data Exchange for Nuclear Warheads — Perry et al. The modern follow-on to zero-knowledge warhead verification.
• Cryptographic Sensing — Ishai et al. Sensors that reveal only the authorised measurement and nothing else.
• Cryptography by Cellular Automata — Applebaum et al. How fast can cryptographic complexity emerge in nature?

Suggest a link via Discord — try #link-sharing.