Multi-agent LLM deliberation for AI policy
An LLM multi-agent framework with RAG context and generated personas, used to simulate stakeholder deliberation.
Role: Research Officer (RO4), Australian National University, Centre for Public Awareness of Science, 2024 to 2025. I worked in a four-person team with a social scientist, a psychology student and a tech lead.
The problem
Policy on AI, and on other contested issues, involves many stakeholders with different interests. The project needed a way to simulate how those stakeholders would deliberate, with an emphasis on risk mitigation and governance.
What I built
- I designed, implemented and deployed an LLM-based multi-agent framework for AI policy deliberation and risk mitigation.
- A retrieval-augmented generation (RAG) system gives each agent context.
- With the psychology student, I designed the generated stakeholder personas, which capture each stakeholder’s interests, needs, constraints and goals.
- LLM judges oversee deliberations and prevent repetitive loops. People can also take part as judges, as members of stakeholder teams or as independent stakeholder representatives.
- I established the evaluation and benchmarking frameworks.
- With the social scientist, I designed the simulations and prepared the data.
Where it was applied
- Australian government AI-governance scenarios with about 400 national and international stakeholders.
- Murray River water disputes.