Large Language Models | Manuela Chacon-Chamorro

Sustaining Collective Welfare Under Disruption: A Pilot Study of Cooperative Resilience in Human and LLM Multi-Agent Systems

Thu, 10 Sep 2026 00:00:00 +0000

✨ About the talk

At FAIEMA 2026, I presented our work “Sustaining Collective Welfare Under Disruption: A Pilot Study of Cooperative Resilience in Human and LLM Multi-Agent Systems.” This research was developed jointly with Juan Sebastián Pinzón, Rubén Manrique, Luis Felipe Giraldo, and Nicanor Quijano at Universidad de los Andes.

The study explores how human groups and LLM-based multi-agent systems respond when cooperation is challenged by environmental and behavioral disruptions. Both populations interacted in the same mixed-motive social dilemma, following a shared experimental and evaluation protocol.

Presenting our work on cooperative resilience in human and LLM-based multi-agent systems.

🔎 Main findings

Our results showed that human groups sustained higher levels of cooperative resilience across the evaluated scenarios. Human participants tended to regulate their use of shared resources and maintain collective welfare more effectively under disruption.

In contrast, LLM-based agents showed more limited adaptation and tended toward higher resource consumption. Communication improved their performance in some scenarios, but it did not fully close the resilience gap between human and LLM-based groups.

These findings highlight the importance of evaluating AI agents not only under stable conditions, but also before, during, and after adverse events.

🌍 The conference experience

Participating in FAIEMA 2026 was a very rewarding experience. I had the opportunity to share our research with people from different institutions and research backgrounds, receive thoughtful questions, and exchange ideas about cooperative AI, multi-agent systems, resilience, and the societal implications of artificial intelligence.

Beyond the presentation itself, the conference created a welcoming environment for meeting other researchers and learning about work across different areas of AI. Sharing the conference activities and dinner with so many people made the experience especially enjoyable and memorable.

Sharing ideas and experiences with other participants during FAIEMA 2026.

💭 Final reflection

Presenting this work in L’Aquila was particularly meaningful because it brought together several dimensions of my doctoral research: defining cooperative resilience, studying collective behavior under disruption, and comparing the capabilities of humans and artificial agents.

The experience also reminded me that conferences are not only spaces for presenting results. They are opportunities to discuss unfinished questions, receive new perspectives, build connections, and recognize how research grows through interaction with others.

Sustaining Collective Welfare Under Disruption: A Pilot Study of Cooperative Resilience in Human and LLM Multi-Agent Systems

Thu, 10 Sep 2026 00:00:00 +0000

Version note. This entry corresponds to the revised camera-ready manuscript accepted at FAIEMA 2026. It is distinct from the earlier arXiv preprint, “Evaluating Cooperative Resilience in Multiagent Systems: A Comparison Between Humans and LLMs,” which remains available as the original preprint. The accepted manuscript is a substantially revised and polished version of the work and will be linked here once it becomes available in the official Springer proceedings.

Abstract. The ability to sustain collective welfare under disruption is key for deploying agents in real-world multi-agent environments. However, most evaluations focus on stable scenarios; as a result, agent behavior under adverse conditions remains poorly understood. In this work, we present a pilot comparative study of cooperative resilience—the ability of a system to act before, during, and after a disruption to preserve collective welfare—between human groups and LLM-based multi-agent systems operating in the same social dilemma under a shared disruption and evaluation protocol. Both populations are evaluated with and without communication, using population-specific observation and communication interfaces while preserving the same task structure, disruption scenarios, and evaluation criteria. Our results indicate that, in this setting, LLM-based agents show more limited adaptation and lower levels of collective welfare compared to human groups. These findings suggest potential limitations in current agent designs and motivate the development of more robust and adaptive multi-agent systems.

📚 Recommended Readings on Cooperative AI (Vol. 1)

Wed, 22 Oct 2025 00:00:00 +0000

Cooperative AI is not only a technical challenge. It also raises a broader question: how can we design artificial agents that pursue their objectives while supporting cooperation and collective welfare?

This question becomes especially important in mixed-motive systems, where individual and collective interests are neither fully aligned nor entirely opposed. In these environments, agents must learn how to coordinate, communicate, adapt to different partners, and respond when cooperation is threatened.

In this first volume, I share five papers that have influenced how I think about cooperative AI, social dilemmas, preference learning, LLM-based agents, and cooperative resilience. Together, they provide a useful path from the general research agenda to concrete methods and experimental environments.

1. 🧭 — Dafoe et al., 2021

This paper presents a broad research agenda for Cooperative AI. It argues that artificial intelligence should explicitly study problems in which agents can improve their outcomes through cooperation.

The authors organize the field around several major challenges, including cooperative capabilities, communication, commitment, institutional design, human–AI interaction, and cooperation among artificial agents. The paper also connects ideas from multi-agent systems, game theory, social choice, behavioral science, and AI alignment.

💡 Why read it: It is one of the best starting points for understanding Cooperative AI as a research field. Rather than proposing a single algorithm, it maps the central questions and explains why cooperation should be treated as a distinct scientific problem.

📌 What to focus on: The distinction between building agents that can cooperate and designing mechanisms or institutions that make cooperation more likely.

2. 🤖 — Smith et al. and Concordia Contest Participants, 2025

LLM-based agents can communicate fluently and appear socially capable, but this does not necessarily mean that they can cooperate reliably across new partners and situations.

This NeurIPS 2025 paper introduces a methodology for evaluating LLM-based agents in zero-shot, mixed-motive environments using Concordia, a natural-language multi-agent simulation framework. The agents are evaluated across diverse scenarios involving negotiation, coordination, collective action, persuasion, and norm enforcement.

The results reveal important gaps between fluent social interaction and robust cooperative generalization. In particular, current agents can struggle in situations that require persuasion, adaptation to unfamiliar partners, or the enforcement of social norms.

This work is especially meaningful to me because I participated in the Concordia Contest and contributed to the collective evaluation effort behind the paper.

💡 Why read it: It demonstrates why evaluating an agent in a single environment is insufficient. Cooperative intelligence requires generalization across different social situations, objectives, and interaction partners.

📌 What to focus on: The experimental definition of general cooperative intelligence and the distinction between performing well in one scenario and generalizing across multiple mixed-motive contexts.

3. 🎯 — Brown et al., 2019

This paper introduces Trajectory-ranked Reward EXtrapolation (T-REX), a method for learning reward functions from approximately ranked demonstrations.

Traditional imitation learning often assumes that demonstrations are close to optimal. T-REX takes a different perspective: even when demonstrations are imperfect, their relative quality can provide useful information about the underlying objective.

Instead of directly reproducing demonstrated behavior, the method learns a reward function from preferences over trajectories. Once the reward has been inferred, a reinforcement learning agent can potentially discover a policy that performs better than the original demonstrations.

This paper has been particularly relevant to my research on preference-based reward learning and cooperative resilience. It provides an important methodological foundation for using ranked trajectories to infer which behaviors should be encouraged in multi-agent environments.

💡 Why read it: It offers a clear example of how preferences and rankings can provide supervision when an explicit reward function is difficult to specify.

📌 What to focus on: How trajectory rankings are transformed into a learned reward function, and why reward inference can allow an agent to outperform imperfect demonstrations.

4. 🏘️ — Park et al., 2023

This paper introduces generative agents: LLM-based computational agents designed to produce believable individual and social behavior.

The proposed architecture combines three central processes:

Memory, which stores the agent’s experiences.
Reflection, which transforms individual observations into higher-level conclusions.
Planning, which uses memories and reflections to guide future behavior.

The authors evaluate the architecture in a sandbox environment populated by 25 agents. From relatively simple initial conditions, the agents produce emergent social behaviors such as forming relationships, sharing information, coordinating activities, and organizing a social event.

💡 Why read it: It provides an influential architecture for building LLM-based agents with memory, reflection, and planning. It also illustrates how individual cognitive mechanisms can produce collective patterns.

📌 What to focus on: The interaction between memory retrieval, reflection, and planning, as well as the evaluation of whether the resulting behavior is genuinely coherent and socially meaningful.

⚠️ A useful critical question: Believable behavior is not necessarily cooperative, robust, or human-equivalent behavior. This distinction is essential when using generative agents to study social systems.

5. 🍎 — Leibo et al., 2021

Melting Pot is an evaluation suite designed to study generalization in multi-agent reinforcement learning. It contains sequential social dilemmas in which agents must interact over time while facing tensions between individual incentives and collective outcomes.

The suite evaluates more than task performance. It examines whether agents can generalize to new social partners and unfamiliar configurations after training. Its environments represent challenges such as resource sharing, coordination, reciprocity, competition, and collective action.

One of its best-known environments is Commons Harvest, which models a tragedy-of-the-commons problem. Agents collect apples from a renewable shared resource, but excessive consumption can prevent regeneration and lead to collective collapse.

Melting Pot has been central to my own research because it provides environments in which cooperative resilience can be studied under disruptions. It allows us to evaluate not only whether agents cooperate under stable conditions, but also whether they can preserve collective welfare when environmental or behavioral conditions change.

💡 Why read it: It connects theoretical questions about cooperation with concrete and reproducible multi-agent environments.

📌 What to focus on: The evaluation protocol for social generalization and the difference between learning a successful policy and learning a policy that remains effective with unfamiliar partners.

🔗 How these papers connect

These five papers address different layers of Cooperative AI:

Open Problems in Cooperative AI defines the broader research agenda.
The Concordia evaluation asks whether LLM-based agents can generalize across mixed-motive social situations.
T-REX shows how preferences over trajectories can be used to infer reward functions.
Generative Agents proposes an architecture for socially situated LLM-based agents.
Melting Pot provides controlled environments for evaluating cooperation and social generalization.

Together, they also reveal an important distinction: an agent can communicate fluently, imitate believable behavior, or perform well in a particular task without necessarily being robustly cooperative.

For me, this distinction motivates the study of cooperative resilience: the capacity of a multi-agent system to preserve collective welfare before, during, and after disruptions.

📖 Suggested reading order

If you are new to the field, I recommend starting with:

Open Problems in Cooperative AI, to understand the overall research landscape.
Melting Pot, to see how cooperation can be evaluated experimentally.
The Concordia paper, to understand the challenges of evaluating LLM-based social agents.
Generative Agents, to explore memory, reflection, and planning in LLM agents.
T-REX, if you are interested in reward inference and preference-based learning.

🌱 Closing thoughts

Cooperative AI brings together technical and social questions. It requires us to think not only about what an individual agent can achieve, but also about how multiple agents interact, how collective outcomes emerge, and how cooperation can be sustained when conditions change.

These papers have helped me connect ideas from multi-agent reinforcement learning, large language models, preference learning, social dilemmas, and resilience. They also show how much remains open: defining meaningful evaluation criteria, understanding generalization, learning appropriate incentives, and designing agents that support collective welfare in uncertain environments.

This is only the first volume. Future editions will include additional work on cooperative resilience, human–AI interaction, multi-agent safety, incentive design, and collective intelligence.