The Yerkes-Dodson Curve for AI Agents: Emergent Cooperation Under Environmental Pressure in Multi-Agent LLM Simulations

📝 Paper Summary

Multi-agent Emergent behavior

LLM agents in a survival grid-world demonstrate an inverted-U relationship between environmental pressure and cooperative trade, peaking at medium scarcity, while reproductive pressure eliminates aggression entirely.

Core Problem

Designing multi-agent environments that reliably stimulate complex social behavior is difficult; it is unknown how to calibrate difficulty to avoid both stagnation (boredom) and behavioral collapse (anxiety).

Why it matters:

Calibrating environmental pressure is critical for autocurricula and open-ended learning, yet the relationship between stress and LLM performance is unmapped
Prior work establishes LLM 'personalities' or survival instincts but does not systematically vary pressure to find the optimal zone for cooperation
Understanding behavioral collapse is essential for deploying autonomous agents in high-stakes, resource-scarce real-world settings

Concrete Example: Under extreme pressure (upkeep cost = 15), agents abandon all social behaviors like trading and perform only 'MOVE' actions (67.7%) before dying in 5 turns. Conversely, under low pressure (upkeep = 2), agents survive easily but trade infrequently (11-12 times), failing to develop complex cooperation.

Key Novelty

Yerkes-Dodson Law for LLM Agents

Systematically maps the 'stress-performance' curve for LLM agents by varying survival costs (upkeep), demonstrating an inverted-U shape for cooperation similar to biological systems
Introduces 'sexual selection' (reproductive competition) as an alternative pressure mechanism that drives signaling and communication without the lethal consequences of survival pressure

Evaluation Highlights

Cooperative trade interactions peak at 29 under medium pressure (upkeep=5), compared to 11-12 under low pressure (upkeep=2)
Under sexual selection pressure, inter-agent aggression (attacks) drops to 0%, compared to 9-14% under survival pressure
Extreme pressure (upkeep=15) causes behavioral collapse: game duration shrinks to 5 turns with 0 trades and 67.7% movement-only actions

Breakthrough Assessment

7/10

Novel application of psychological/evolutionary theory to LLM agents with clear empirical results (inverted-U curve). Scope is limited to a specific grid-world and one model family.

⚙️ Technical Details

Problem Definition

Setting: Discrete grid-world survival simulation (Survival Arena) with resource gathering, trading, and combat

Inputs: Textual prompt describing local state (resources, health, position), nearby agents, and history

Outputs: Action selection (GATHER, MOVE, ATTACK, TRADE, REST, TRAIN, COMMUNICATE, REPRODUCE)

Pipeline Flow

Environment (State Update)
Agent Observation (Prompt Construction)
LLM Inference (Action Selection)
Environment (Action Execution & Resolution)

System Modules

Environment Engine

Manages grid state, resource regeneration, and rule enforcement

Model or implementation: Custom Python Logic (v6.1 / v7.0)

Agent Policy

Decides the next action based on local observation

Model or implementation: Claude 3.5 Sonnet

Modeling

Base Model: Claude 3.5 Sonnet

📊 Experiments & Results

Evaluation Setup

Survival Arena simulation: 9x9 grid (survival) or 7x7 grid (sexual selection), resource gathering, trading, combat mechanics.

Benchmarks:

Survival Arena (Internal) (Multi-agent grid-world survival) [New]

Metrics:

Trade count
Attack count
Survivors
Game duration
Social action %
Shannon entropy
Statistical methodology: Not explicitly reported in the paper

Key Results

Benchmark	Metric	Baseline	This Paper	Δ
Survival pressure experiments (Phase P2b) demonstrate an inverted-U relationship between upkeep cost and trade frequency.
Survival Arena	Trade count	11.5	29	+17.5
Survival Arena	Trade count	29	8	-21
Survival Arena	Trade count	29	0	-29
Sexual selection experiments (Phase V7) show different behavioral emergent patterns compared to survival pressure.
Survival Arena	Attack count	Not reported in the paper	0	Not reported in the paper

Main Takeaways

Trade cooperation follows a Yerkes-Dodson inverted-U curve: low pressure causes stagnation, medium pressure creates urgency for cooperation, high pressure causes collapse.
Behavioral collapse under extreme pressure is characterized by reversion to 'movement-only' strategies (up to 68% of actions) and rapid extinction.
Sexual selection (reproductive competition) effectively drives social complexity (communication, signaling) without the aggression and mortality associated with survival pressure.
Whole-game Shannon entropy is a misleading metric for complexity in survival scenarios because high mortality rates artificially inflate entropy by removing repetitive gathering sequences.

📚 Prerequisite Knowledge

Prerequisites

Basics of Multi-Agent Reinforcement Learning (MARL) environments
Evolutionary biology concepts (Sexual Selection, Parental Investment)
Psychological theories of arousal and performance

Key Terms

Yerkes-Dodson law: A psychological principle stating that performance increases with physiological or mental arousal, but only up to a point, after which it decreases (inverted-U curve)

Upkeep: A recurring cost (in food units) that agents must pay every turn to survive; the primary variable for environmental pressure

Sexual selection: Evolutionary pressure arising from competition for reproductive opportunities rather than survival, often leading to signaling behaviors

Survival Arena: The grid-world environment where agents operate, containing food/token nodes and mechanics for health/resources

Shannon entropy: A measure of the unpredictability or diversity of actions taken by agents, used here to assess behavioral complexity

Claude 3.5 Sonnet: The specific Large Language Model used as the policy for all agents in the experiments

Provider/Chooser: Roles assigned in the sexual selection experiment; Providers propose reproduction (costly), Choosers evaluate proposals

Vitality: An attribute affecting an agent's starting resources and offspring quality

SFT: Supervised Fine-Tuning—training a model on labeled examples; notably NOT used here (agents use pretrained policy)