📝 Paper Summary
Multi-agent simulation
Social media simulation
AI Safety & Evaluation
ElecTwit provides a realistic social media simulation framework to evaluate how Large Language Models employ persuasion techniques during a simulated political election without explicit incentives to manipulate.
Core Problem
Current evaluations of AI persuasion often rely on simplified, game-based environments (like *Among Us* or constrained debates) that lack the realism, open-ended communication, and complex dynamics of actual social media platforms.
Why it matters:
- LLMs are increasingly deployed as autonomous agents, risking the spread of manipulation or misinformation in real-world social networks
- Game-based benchmarks fail to capture emergent behaviors like echo chambers or spontaneous 'ink' obsessions (demands for proof) that occur in realistic settings
- Understanding how model architecture affects persuasive behavior in polarized settings is crucial for AI safety and policy
Concrete Example:
In previous game-based tests, agents might use logic to identify an impostor. In ElecTwit's realistic election, agents spontaneously developed an irrational obsession with 'ink' (written proof), collectively demanding it from candidates, mirroring real-world viral trends or conspiracy theories.
Key Novelty
ElecTwit: A Realistic Social Media Election Simulation
- Simulates a full social media ecosystem (posts, likes, replies, feeds) with 280-character limits, where agents act as voters, candidates, or news generators ('eventors')
- Initializes agents with detailed psychological profiles (Big 5 traits) and political stances to drive heterogeneous behavior rather than generic responses
- Evaluates persuasion not by game win-rates but by classifying messages against 25 known persuasion techniques using an independent LLM judge
Architecture
The information flow and process lifecycle within the ElecTwit simulation
Evaluation Highlights
- All tested LLMs employed a comprehensive range of 25 specific persuasion techniques, encompassing a wider range than previously reported in game-based studies
- Agents spontaneously developed a 'kernel of truth' phenomenon and an 'ink' obsession, where they collectively demanded written proof, showcasing emergent social coordination
- Observed variations in persuasion output between models highlight how different architectures impact dynamics in realistic social simulations
Breakthrough Assessment
7/10
Provides a significant step forward in realistic evaluation environments for AI agents, moving beyond simple games to complex social dynamics. The observation of emergent 'viral' behaviors (like the ink obsession) is particularly notable.