SwarmAgentic: Towards Fully Automated Agentic System Generation via Swarm Intelligence

📝 Paper Summary

Self-evolving Agentic reasoning Multi-agent

SwarmAgentic adapts Particle Swarm Optimization to the language domain to autonomously generate, refine, and coordinate multi-agent systems from scratch without predefined templates.

Core Problem

Existing agentic generation frameworks lack full autonomy, relying on fixed templates, seed agents, or manual engineering for collaboration strategies, which limits adaptability to open-ended tasks.

Why it matters:

Rigid designs suppress the emergence of self-optimizing system behaviors necessary for complex, exploratory tasks
Manually designing agents and collaboration strategies for open-ended tasks is prohibitively complex and labor-intensive
Current frameworks (e.g., ADAS, AutoAgents) fail to satisfy all three autonomy criteria: from-scratch generation, self-optimizing functionality, and self-optimizing collaboration

Concrete Example: In a travel planning task, standard methods might fail to check budget constraints effectively. SwarmAgentic autonomously identifies this flaw and introduces a specific 'Quality Assurance Specialist' role with a verification step to ensure the plan meets the user's budget.

Key Novelty

Language-driven Particle Swarm Optimization (PSO) for Agentic Systems

Reformulates PSO for discrete, symbolic search spaces where 'position' is an agentic system configuration and 'velocity' is a textual update plan
Jointly optimizes agent functionalities (roles, prompts) and collaboration structures (workflows) as interdependent components rather than separately
Introduces Failure-Aware Velocity Updates, using LLMs to analyze execution errors and guide the swarm toward better configurations based on personal and global bests

Architecture

The complete SwarmAgentic pipeline, illustrating the transition from initialization to iterative PSO-based updates.

Evaluation Highlights

+261.8% relative improvement over ADAS on the TravelPlanner benchmark, highlighting effectiveness in structurally unconstrained tasks
Achieved state-of-the-art performance on 6 real-world tasks (including Natural Plan and Creative Writing) given only a task description and objective function
Outperforms GPT-4o-based direct prompting by large margins (e.g., +41.9% score improvement on Creative Writing)

Breakthrough Assessment

9/10

Significantly advances automated agent design by removing the need for seed agents or templates. The adaptation of PSO to symbolic/language space is a novel and effective mechanism for structural search.

⚙️ Technical Details

Problem Definition

Setting: Automated generation of an agentic system S comprising an agent set A and collaborative structure W to maximize a task-specific fitness function J(S).

Inputs: Task description and an objective function (fitness metric)

Outputs: Optimized agentic system S (roles, responsibilities, policies, and workflow steps)

Pipeline Flow

Initialization: Generate diverse population of agentic systems
Loop: Evaluate → Flaw Identification → Velocity Update → Position Update
Output: Best found agentic system

System Modules

Particle Initialization

Generate initial population of agentic systems with temperature-controlled sampling for diversity

Model or implementation: GPT-4o-mini

Flaw Identification (Optimization Loop)

Analyze execution results against objective function to find agent and structure flaws

Model or implementation: GPT-4o-mini

Failure-Aware Velocity Update (Optimization Loop)

Generate textual optimization plan by integrating failure history, personal best, and global best guidance

Model or implementation: GPT-4o-mini

Position Update (Optimization Loop)

Apply the velocity plan to transform the agentic system structure (add/remove agents, reorder steps)

Model or implementation: GPT-4o-mini

Novel Architectural Elements

Language-driven PSO: Reinterpretation of numerical PSO vectors (velocity/position) as structured text and semantic transformations
Joint Optimization: Simultaneous refinement of Agent Set (roles/policies) and Collaborative Structure (workflow/dependencies) in a single loop
Failure-Aware Velocity: Explicit memory mechanism where the optimizer analyzes past failed updates to prevent cyclic errors

Modeling

Base Model: GPT-4o-mini (Optimizer), GPT-3.5-turbo/GPT-4o/Claude-3.5/DeepSeek-V3/Gemini-1.5 (Executors)

Comparison to Prior Work

vs. ADAS: SwarmAgentic initializes from scratch (no seed agents) and uses population-based PSO rather than single-trajectory meta-agent search
vs. AutoAgents: Optimizes collaboration structure dynamically rather than using fixed collaboration templates
vs. EvoAgent: Jointly optimizes structure and agents; EvoAgent relies on fixed workflow structures
+ 1 more
vs. GPTSwarm [not cited in paper]: GPTSwarm optimizes graph connections; SwarmAgentic optimizes the semantic definition of agents and the graph structure simultaneously from scratch

Limitations

Relies on LLMs for reasoning, making it susceptible to hallucinations propagating through optimization cycles
Operating purely in text limits applicability to embodied or multimodal environments without external integration
Lack of inductive priors (templates) might slow convergence compared to methods using strong domain-specific starting points

Reproducibility

Code: https://github.com/SwarmAgentic

Publicly available code at github.com/SwarmAgentic. Uses GPT-4o-mini as the optimizer. Experiments use specific model versions (GPT-4o-0806, GPT-3.5-turbo-0125, etc.).

📊 Experiments & Results

Evaluation Setup

Evaluated on 6 open-ended tasks using only task descriptions and objective functions.

Benchmarks:

TravelPlanner (TP) (Long-horizon planning with constraints)
Natural Plan (NP) (Multi-agent scheduling (Trip, Meeting, Calendar))
Creative Writing (CW) (Coherent multi-paragraph generation)
MGSM (Multilingual math reasoning)

Metrics:

Delivery Rate
Commonsense Pass Rate
Hard Constraint Pass Rate
Final Pass Rate (Micro/Macro)
Coherence Score (CW)
Accuracy (MGSM)
Statistical methodology: Not explicitly reported in the paper

Key Results

Benchmark	Metric	Baseline	This Paper	Δ
TravelPlanner results showing massive gains over baselines, particularly in handling hard constraints.
TravelPlanner	Final Pass Rate (Macro)	8.9	32.2	+23.3
TravelPlanner	Hard Constraint Pass Rate (Macro)	50.2	66.7	+16.5
Natural Plan and MGSM results showing consistent state-of-the-art performance.
Natural Plan - Trip Planning	Accuracy	3.7	13.1	+9.4
Natural Plan - Meeting Planning	Accuracy	50.0	56.0	+6.0
MGSM	Accuracy	87.3	88.4	+1.1
Creative Writing	Score	8.4	8.8	+0.4
Creative Writing	Score	6.4	9.0	+2.6

Experiment Figures

Search trajectory on TravelPlanner showing Success Rate improvement over iterations.

Main Takeaways

SwarmAgentic consistently outperforms fixed baselines (CoT, Self-Refine) and automated frameworks (ADAS, EvoAgent) on open-ended planning tasks.
The framework shows strong cross-model transferability; systems optimized by GPT-4o-mini perform well when executed by DeepSeek-V3, Claude-3.5, and Gemini-1.5.
Ablation studies confirm that all three PSO components (Failure-Driven Adjustments, Personal Best, Global Best) are critical for peak performance.
Full automation does not trade off performance on structured tasks (MGSM), while excelling at unstructured tasks (Creative Writing, TravelPlanner).

📚 Prerequisite Knowledge

Prerequisites

Particle Swarm Optimization (PSO) algorithm fundamentals
Large Language Model (LLM) prompting and role-playing
Multi-agent system architectures

Key Terms

PSO: Particle Swarm Optimization—a computational method that optimizes a problem by iteratively trying to improve a candidate solution (particle) with regard to a given measure of quality

Agentic System: A system where autonomous agents with specific roles collaborate to solve complex tasks

Velocity Update: In SwarmAgentic, the generation of a textual optimization plan based on failures, personal bests, and global bests

Position Update: The application of the velocity update plan to modify the actual code/structure of the agentic system

Flaw Identification: An LLM-driven process that analyzes execution logs to categorize errors into agent flaws (e.g., missing role) or structure flaws (e.g., bad sequencing)

Collaborative Structure: The defined workflow or graph that dictates how agents interact and the order of task execution

MGSM: Multilingual Grade School Math benchmark—a dataset for testing mathematical reasoning capabilities

ADAS: Automated Design of Agentic Systems—a baseline framework that uses a meta-agent to discover agent systems in code

Direct Prompting: Using a standard LLM prompt to solve a task without a multi-agent structure