OFA-MAS: One-for-All Multi-Agent System Topology Design based on Mixture-of-Experts Graph Generative Models

📝 Paper Summary

Multi-Agent Systems (MAS) Agent Collaboration Topology

OFA-MAS is a universal graph generative framework that designs adaptive Multi-Agent System topologies for any task domain using a single model, replacing domain-specific specialized designers.

Core Problem

Existing MAS topology design methods follow a 'one-for-one' paradigm, training specialized models for specific domains, which fails to generalize to unseen cross-domain queries and cannot leverage shared structural patterns.

Why it matters:

Real-world web services (e.g., search engines) face queries from unpredictable domains, making domain-specific models impractical to deploy
Training separate models for every new task creates an intractable maintenance burden and prohibitive scalability costs
Isolated models fail to learn abstract, reusable collaboration patterns (e.g., 'Analyst → Solver' flows) that exist across different disciplines like math and coding

Concrete Example: A mathematical theorem proving task and a software debugging task often share a similar collaboration pattern (e.g., 'Analyst → Inspector → Solver'). A specialized model trained only on math cannot transfer this structural knowledge to debugging, while separate models for each require redundant training.

Key Novelty

Universal 'One-for-All' MAS Topology Designer

Reframes topology design as an autoregressive graph generation problem where a single model constructs agent roles and connections step-by-step based on task semantics
Uses a Mixture-of-Experts (MoE) architecture to dynamically route generation decisions to specialized sub-networks, allowing one model to adapt its 'design strategy' for diverse task types
Employs a three-stage training curriculum (unconditional priors → LLM-synthesized mapping → empirical fine-tuning) to build cross-domain capability without catastrophic forgetting

Architecture

The architecture of the OFA-MAS framework, highlighting the Task-Aware Graph State Encoder (TAGSE) and the Mixture-of-Experts (MoE) generation module.

Breakthrough Assessment

8/10

Pioneers the shift from 'one-for-one' to 'one-for-all' in MAS design. The combination of MoE for structural adaptability and a synthesized training curriculum addresses key scalability hurdles in agentic systems.

⚙️ Technical Details

Problem Definition

Setting: Conditional graph generation where a model maps a task query and a universal role pool to a directed acyclic graph (DAG) representing agent collaboration.

Inputs: Natural language task query Q and a universal set of agent roles R_OFA

Outputs: Optimal collaboration topology G = (V, E), defining agents (nodes) and information flow (edges)

Pipeline Flow

Task Encoding (Query → Task Vector)
Graph State Encoding (Partial Graph + Task Vector → Node Embeddings)
Role Prediction (MoE selects next Agent Role)
Edge Prediction (MoE predicts connections to new Agent)

System Modules

Task Encoder

Convert natural language query into a global conditioning vector

Model or implementation: Sentence Transformer + MLP

Task-Aware Graph State Encoder (TAGSE)

Encode the partially generated graph while filtering information irrelevant to the current task

Model or implementation: Context-Gated GNN

Expert Gating Network

Decide which expert networks to activate for the current generation step

Model or implementation: Lightweight MLP

MoE Prediction Heads

Predict the next node role and incoming edges using selected experts

Model or implementation: Mixture of Experts (Ensemble of MLPs)

Novel Architectural Elements

Task-Aware Graph State Encoder (TAGSE) with context-gated message passing to inject global task context into local node updates
Mixture-of-Experts (MoE) generation module that dynamically routes topology design decisions to specialized sub-networks based on task type

Modeling

Base Model: Custom Autoregressive Graph Generative Model (OFA-MAS)

Training Method: Three-stage curriculum learning

Objective Functions:

Purpose: Ensure accurate reproduction of optimal graph structures.

Formally: L_graph = L_node (CrossEntropy) + L_edge (Binary CrossEntropy)
Purpose: Encourage usage of all experts to maximize model capacity.

Formally: L_balance (Load balancing auxiliary loss)
Purpose: Promote selective information flow in the encoder.

Formally: L_gate (L1 regularization on gate values)

Training Data:

Stage 1: D_graphs (Canonical topologies like Chain, Star, FullyConnected)
Stage 2: D_gen (Large-scale synthetic 'query-topology' pairs generated by LLM)
Stage 3: D_fine-tune (Empirically validated high-performing topologies)

Compute: Not reported in the paper

Reproducibility

Code: https://github.com/Shiy-Li/OFA-MAS

Code is publicly available at https://github.com/Shiy-Li/OFA-MAS. The paper describes an LLM-driven data synthesis pipeline used to create training data. The specific LLM used for data synthesis is not specified in the extracted text.

📊 Experiments & Results

Evaluation Setup

Evaluation of generated MAS topologies on diverse task domains.

Benchmarks:

Six diverse multi-domain benchmarks (Cross-domain MAS tasks (e.g., reasoning, coding))

Metrics:

Not explicitly reported in the paper
Statistical methodology: Not explicitly reported in the paper

Main Takeaways

OFA-MAS successfully transitions MAS topology design from a 'one-for-one' to a 'one-for-all' paradigm, enabling a single model to handle diverse domains.
The Mixture-of-Experts architecture allows the model to specialize its generation strategy (e.g., switching between sequential chains for math and hierarchical reviews for coding) without separate training.
The three-stage training strategy (Unconditional Prior -> LLM Synthesis -> Empirical Tuning) effectively mitigates the data scarcity problem for optimal topology training data.

📚 Prerequisite Knowledge

Prerequisites

Multi-Agent Systems (MAS)
Graph Neural Networks (GNNs)
Autoregressive Generation
Mixture of Experts (MoE)

Key Terms

MAS: Multi-Agent System—a network of autonomous agents that collaborate to solve complex problems

Topology: The structural design of a MAS, determining which agents participate and how they communicate (the 'org chart' of the agents)

TAGSE: Task-Aware Graph State Encoder—a module proposed in this paper that filters node information based on the specific task query using gated message passing

MoE: Mixture of Experts—a neural architecture where different sub-networks ('experts') are dynamically selected to handle different inputs

One-for-One: A design paradigm where a specific model is trained for and applicable to only one specific task domain

One-for-All: A design paradigm where a single universal model handles tasks across multiple diverse domains

Autoregressive: Generating a structure one part at a time, where each step depends on the previous steps (e.g., adding node 1, then node 2, then edges)