🔍 Agentic AI

What: This topic covers research on LLM-based agents that generate a plan for completing a task and then execute it by making multiple sequential or parallel tool calls. The plan is typically generated upfront and followed during execution, with methods varying in how they train, evaluate, and secure such agents.

Why: Real-world tasks—from travel planning to scientific discovery—require LLMs to go beyond text generation and actively interact with external tools (APIs, code interpreters, databases) across multiple steps. Enabling reliable, efficient multi-step tool use is essential for deploying agents in high-stakes, production environments.

Baseline: The conventional baseline is a single-step prompted LLM that either answers from parametric knowledge alone or uses naive Chain-of-Thought prompting. For tool use, a simple ReAct loop where the model alternates between reasoning and single tool calls without structured planning or learning serves as the starting point.

• Long-horizon planning coherence: agents must maintain consistent plans across 5-20+ sequential tool calls, where small early errors compound into catastrophic failures
• Tool selection at scale: real-world environments expose hundreds of tools with overlapping semantics, and agents must correctly identify and parameterize the right ones
• Training signal sparsity: reinforcement learning for multi-step tool use faces sparse rewards (only final success/failure), making credit assignment to intermediate steps extremely difficult
• Security and reliability: agents with tool access can cause irreversible real-world state changes, and are vulnerable to prompt injection, tool misuse, and hallucinated tool calls

What: Tool creation and profiling encompasses methods that enable LLMs to autonomously create new tools (as reusable code) and generate or optimize detailed tool descriptions (profiles) so that agents can select the right tool and invoke it with correct parameters.

Why: As LLM agents are expected to handle thousands of diverse APIs, pre-defined toolsets become a bottleneck: they cannot cover every task, their documentation is often noisy or incomplete, and poor descriptions cause tool selection failures and parameter hallucinations.

Baseline: The conventional approach provides LLMs with raw, human-written API documentation and a fixed set of pre-implemented tools, relying on few-shot demonstrations or simple prompting to guide tool selection and argument generation.

• Tool documentation is heterogeneous, incomplete, or overly verbose, causing LLMs to misunderstand tool capabilities and generate incorrect parameters
• Scaling to thousands of tools exceeds context limits, requiring effective retrieval and ranking to surface the right tool from massive libraries
• Creating new tools on-the-fly demands the LLM handle complex dependencies (package installation, environment setup) and verify correctness autonomously
• Aligning tool-use training data with real-world complexity is difficult—synthetic data often contains parameter errors, and real API responses are noisy

What: Tool-use post-training encompasses methods that teach LLMs to understand tool descriptions, select appropriate tools, and generate correct tool calls through fine-tuning, instruction tuning, reinforcement learning, and synthetic data generation.

Why: While LLMs excel at text reasoning, they struggle with precise computation, real-time information retrieval, and API invocation—capabilities that external tools can provide. Post-training bridges this gap, transforming general-purpose models into effective tool-using agents.

Baseline: The conventional approach relies on few-shot prompting or simple supervised fine-tuning on small, manually annotated datasets, which limits tool diversity, fails to teach models when NOT to use tools, and often causes hallucinated API calls.

• Generating diverse, high-quality training data that covers realistic multi-tool, multi-turn scenarios at scale without prohibitive human annotation costs
• Teaching models to decide WHEN to use tools (avoiding unnecessary calls on simple tasks) and WHICH tool to select from massive, evolving tool libraries
• Preventing hallucinated API calls—models frequently invent non-existent tools or generate incorrect parameters, especially for lesser-known libraries
• Maintaining general language understanding and instruction-following capabilities while specializing for tool use, avoiding catastrophic forgetting

What: Tool retrieval and selection addresses how LLM-based agents identify and choose the most appropriate tool(s) from a large, often dynamic library given a natural language task description. It spans retrieval, ranking, filtering, and decision-making over tool inventories that can range from dozens to tens of thousands of APIs.

Why: As the ecosystem of available tools and APIs grows into the thousands, it becomes infeasible to inject all tool descriptions into an LLM's context. Efficient, accurate tool retrieval is the critical bottleneck that determines whether an agent can successfully leverage external capabilities.

Baseline: The conventional approach either injects all tool descriptions into the prompt (full-prompt injection), which causes high latency and confusion at scale, or uses simple dense vector similarity between user queries and tool documentation, which suffers from semantic mismatch and ignores tool dependencies.

• Semantic gap: User queries are expressed in natural, high-level language while tool documentation is technical and heterogeneous, causing standard retrievers to miss relevant tools.
• Scalability: Context window limits prevent loading thousands of tool descriptions simultaneously, requiring efficient pre-filtering without sacrificing recall.
• Tool interdependencies: Many tasks require multiple tools used in sequence, but retrievers treat each tool independently, missing prerequisite tools that are semantically unrelated to the query.
• Dynamic tool inventories: Tools are frequently added, updated, or deprecated, requiring selection methods that generalize to unseen tools without retraining.

What: This topic covers AI agents that generate multi-step plans for complex tasks and execute them by calling external tools (search engines, code interpreters, APIs), dynamically adapting the plan based on intermediate results. It encompasses the broad design space of flexible planning with tool use that does not fit narrowly into specific sub-topics like code generation or retrieval-only settings.

Why: Real-world tasks rarely decompose into a single query or a fixed pipeline. Agents must reason about what information to gather, which tools to invoke, and how to revise their strategy when initial results are unexpected—capabilities essential for autonomous scientific discovery, software engineering, web navigation, and enterprise automation.

Baseline: The conventional approach is single-turn prompting or static retrieval-augmented generation (RAG), where an LLM generates an answer in one pass, possibly after a single retrieval step. These baselines fail on multi-step tasks because they cannot iteratively refine their approach, recover from errors, or coordinate across multiple tools.

• Credit assignment in long-horizon tasks: sparse outcome rewards make it hard to identify which intermediate actions were critical versus irrelevant, leading to inefficient RL training.
• Balancing exploration and exploitation: agents must decide when to gather more information (explore) versus when to act on current knowledge (exploit), avoiding both overthinking and premature commitment.
• Scalability of experience generation: training agentic models requires generating diverse multi-turn interaction trajectories, which is orders of magnitude slower than static dataset training.
• Coordination and error recovery: in multi-agent systems, failures can cascade through agent chains, and identifying the root cause of failure across long execution traces remains an open challenge.

What: This topic covers methods where tool APIs are either well understood by the model (e.g., web search, calculator, code interpreter) or internalized into the model's parameters, enabling the agent to invoke them fluently without explicit API specifications.

Why: When models deeply internalize how tools work, they can discover novel and more effective strategies for tool invocation rather than rigidly following demonstrated patterns, leading to stronger reasoning and problem-solving capabilities.

Baseline: The conventional approach uses Supervised Fine-Tuning (SFT) on distilled tool-use trajectories, training models to imitate fixed patterns of tool invocation demonstrated by stronger models or human annotations.

• SFT-based tool training restricts models to imitating demonstrated patterns, preventing exploration of potentially superior tool-use strategies
• Integrating external tool execution (e.g., code interpreters) into the reinforcement learning loop introduces complexity in managing asynchronous interactions and reward assignment
• Ensuring that models learn when and how to invoke internalized tools effectively rather than defaulting to purely textual reasoning

What: RL-based Tool Use trains language model agents to autonomously invoke external tools (search engines, code interpreters, APIs) through reinforcement learning, optimizing multi-turn interaction policies with reward signals derived from final task success, intermediate step quality, or tool call correctness.

Why: Standard prompting and supervised fine-tuning teach agents what actions to take but not when or why, leading to brittle behaviors like excessive searching, hallucinated tool calls, or failure to recover from errors. RL enables agents to learn adaptive strategies through trial-and-error interaction with real environments.

Baseline: The conventional approach uses supervised fine-tuning on expert trajectories (imitation learning) or few-shot prompting with large proprietary models. These methods produce agents that mimic demonstrated tool use patterns but cannot generalize to novel situations or learn from failures.

• Sparse rewards: In multi-turn tool use, only the final outcome provides a reward signal, making it extremely difficult to assign credit to individual tool calls across long trajectories
• Training instability: Multi-turn interactions create non-stationary dynamics and off-policy drift, frequently causing training collapse where the agent degenerates into repetitive or empty tool calls
• Capability interference: Jointly optimizing reasoning and tool-use skills on shared model parameters causes gradient conflicts, where improving one capability degrades the other
• Exploration difficulty: The combinatorial space of multi-step tool interactions makes it nearly impossible for agents to discover successful trajectories through random exploration alone

What: Reflection-based reasoning equips LLM agents with the ability to analyze their own failed or suboptimal tool-use attempts—comparing them against expert actions or successful outcomes—to diagnose errors and improve future decisions.

Why: Standard imitation learning teaches agents what to do but not why, leaving them brittle when they encounter unfamiliar states or evolving tool environments; reflection closes this gap by enabling agents to learn from mistakes at inference or training time.

Baseline: Conventional approaches use single-episode reinforcement learning with sparse outcome rewards, or supervised fine-tuning on static expert demonstrations, neither of which teaches the agent to reason about why one action is better than another.

• Credit assignment over multi-step tool-use trajectories is difficult when only a final sparse reward is available, making it hard to identify which intermediate actions caused failure
• Real-world tools and APIs evolve over time (renamed parameters, deprecated endpoints), so agents trained on static documentation degrade when deployed in dynamic environments
• Imitation of pre-generated critique text does not produce genuine reasoning—agents learn to parrot reflections rather than develop transferable discriminative judgment
• Exploring the combinatorial space of possible tool calls and argument values is intractable without structured search, yet greedy step-by-step reasoning gets trapped in local optima

What: This topic covers research on AI agents that engage in multi-turn interactions with users or other agents, spanning task decomposition, dialogue management, tool use, and iterative refinement across extended conversational contexts.

Why: Real-world tasks rarely resolve in a single exchange; they require agents to gather information incrementally, handle ambiguity, adapt to feedback, and coordinate multiple steps—capabilities that static single-turn systems fundamentally lack.

Baseline: The conventional approach uses single-turn prompting or basic retrieval-augmented generation (RAG), where a user query is processed in one pass without iterative refinement, feedback loops, or dynamic task decomposition.

• Maintaining coherent context and intent across many interaction turns without information loss or hallucination
• Balancing agent autonomy with user control—knowing when to act independently versus when to seek human guidance
• Scaling multi-agent coordination without exponential cost growth in compute, latency, and token consumption
• Ensuring safety, privacy, and trust as agents gain access to tools, personal data, and external services over extended interactions

What: This topic covers systems and frameworks where humans and AI agents iteratively co-specify tasks, share control, and refine outcomes through interactive feedback loops—ranging from co-planning interfaces to human-in-the-loop multi-agent pipelines deployed in safety-critical domains.

Why: As AI agents grow more autonomous, purely automated systems frequently fail on complex real-world tasks, produce unsafe actions, or misalign with user intent. Interactive collaboration enables humans to inject domain expertise, maintain oversight, and steer agents toward better outcomes than either party achieves alone.

Baseline: The conventional approach treats AI as either a fully autonomous agent that executes tasks end-to-end without human input, or a passive tool that responds only to explicit user prompts—neither of which adequately handles the nuanced, iterative nature of complex real-world tasks.

• Calibrating the right level of human control: too much oversight negates efficiency gains, while too little risks unsafe or misaligned outcomes
• Designing interaction modalities that allow fluid, low-friction handoffs between human and AI without disrupting workflow or cognitive flow
• Ensuring safety and trust in high-stakes domains where AI errors carry significant consequences (healthcare, scientific facilities, finance)
• Measuring and achieving genuine human-AI complementarity rather than simple task delegation, where the team outperforms either party alone

What: Conversational agent design encompasses patterns and frameworks for building multi-turn AI systems that maintain context across dialogue turns, manage user state, adopt consistent personas, and deliver natural interactions in domains such as mental health, education, and healthcare.

Why: As LLM-powered conversational agents become widely deployed in sensitive domains like therapy and clinical care, designing agents that sustain coherent, ethical, and psychologically safe multi-turn interactions is critical to user trust and real-world effectiveness.

Baseline: Traditional conversational agents use rule-based or retrieval-based dialogue management with scripted responses, treating each exchange as largely independent and offering generic, one-size-fits-all interactions without persistent persona or user adaptation.

• Maintaining persona consistency and avoiding identity hallucination across extended multi-turn conversations
• Ensuring psychological safety and ethical interaction beyond surface-level content filtering (e.g., detecting cumulative relational harms)
• Bridging the intention-action gap—moving users from receiving information to actually changing behavior through proactive, coaching-style dialogue
• Adapting conversational strategies across diverse domains (mental health, education, precision medicine) while grounding responses in domain-specific evidence

What: Multi-task planning addresses scenarios where an AI agent must decompose a large goal into multiple subtasks and coordinate their execution — spanning task decomposition, scheduling, workflow generation, and cross-task dependency management.

Why: Real-world problems rarely consist of a single atomic action; they require agents to plan across many interdependent steps while managing resources, constraints, and unforeseen failures. Getting this right is essential for deploying agents in enterprise, scientific, and safety-critical domains.

Baseline: The conventional approach uses a single LLM in a plan-then-execute loop: the model generates a sequential plan in natural language (or PDDL), then attempts to execute each step one by one, with human oversight at each decision point.

• Task decomposition quality: breaking a complex goal into the right subtasks without omitting critical steps or introducing irrelevant ones
• Scalability in long-horizon settings: as the number of subtasks and objects grows, LLMs suffer from context overload, hallucinations, and compounding errors
• Robustness and consistency: identical tasks phrased differently can yield wildly different workflows, undermining reliability
• Safety and alignment: agents with elevated privileges can leak private data, execute harmful actions, or drift from the user's original intent during multi-step execution

What: Task decomposition and subtask management covers methods that break complex goals into smaller, structured subtasks—tracking dependencies between them and determining execution order to enable efficient, parallelizable workflows for LLM-based agents.

Why: Real-world agent tasks (code generation, genomics analysis, safety filtering) are too complex for a single monolithic LLM call; decomposing them into focused subtasks reduces error rates, enables specialization, and unlocks parallel execution.

Baseline: The conventional approach is sequential chain-of-thought or single-pass prompting, where the LLM attempts to solve an entire complex task in one generation step without explicit subtask structure or dependency management.

• Determining the right granularity of decomposition—too coarse loses the benefit, too fine adds orchestration overhead
• Tracking dependencies between subtasks so that parallel execution does not violate ordering constraints
• Recovering gracefully when an individual subtask fails at runtime without restarting the entire workflow
• Scaling decomposition strategies to domain-specific tasks (e.g., genomics, social science) where subtask boundaries require expert knowledge

What: This topic covers methods that enable AI agents to accomplish complex tasks requiring many sequential steps by decomposing high-level goals into structured layers of increasingly concrete sub-tasks and actions.

Why: Real-world tasks such as assembling items in Minecraft, navigating websites, or coordinating robot teams involve tens to hundreds of dependent steps; flat planning approaches collapse under this combinatorial complexity, demanding hierarchical abstractions.

Baseline: The conventional approach uses a single-level LLM or RL policy that maps goals directly to low-level actions, often failing at long horizons due to compounding errors, context-window limits, and inability to recover from mid-plan failures.

• Compounding errors: small mistakes early in a long plan cascade, making later steps unreachable without structured error detection and correction
• Abstraction alignment: high-level sub-goals must be faithfully translatable into executable low-level actions, yet mismatches between planner assumptions and execution reality are common
• Scalability to real-world complexity: plans must handle dynamic environments, partial observability, and coordination among multiple agents over extended horizons
• Knowledge grounding: agents need access to domain-specific knowledge (recipes, object properties, spatial layouts) that LLMs may hallucinate without external retrieval or verification

What: Dynamic task routing and scheduling addresses how autonomous agents—whether software-based or physically embodied—discover, allocate, and redistribute tasks in real time as conditions, capabilities, and priorities shift.

Why: As multi-agent deployments scale from isolated prototypes to production fleets spanning cloud and edge infrastructure, rigid static allocation collapses under dynamic obstacles, heterogeneous capabilities, and economic constraints; adaptive routing is essential for robust, scalable coordination.

Baseline: Traditional approaches use hierarchical decomposition where a central planner assigns tasks to executors in a one-shot fashion, with no feedback loop between execution difficulty and the allocation decision, leading to bottlenecks and redundant work.

• Bridging the gap between global task allocation and local execution realities—allocators often lack awareness of on-the-ground navigability or agent load, causing clustering and deadlock.
• Achieving decentralized coordination without explicit communication—agents must implicitly negotiate roles and spatial coverage using only local observations.
• Ensuring incentive compatibility and fair compensation when heterogeneous agents from different organizations dynamically form coalitions across ownership boundaries.
• Scaling coordination mechanisms from small homogeneous teams to large heterogeneous fleets spanning cloud and edge, while maintaining low-latency task matching.

What: This topic covers AI agents that autonomously improve their reasoning, workflows, and decision-making over time by incorporating feedback, adapting strategies, and accumulating experience without constant human intervention.

Why: Static AI agents cannot adapt to new tasks, shifting environments, or increasing complexity without manual retraining, creating a bottleneck for real-world deployment at scale.

Baseline: Conventional approaches use fixed prompts, static workflows, and uniform reasoning effort across all tasks, relying on human engineers to manually redesign agent pipelines when performance degrades or requirements change.

• Balancing computational cost with reasoning quality: high-effort reasoning is expensive, but low-effort reasoning degrades performance significantly (up to ~20% drop)
• Designing evolution mechanisms that generalize across domains without task-specific hand-tuning of agent topologies and prompts
• Evaluating self-evolving agents reliably, since traditional outcome-only metrics miss intermediate reasoning quality and step-level improvements
• Avoiding catastrophic forgetting or drift during continuous self-improvement cycles

What: Feedback-driven self-improvement encompasses agent architectures that integrate evaluative signals—from self-generated annotations, peer reviews, environmental outcomes, or judge models—into closed-loop refinement cycles that autonomously improve reasoning, task execution, and resource allocation.

Why: Static agent pipelines degrade on complex or shifting tasks; feedback-driven loops are essential for agents to adapt autonomously, but the reliability and design of that feedback fundamentally determines whether improvement or destabilization occurs.

Baseline: The conventional approach relies on fixed prompts or manually configured multi-agent pipelines without iterative self-correction, requiring human intervention to adapt to new domains, detect performance regressions, or allocate tasks efficiently across heterogeneous agent pools.

• Ensuring feedback reliability: judge models can hallucinate, exhibit bias, or be adversarially manipulated, causing agents to abandon correct solutions
• Designing self-annotation and discriminator loops that produce high-quality training signal without labeled data
• Scaling feedback-driven improvement to heterogeneous agent pools with varying capability levels and cost profiles
• Detecting and recovering from performance regressions in deployed autonomous systems without human oversight

What: This topic covers methods that enable AI agents to evaluate their own outputs, identify mistakes or suboptimal decisions, and iteratively refine their reasoning and actions through self-generated or externally provided feedback.

Why: Complex agentic tasks often have low success rates (20-30%), and single-pass generation rarely produces optimal solutions. Self-reflection allows agents to learn from their own failures during inference, closing the gap without requiring additional training data or human supervision.

Baseline: The conventional approach is single-pass generation or Best-of-N (BoN) sampling, where multiple candidate solutions are generated independently and the best is selected. These baselines treat each attempt in isolation, unable to learn from prior failures within a task.

• Converting numerical or scalar feedback into actionable guidance that helps models improve specific aspects of their output
• Smaller or less capable models often fail to recognize their own errors, limiting the applicability of self-critique to only the most advanced models
• Many real-world environments lack verifiable reward signals, making it difficult to determine whether an agent's self-assessment is accurate

What: This topic covers methods by which AI agents accumulate knowledge from past interactions, integrate new experiences over time, and continuously improve their performance without forgetting prior capabilities.

Why: Static, pre-trained models are inherently bounded by their training data, making them brittle when faced with novel tasks or evolving knowledge. Continual learning enables agents to adapt autonomously, reducing the need for costly retraining while maintaining relevance in dynamic environments.

Baseline: The conventional approach relies on fixed pre-trained LLMs that do not update from deployment experience. When new knowledge is needed, the entire model must be retrained or prompted with static few-shot examples, leading to knowledge staleness and inability to learn from mistakes.

• Catastrophic forgetting: agents must incorporate new knowledge without overwriting previously learned skills or facts.
• Knowing when to learn: agents need metacognitive ability to recognize the boundaries of their own knowledge and decide when to seek external help versus act autonomously.
• Scalable knowledge sharing: in multi-agent settings, experience must be efficiently communicated and integrated across distributed units without central bottlenecks.
• Evaluation difficulty: measuring continual improvement is hard because standard benchmarks are static and do not capture longitudinal adaptation over time.

What: Multi-agent systems coordinate multiple LLM-powered agents—each with distinct roles, tools, or knowledge—to collaboratively solve tasks that exceed the capability of any single agent. This topic encompasses role differentiation, collaboration protocols, orchestration strategies, and collective evolution mechanisms.

Why: Complex real-world tasks (scientific discovery, software engineering, incident response) require diverse expertise that no single model can reliably provide. Multi-agent architectures decompose these tasks into specialized sub-problems, enabling parallel execution, iterative refinement, and emergent capabilities that monolithic systems cannot achieve.

Baseline: The conventional approach uses a single LLM prompted with all instructions at once, relying on chain-of-thought or few-shot prompting to handle complex tasks. This single-agent paradigm suffers from context window limitations, cascading hallucinations, inability to parallelize, and lack of built-in verification or self-correction.

• Cascading errors: mistakes by one agent propagate through the system, compounding into larger failures that are difficult to diagnose and attribute
• Coordination overhead: inter-agent communication, role assignment, and workflow orchestration add latency and token cost, sometimes exceeding the gains from decomposition
• Security and trust: multi-agent communication channels create novel attack surfaces including prompt infection, secret collusion, and cascading injection that single-agent safety measures cannot address
• Evaluation complexity: binary task-completion metrics fail to capture the non-deterministic, multi-step behavioral patterns of multi-agent workflows, making it hard to benchmark and compare systems

What: Role differentiation studies how multiple agents are assigned distinct functional roles—such as lead/worker hierarchies, specialist extractors, or moral perspectives—and how these role structures affect coordination, reliability, and emergent behavior in multi-agent systems.

Why: As multi-agent systems scale beyond simple tool-calling, the way roles are divided and coordinated fundamentally determines system reliability, alignment quality, and whether agents can self-organize without centralized control.

Baseline: A single monolithic LLM handles all sub-tasks (extraction, reasoning, synthesis) within one prompt or chain, with no explicit division of labor or cross-model validation.

• Role assignments can introduce structural instability: even at temperature zero, assigning roles like 'Chair' to committee members amplifies divergence across runs
• Aggregating outputs from role-differentiated agents without losing semantic coherence or introducing biases from dominant agents
• Designing decentralized coordination protocols that allow agents to discover, authenticate, and collaborate without centralized orchestrators
• Balancing specialization depth against the overhead of inter-agent communication and consensus resolution

What: This topic covers how multiple LLM-based agents exchange intermediate reasoning, negotiate consensus, and divide labor to solve tasks that exceed the capabilities of any single agent. It spans debate protocols, role-based orchestration, agent-to-agent communication standards, and dynamic ensemble methods.

Why: Complex real-world tasks—medical diagnosis, scientific research, software engineering—require diverse expertise, cross-validation of reasoning, and structured coordination that no single model can reliably provide. Multi-agent collaboration enables error correction through debate, specialized labor division, and scalable composition of heterogeneous capabilities.

Baseline: The conventional approach uses a single LLM (or simple chain-of-thought prompting) to handle all aspects of a task in one pass, sometimes augmented with self-consistency voting or self-reflection. These baselines lack external cross-examination, cannot divide specialized labor, and often suffer from hallucination consensus.

• Agents using identical models converge on shared blind spots, producing 'hallucination consensus' rather than genuine error correction
• Communication overhead grows rapidly with agent count, increasing latency and cost without guaranteed quality improvement
• No universal protocol exists for heterogeneous agents to discover, authenticate, and negotiate with each other across platforms
• Balancing agent autonomy with coordination—too much structure stifles adaptability, too little leads to incoherent or redundant outputs

What: Collective Evolution studies how groups of AI agents develop shared or distributed state—communication norms, coordination protocols, and adaptive behaviors—that sustain long-term cooperation and continual adaptation without centralized control.

Why: As autonomous AI agents are deployed at scale in social platforms, wireless networks, and resource-constrained environments, understanding how collective dynamics emerge (and sometimes fail) is critical for designing systems that remain stable, fair, and effective over time.

Baseline: Conventional multi-agent approaches rely on simple voting, unstructured debate, or centralized orchestration, treating agents as interchangeable rational actors who communicate via raw text without distinguishing reasoning move types or tracking evolving shared state.

• Emergent pathologies: agents may converge on formulaic or self-referential discourse rather than productive coordination, as seen when over 56% of AI-to-AI comments become ritualized signaling
• Sophistication paradox: increasing individual agent intelligence (learning, tribal sensing) can paradoxically worsen collective outcomes under resource scarcity, creating 'Lord of the Flies' dynamics
• Protocol design: structuring agent interactions to preserve genuine disagreement, avoid premature consensus, and guarantee bounded convergence remains an open challenge
• Scalability of shared state: maintaining coherent collective knowledge across thousands of agents with heterogeneous capabilities and evolving emotional or strategic states

What: Multi-agent simulation studies how multiple LLM-powered agents interact within virtual environments, examining emergent social behaviors, strategic reasoning, and collective dynamics at scale.

Why: Understanding how AI agents behave in social settings is critical for deploying them safely in high-stakes domains (military, policy, social platforms) and for using simulations as testbeds for alignment and safety research.

Baseline: Traditional approaches use rule-based or game-theoretic agent models with fixed strategies, or evaluate single LLMs in isolation on static benchmarks, missing the dynamic and emergent properties of multi-agent interaction.

• Emergent behaviors in multi-agent systems are unpredictable from single-agent evaluations, making safety guarantees difficult
• Scaling simulations to hundreds or thousands of agents while maintaining behavioral fidelity requires novel parallel architectures
• Validating that LLM agent behavior meaningfully reflects human social dynamics rather than model artifacts
• Calibrating environmental pressure to elicit complex behaviors without causing agent collapse or degenerate strategies

What: Multi-agent reinforcement learning (MARL) studies how multiple autonomous agents learn to coordinate, compete, or cooperate in shared environments, increasingly integrating large language models (LLMs) with RL for planning, trust assessment, and dynamic team formation.

Why: Complex real-world tasks—from network security to scientific discovery—require multiple agents to act jointly under partial observability and dynamic conditions, exceeding the capabilities of any single agent or static pipeline.

Baseline: Conventional approaches assign agents fixed, predefined roles with static coordination protocols, relying on centralized orchestration and hand-crafted rules that cannot adapt to changing task demands or adversarial interference.

• Quantifying and maintaining trust among agents when some may be unreliable, adversarial, or compromised during execution
• Scaling coordination protocols to dynamic environments where agent teams must be formed, dissolved, or restructured on the fly
• Detecting emergent misbehavior and compounding decision errors that arise only at runtime in multi-agent loops
• Bridging the gap between high-level LLM reasoning (which may hallucinate) and low-level RL control (which lacks generalization) in hybrid architectures

What: This topic covers foundational infrastructure, frameworks, protocols, and evaluation methodologies for building, deploying, and assessing agentic AI systems — autonomous LLM-powered agents that use tools, execute multi-step plans, and interact with real-world environments.

Why: As LLMs transition from passive question-answering to autonomous agents with file-system access, network connectivity, and tool use, new infrastructure is needed to ensure these systems are safe, observable, governable, and reliably evaluated.

Baseline: Conventional approaches treat agents as black boxes evaluated only on final outputs, rely on static compliance checks designed for traditional software, and use either costly manual red-teaming or hallucination-prone LLM simulators for safety testing.

• Agent behaviors are non-deterministic and context-sensitive, making reproducible evaluation and debugging extremely difficult
• Safety and security risks emerge from dynamic interactions between models, tools, and data — not from any single component in isolation
• Existing governance structures rely on episodic, siloed approvals that cannot oversee continuously operating autonomous agents
• Hallucinations in multi-step workflows propagate and compound across steps, but current methods cannot localize which step caused the initial error

What: This topic covers the software frameworks, platforms, and architectural patterns for building, deploying, scaling, and governing AI agent systems powered by large language models, including declarative specifications, workflow orchestration, security hardening, cost optimization, and production engineering practices.

Why: As LLM-based agents move from research prototypes to production deployments across enterprises and scientific domains, standardized frameworks are essential to ensure interoperability, reliability, security, cost-effectiveness, and trustworthy operation at scale.

Baseline: Early agent systems relied on ad-hoc prompt chaining, monolithic codebases, or framework-specific implementations (e.g., LangChain, AutoGen) that tightly coupled agent logic to a single runtime, making agents non-portable, difficult to test, expensive to operate, and vulnerable to security exploits.

• Framework fragmentation: agents defined in one system cannot be reused, compared, or executed in another due to incompatible abstractions and execution semantics
• Security brittleness: all 22 frontier models tested were compromised via prompt injection within 100 queries, and 46.6% of web agents execute malicious commands that standalone LLMs refuse
• Evaluation blind spots: benchmarks focus narrowly on accuracy while ignoring cost, reproducibility, and real-world robustness, leading to over-engineered agent architectures that are 3-5x more expensive than necessary
• Production-research gap: 70% of deployed agents use simple prompting rather than complex reasoning, and 74% rely on human evaluation, yet research continues to pursue autonomous multi-step systems

What: This topic covers standardized protocols for AI agent communication, tool integration, and interoperability—most prominently the Model Context Protocol (MCP), which defines a universal client-server interface enabling LLMs to connect with external tools and data sources.

Why: As AI agents proliferate, fragmented custom integrations create security gaps and scaling bottlenecks; standardized protocols are essential for secure, plug-and-play agent ecosystems.

Baseline: Before MCP, each AI agent integration required bespoke connector logic with ad-hoc authentication, inconsistent schemas, and no shared security model—making every new tool connection a one-off engineering effort.

• Optional protocol clauses create a gap between specification and implementation, leaving critical security guardrails unenforced in practice
• Stateful authorization models in MCP servers fail to distinguish between different callers, enabling identity confusion attacks across multi-agent environments
• No universal discovery mechanism exists for agents to find, verify, and trust one another across heterogeneous protocol ecosystems (MCP, A2A, ACP)
• The rapid adoption of MCP has outpaced security research, leaving protocol-layer vulnerabilities largely uncharacterized

What: This topic covers evaluation methodologies, benchmarks, metrics, and frameworks for assessing the capabilities, reliability, safety, and cost-effectiveness of LLM-based agents operating in dynamic, multi-step environments.

Why: As LLM agents move from research prototypes to real-world deployments in finance, healthcare, and web navigation, rigorous evaluation is essential to ensure they are safe, reliable, and cost-effective—not just accurate on narrow benchmarks.

Baseline: Conventional evaluation relies on static, single-turn benchmarks (e.g., MMLU, exact-match QA) that measure accuracy or F1 on isolated tasks, ignoring the sequential decision-making, tool use, cost, and safety dimensions critical to agentic systems.

• Agents operate in dynamic, multi-step environments where errors compound across turns, making single-metric accuracy scores misleading
• Evaluation stochasticity—the same agent on the same task can produce different results across runs—undermines reproducibility and meaningful comparisons
• LLM-based user simulators used for evaluation systematically overestimate agent quality compared to real human interactions, creating Sim2Real gaps
• Cost, safety, and determinism are orthogonal to accuracy but rarely measured, leading to over-engineered agents that are expensive and potentially unsafe

What: This category covers papers on LLM-based agent systems that span security, evaluation, governance, domain-specific applications, and infrastructure—topics that do not fit neatly into the main taxonomy of agent architectures, planning, memory, or tool use.

Why: As AI agents transition from research prototypes to production deployments, critical cross-cutting concerns—security vulnerabilities, reliable benchmarking, ethical governance, and real-world domain adaptation—must be addressed to enable safe and trustworthy autonomous systems.

Baseline: The conventional approach treats agents as isolated LLM instances evaluated on narrow, static benchmarks with post-hoc safety checks, manual security audits, and ad-hoc governance policies borrowed from traditional software systems.

• Agentic systems introduce novel attack surfaces (prompt injection, memory poisoning, tool misuse) that span multiple architectural layers and cannot be addressed by model-level safety alone
• Existing benchmarks are unreliable for measuring agent capabilities due to flawed reward designs, data contamination, and high stochastic variance from single-run evaluations
• Autonomous agents create governance gaps where no clear framework assigns accountability, manages risk attitudes, or enforces compliance across agent-human-environment interactions
• Domain-specific deployment requires grounding agents in specialized knowledge (medical protocols, industrial constraints, scientific domains) while preventing hallucination and ensuring verifiable correctness

What: This topic covers autonomous agent systems—exemplified by OpenClaw-style architectures—that execute real-world actions (financial trades, tool calls, robotic manipulation) on behalf of users, as well as robotic agents that physically grasp and manipulate tools through contact-rich sensing.

Why: As LLM-based agents gain the ability to act autonomously with high-privilege execution, ensuring their safety, security, trustworthiness, and learnability becomes critical to preventing catastrophic real-world failures.

Baseline: Conventional approaches treat agent outputs as direct commands and rely on prompt-level safeguards or single-modality perception, lacking systematic execution-layer defenses, continuous online learning from interaction signals, or multimodal contact sensing for physical manipulation.

• Execution-induced loss: agent errors translate directly into irreversible real-world consequences (financial losses, physical damage) rather than mere wrong answers
• Expanded attack surfaces: persistent memory, tool access, and skill supply chains create multi-stage security threats that point-based defenses cannot address
• Signal waste: valuable corrective signals from user replies, tool outputs, and environment changes are discarded rather than used for continuous policy improvement
• Contact complexity in physical manipulation: tool-environment interactions involve unobservable extrinsic contacts that vary across tools and tasks