← Back to Paper List

MarsCode Agent: AI-native Automated Bug Fixing

Yizhou Liu, Pengfei Gao, Xinchen Wang, Jie Liu, Yexuan Shi, Zhao Zhang, Chao Peng

Harbin Institute of Technology, Shenzhen

arXiv.org (2024)

Agent KG Benchmark

📝 Paper Summary

Multi-agent collaboration Automated Program Repair (APR)

MarsCode Agent automates bug fixing by coordinating specialized agents equipped with IDE-native tools like Language Server Protocols and Code Knowledge Graphs to replicate human debugging workflows.

Core Problem

Applying LLMs to real-world bug fixing is difficult because they struggle to navigate complex file dependencies, lack precise fault localization context, and often generate syntactically incorrect patches or invalid line numbers.

Why it matters:

Traditional heuristic-based repair tools have limited scope and cannot adapt to novel bugs
Simple LLM prompting fails on large repositories where the bug involves cross-file dependencies
Standard LLM edit formats (like unified diffs) are fragile due to line number hallucinations, breaking the patch application process

Concrete Example: When an LLM attempts to generate a 'unified diff' format patch, it frequently miscalculates line number increments, causing the patch to fail. Furthermore, without a reproduction script, an agent might 'fix' code but introduce new syntax errors that go undetected until manual review.

Key Novelty

IDE-Native Multi-Agent Collaboration

Integrates 'Language Server Protocol' (LSP) and 'Code Knowledge Graphs' (CKG) directly into the agent's toolset, allowing it to navigate definitions and references exactly like a human developer in an IDE
Uses a conflict-marker based editing strategy (AutoDiff) combined with LSP static syntax checking to ensure generated patches are syntactically valid and free of line-number errors
Deploys a dynamic planning system that chooses between a 'static repair' path (direct editing) or a 'dynamic debugging' path (reproduction -> fix -> verify) based on issue complexity

Architecture

Architecture Figure Figure 1

The multi-agent collaboration framework showing the roles and their interactions

Breakthrough Assessment

7/10

Strong engineering contribution integrating standard developer tools (LSP/CKG) into an agent framework. While the multi-agent concept is known, the specific tooling implementation for robustness is valuable.

⚙️ Technical Details

Problem Definition

Setting: Automated Bug Fixing in Software Repositories

Inputs: Bug report (issue description) and the target software repository

Outputs: A code patch (diff) that resolves the issue without introducing new errors

Pipeline Flow

Searcher collects context using CKG/LSP
Planner selects Static or Dynamic workflow
Dynamic: Reproducer creates script → Tester runs it → Programmer fixes code → Tester verifies
Static: Editor generates multiple candidates → Voting mechanism selects best

System Modules

Searcher

Collects relevant code snippets using CKG and LSP tools

Model or implementation: LLM (Unspecified variant)

Planner

Analyzes code and issue to classify the task as needing 'dynamic debugging' or 'static repair'

Model or implementation: LLM (Unspecified variant)

Reproducer (Dynamic Debugging)

Writes a reproduction script to confirm the bug exists

Model or implementation: LLM (Unspecified variant)

Programmer (Dynamic Debugging)

Iteratively edits code to fix the bug based on Tester feedback

Model or implementation: LLM (Unspecified variant)

Tester (Dynamic Debugging)

Executes the reproduction script to verify fixes

Model or implementation: N/A (Execution Environment)

Editor

Generates multiple candidate patches and selects the best via voting

Model or implementation: LLM (Unspecified variant)

Novel Architectural Elements

Integration of a Code Knowledge Graph (CKG) with Language Server Protocol (LSP) for retrieval, handling both cross-file semantic relationships and precise definition lookups
Conflict-marker based 'AutoDiff' module that performs a post-edit LSP static syntax check to reject invalid edits before they reach the testing phase

Modeling

Base Model: Large Language Models (Specific variants like GPT-4o or Doubao Pro are mentioned as examples of progress, but the specific model used for the agent is not detailed in the provided text)

Compute: Not reported in the paper

Comparison to Prior Work

vs. SWE-agent: MarsCode uses specialized IDE tools (LSP/CKG) for navigation instead of just grep/find, and separates the pipeline into static vs. dynamic workflows
vs. Agentless: MarsCode adds a dynamic debugging loop (Reproducer/Tester agents) in a sandbox, whereas Agentless is purely static
vs. Aider [cited as inspiration]: MarsCode wraps the conflict-based editing in a multi-agent framework with additional LSP-based syntax verification steps

Limitations

CKG cannot accurately retrieve definitions from external/third-party libraries (addressed partially by LSP)
LSP navigation relies on fuzzy positioning if the agent provides imprecise line numbers
Requires a reproduction script for the dynamic workflow; if the agent fails to write one, it falls back or fails

Reproducibility

The provided text is incomplete (cuts off at Section 3.2), so experimental results, specific model versions, and performance metrics are missing. The text mentions 'evaluated on SWE-bench' but does not provide the data. Code availability is not provided.

📊 Experiments & Results

Evaluation Setup

Automated bug fixing on real-world repositories

Benchmarks:

SWE-bench (Real-world software issue resolution (GitHub issues))

Metrics:

Success rate (implied by text, exact metric names not in snippet)
Statistical methodology: Not explicitly reported in the paper

Main Takeaways

The paper claims to outperform most existing automated bug fixing tools on SWE-bench, though specific numbers are not available in the provided text snippet.
The Multi-Agent framework allows flexibility: simple bugs are fixed via static analysis (Editor agent), while complex logic errors trigger a dynamic debugging loop (Reproducer/Tester agents).
The 'AutoDiff' tool addresses the common failure mode where LLMs cannot calculate line numbers for diffs; by using conflict markers and LSP syntax checking, the system ensures patches are at least syntactically valid.

📚 Prerequisite Knowledge

Prerequisites

Understanding of Large Language Models (LLMs) for code
Familiarity with software testing and debugging lifecycles
Knowledge of ASTs and static analysis

Key Terms

LSP: Language Server Protocol—a standard protocol used by IDEs (like VS Code) to provide features like 'Go to Definition' and syntax checking

CKG: Code Knowledge Graph—a graph data structure representing code entities (classes, functions) and their relationships (calls, inheritance) to aid retrieval

AutoDiff: A custom code editing tool that uses conflict markers (<<<<<<< SEARCH) instead of line numbers to locate and replace code, reducing LLM errors

AST: Abstract Syntax Tree—a tree representation of the abstract syntactic structure of source code, used here to normalize code candidates

SWE-bench: A benchmark for evaluating LLMs on real-world software engineering issues collected from Python repositories on GitHub

SBFL: Spectrum-based Fault Localization—a dynamic technique using test coverage to pinpoint buggy code

Unified Diff: A standard file format used to show changes between two versions of a file (using + and - lines)