Jatin Ganhotra, Sami Serhan, Antonio Abu Nassar, Avraham Shinnar, Ziv Nevo, Martin Hirzel
IBM
arXiv
(2026)
AgentBenchmarkReasoning
📝 Paper Summary
Software Engineering AgentsAutomated Program Repair
iSWE is a specialized Java coding agent that decomposes issue resolution into localization and editing sub-agents, utilizing rule-based static analysis and containerized compilation to ensure correctness.
Core Problem
Most automated software engineering agents are optimized for Python and perform poorly on Java due to its strict type system, compilation requirements, and verbose object-oriented structure.
Why it matters:
Enterprise software relies heavily on Java, yet existing leaderboards (like SWE-bench) and agents focus disproportionately on Python
Java's compiled nature and multi-file dependencies make Python-centric approaches (which often rely on simple linters or bash scripts) ineffective or unsafe
Generic agents often fail to localize bugs in large Java repositories where functionality is spread across many files
Concrete Example:In an `apache/rocketmq` issue, a generic agent might fail to identify that a fix requires editing a second file not explicitly mentioned in the bug report. iSWE's localization agent uses call-graph tools to find this dependency.
Key Novelty
Language-Aware Two-Stage Agency (iSWE)
Decomposes the task into two specialized ReAct (Reasoning + Acting) sub-agents: one dedicated solely to finding where to edit (Localization) and one to applying changes (Editing)
Replaces generic text-processing tools with strict, rule-based Java static analysis tools (via CLDK) to provide the LLM with reliable symbol, inheritance, and call-graph information
Validates edits using a containerized Java compiler and build environment rather than just a linter, ensuring type safety before submission
Architecture
The two-agent pipeline showing the flow from Issue Description to Final Code
Evaluation Highlights
Achieves state-of-the-art issue resolution rates on the Java splits of both Multi-SWE-bench and SWE-PolyBench [Specific percentages not included in input text]
Reduces LLM inference costs by 2x to 3x compared to leading agents by using efficient, language-aware tools that require fewer interaction turns
Demonstrates high localization capability, successfully identifying necessary code locations not mentioned in issue descriptions (e.g., in `apache/rocketmq`)
Breakthrough Assessment
7/10
Strong engineering contribution applying language-specific rigor to coding agents. Addresses a neglected domain (Java) with appropriate tooling, though the core architecture (ReAct) is standard.
⚙️ Technical Details
Problem Definition
Setting: Given an issue description and a code repository, generate a patch that passes fail-to-pass (F2P) tests without breaking pass-to-pass (P2P) tests.
Inputs: Issue description d_issue, current repository code c_old
Identify the set of file paths and line ranges that need modification
Model or implementation: LLM-agnostic (configurable)
Editing Agent
Generate the actual code patch based on identified locations
Model or implementation: LLM-agnostic (configurable)
Novel Architectural Elements
Two-agent decomposition where the intermediate output is a strictly typed, sanitized JSON object (locations) rather than natural language
Integration of rule-based Java analysis (CLDK) directly into the agent's toolset to replace approximate text search
Modeling
Base Model: LLM-agnostic (designed to work with various models; input text does not specify a single fixed model)
Compute: Runs primarily in user-space; only the compilation step of the Editing Agent runs in a containerized sandbox. 2x-3x cost reduction claimed vs baselines.
Comparison to Prior Work
vs. SWE-Agent: iSWE uses Java-specific static analysis tools and compiler feedback, whereas SWE-Agent relies heavily on Python-centric tooling
vs. CodeAct: iSWE uses structured, safe tool interfaces (PDL) instead of arbitrary code execution, avoiding side effects and sandbox overhead for read-only operations
Limitations
Depends on the availability of a buildable environment (Gradle/Maven) to get compiler feedback
Localization accuracy limits the editing agent; if the first agent misses a file, the second cannot fix it
Requires Java-specific tooling (CLDK), making it less immediately portable to other languages without similar static analysis libraries
Reproducibility
The system is built using open-source libraries PDL and CLDK. The specific iSWE agent code availability is not explicitly stated as a URL in the text. Evaluation uses public benchmarks (Multi-SWE-bench, SWE-PolyBench).
📊 Experiments & Results
Evaluation Setup
Execution-based evaluation on Java issue resolution benchmarks
Benchmarks:
Multi-SWE-bench (Java subset) (Issue Resolution)
SWE-PolyBench (Java subset) (Issue Resolution)
Metrics:
Issue resolution rate (percentage of issues fixed)
Localization precision and recall
Inference cost (USD)
Statistical methodology: Not explicitly reported in the paper
Key Results
Benchmark
Metric
Baseline
This Paper
Δ
The paper claims cost efficiency gains over other agents using the same LLM.
Multi-SWE-bench / SWE-PolyBench
Inference Cost
Not reported in the paper
Not reported in the paper
-
Main Takeaways
Specialized tooling allows iSWE to solve Java issues more cheaply (2x-3x cost reduction) by reducing the number of LLM turns required compared to generic agents
Decomposing the problem into Localization and Editing simplifies the context for the LLM, allowing it to focus on finding files first before attempting complex edits
Java issue resolution requires more than just file-level editing; it often involves multi-file changes and strict compilation checks that Python-centric agents miss
📚 Prerequisite Knowledge
Prerequisites
Understanding of ReAct (Reason+Act) agent loops
Familiarity with Java build systems (Maven/Gradle) and static analysis
Knowledge of SWE-bench evaluation metrics
Key Terms
ReAct: Reason+Act—a paradigm where LLMs generate reasoning traces and tool calls in an interleaved manner to solve tasks
PDL: Prompt Declaration Language—a declarative YAML-based language for managing prompts and control flow in agentic systems
CLDK: Cloud Native Language Development Kit—a program analysis framework used here to provide static analysis tools for Java
F2P: Fail-to-Pass—test cases that fail on the buggy code but must pass after the fix is applied
P2P: Pass-to-Pass—regression tests that passed before the fix and must continue to pass
Localization: The sub-task of identifying exactly which files and lines of code need to be modified to resolve an issue
Search-Replace Block: A text format used by the agent to specify edits, similar to merge conflict markers (<<<<<<< / ======= / >>>>>>>)