Evaluation Setup
Large-scale security measurement of open-source MCP servers
Benchmarks:
- Real-world MCP Server Corpus (Vulnerability Scanning) [New]
- Top-Starred Projects Subset (Manual Security Audit) [New]
Metrics:
- Number of insecure servers
- Percentage of servers with Caller Identity Confusion
- Number of confirmed RCE vulnerabilities
- Statistical methodology: Not explicitly reported in the paper
Key Results
| Benchmark |
Metric |
Baseline |
This Paper |
Δ |
| Large-scale measurement results quantifying the prevalence of Caller Identity Confusion across the MCP ecosystem. |
| Real-world MCP Server Corpus |
Insecure Servers (Count) |
6137 |
2846 |
46.4%
|
| Real-world MCP Server Corpus |
Insecure Developer Tools (Percentage) |
100 |
52 |
52%
|
| Detailed audit of high-profile projects reveals critical exploitable vulnerabilities. |
| Top-Starred Projects Subset |
Projects with RCE |
87 |
8 |
9.2%
|
Main Takeaways
- Caller Identity Confusion is pervasive (46.4% of servers), not isolated to low-quality projects.
- Insecure patterns persist across all functional domains, with Developer Tools being the most affected (52%).
- The vulnerability enables high-impact attacks including Remote Command Execution (RCE) and unauthorized UI control without requiring credential theft.