Evaluation Setup
Compliance and security analysis of 10 official MCP SDKs against the MCP specification (2025-06-08 version)
Benchmarks:
- Official MCP SDKs (Vulnerability Detection)
Metrics:
- False Positive Rate
- False Negative Rate
- Precision
- Recall
- Number of Discovered Risks
- Statistical methodology: Not explicitly reported in the paper
Main Takeaways
- The tension between agent diversity and standardization forces MCP to have many optional clauses (78.5%), creating an intrinsic attack surface
- Compliance gaps are pervasive: 1,270 non-implementations were found across 10 SDKs, with 99.6% (1,265) deemed exploitable
- The attack surface allows for severe consequences like silent prompt injection and Denial of Service (DoS)
- Community response confirms the practicality of the findings; manual reporting is impractical due to volume, leading to tool integration into the protocol's official testing suite