Open-source MCP protocol-level security test harness (10 tests) #2430
Replies: 1 comment
-
|
This is a useful direction because protocol-level testing helps make agent security concrete instead of philosophical. One thing I hope these harnesses keep emphasizing is that the real boundary is not whether the model can be influenced, but whether influenced output can cross into authority without a narrower enforcement layer in between. In practice, a lot of failures seem to come from collapsing interpretation and execution into the same trust zone. So tests for prompt or feature injection are great, but the most valuable ones may be the ones that reveal where capability scoping, default deny behavior, and tool mediation are still too soft. |
Beta Was this translation helpful? Give feedback.
Uh oh!
There was an error while loading. Please reload this page.
-
Built an open-source security test harness that tests MCP servers at the JSON-RPC 2.0 wire level — not HTTP application-layer testing.
Supports two transports:
python -m protocol_tests.mcp_harness --transport http --url http://localhost:8080/mcppython -m protocol_tests.mcp_harness --transport stdio --command "node my-server.js"10 tests across 7 categories:
tools/listintegrity — scans for exfiltration URLs, hidden instructions in tool descriptionstools/callinjection — tries to register tools by calling non-existent onesinitializecapability escalation — claims admin/bypass capabilitiesresources/readpath traversal —../../etc/passwd,/proc/self/environprompts/getargument injection — exfiltration URLs in prompt argumentssampling/createMessagecontext exfiltration — attempts to extract API keys and secretsZero external dependencies (Python 3.10+ stdlib only). JSON report generation. Statistical mode with Wilson score CIs (NIST AI 800-2 aligned).
Part of a larger framework (175 total tests) that also covers A2A protocol testing, GTG-1002 APT simulation, and 20 enterprise platform adapters.
https://github.com/msaleme/red-team-blue-team-agent-fabric
Feedback welcome — especially interested in:
Beta Was this translation helpful? Give feedback.
All reactions