MCP SDK Tier Audit: modelcontextprotocol/ruby-sdk
SDK: modelcontextprotocol/ruby-sdk
Date: 2026-03-03
Branch: main
Requested tier: Tier 3
Auditor: mcp-sdk-tier-audit skill (automated + subagent evaluation)
Tier Assessment: Tier 3
The Ruby SDK is at Tier 3 (Experimental). Server conformance passes the Tier 2 threshold at 83.3%, all 12 labels are present, triage is at 100%, and core documentation passes Tier 2. However, two requirements block advancement: no stable release (v0.8.0) and no published roadmap. All 5 server conformance failures are documented in conformance/expected_failures.yml.
Requirements Summary
| # |
Requirement |
Tier 1 Standard |
Tier 2 Standard |
Current Value |
T1? |
T2? |
Gap |
| 1a |
Server Conformance |
100% pass rate |
>= 80% pass rate |
83.3% (25/30) |
FAIL |
PASS |
5 failures: sampling, elicitation x3, progress |
| 1b |
Client Conformance |
100% pass rate |
>= 80% pass rate |
Skipped (no client-cmd) |
FAIL |
N/A |
Client conformance not tested |
| 2 |
Issue Triage |
>= 90% within 2 biz days |
>= 80% within 1 month |
100% (7/7) |
PASS |
PASS |
None |
| 2b |
Labels |
12 required labels |
12 required labels |
12/12 |
PASS |
PASS |
None |
| 3 |
Critical Bug Resolution |
All P0s within 7 days |
All P0s within 2 weeks |
0 open |
PASS |
PASS |
None |
| 4 |
Stable Release |
Required + clear versioning |
At least one stable release |
0.8.0 |
FAIL |
FAIL |
Pre-1.0 release |
| 4b |
Spec Tracking |
Timeline agreed per release |
Within 6 months |
46d gap |
FAIL |
PASS |
46 days since latest spec release |
| 5 |
Documentation |
Comprehensive w/ examples |
Basic docs for core features |
22/48 PASS, 1 PARTIAL, 25 FAIL |
FAIL |
PASS |
25 features undocumented; core features covered |
| 6 |
Dependency Policy |
Published update policy |
Published update policy |
dependabot.yml (GitHub Actions) |
PASS |
PASS |
Covers GH Actions only, not bundler gems |
| 7 |
Roadmap |
Published roadmap |
Plan toward Tier 1 |
Not found |
FAIL |
FAIL |
No ROADMAP.md or docs/roadmap.md |
| 8 |
Versioning Policy |
Documented breaking change policy |
N/A |
Not found |
FAIL |
N/A |
No VERSIONING.md or BREAKING_CHANGES.md |
Tier Determination
- Tier 1: FAIL — 4/10 requirements met (passing: triage, labels, P0 resolution, dependency policy; failing: server conformance, client conformance, stable release, spec tracking, documentation, roadmap, versioning policy)
- Tier 2: FAIL — 7/9 requirements met (passing: server conformance, triage, labels, P0 resolution, spec tracking, documentation, dependency policy; failing: stable release, roadmap)
- Final Tier: 3
Server Conformance Details
Pass rate: 83.3% (25/30)
By Spec Version
| Spec Version |
Passed |
Failed |
Total |
Rate |
| 2025-06-18 |
23 |
3 |
26 |
88.5% |
| 2025-11-25 |
25 |
5 |
30 |
83.3% |
Scenario Results
| Scenario |
Status |
Checks |
Spec Versions |
| server-initialize |
PASS |
1/1 |
2025-06-18, 2025-11-25 |
| ping |
PASS |
1/1 |
2025-06-18, 2025-11-25 |
| logging-set-level |
PASS |
1/1 |
2025-06-18, 2025-11-25 |
| completion-complete |
PASS |
1/1 |
2025-06-18, 2025-11-25 |
| tools-list |
PASS |
1/1 |
2025-06-18, 2025-11-25 |
| tools-call-simple-text |
PASS |
1/1 |
2025-06-18, 2025-11-25 |
| tools-call-image |
PASS |
1/1 |
2025-06-18, 2025-11-25 |
| tools-call-audio |
PASS |
1/1 |
2025-06-18, 2025-11-25 |
| tools-call-embedded-resource |
PASS |
1/1 |
2025-06-18, 2025-11-25 |
| tools-call-mixed-content |
PASS |
1/1 |
2025-06-18, 2025-11-25 |
| tools-call-with-logging |
PASS |
1/1 |
2025-06-18, 2025-11-25 |
| tools-call-error |
PASS |
1/1 |
2025-06-18, 2025-11-25 |
| tools-call-with-progress |
FAIL |
0/1 |
2025-06-18, 2025-11-25 |
| tools-call-sampling |
FAIL |
0/1 |
2025-06-18, 2025-11-25 |
| tools-call-elicitation |
FAIL |
0/1 |
2025-06-18, 2025-11-25 |
| elicitation-sep1034-defaults |
FAIL |
0/1 |
2025-11-25 |
| elicitation-sep1330-enums |
FAIL |
0/1 |
2025-11-25 |
| sse-multiple-streams |
PASS |
1/1 |
2025-11-25 |
| dns-rebinding-protection |
PASS |
2/2 |
2025-11-25 |
| resources-list |
PASS |
1/1 |
2025-06-18, 2025-11-25 |
| resources-read-text |
PASS |
1/1 |
2025-06-18, 2025-11-25 |
| resources-read-binary |
PASS |
1/1 |
2025-06-18, 2025-11-25 |
| resources-templates-read |
PASS |
1/1 |
2025-06-18, 2025-11-25 |
| resources-subscribe |
PASS |
1/1 |
2025-06-18, 2025-11-25 |
| resources-unsubscribe |
PASS |
1/1 |
2025-06-18, 2025-11-25 |
| prompts-list |
PASS |
1/1 |
2025-06-18, 2025-11-25 |
| prompts-get-simple |
PASS |
1/1 |
2025-06-18, 2025-11-25 |
| prompts-get-with-args |
PASS |
1/1 |
2025-06-18, 2025-11-25 |
| prompts-get-embedded-resource |
PASS |
1/1 |
2025-06-18, 2025-11-25 |
| prompts-get-with-image |
PASS |
1/1 |
2025-06-18, 2025-11-25 |
Failed Scenario Analysis
All 5 failures are documented in conformance/expected_failures.yml and concentrated in sampling, elicitation, and progress features:
- tools-call-with-progress: The SDK does not extract
_meta.progressToken from tool call requests or deliver notifications/progress to tools
- tools-call-sampling: The SDK lacks
Transport#send_request (server-to-client request), required for the sampling flow
- tools-call-elicitation, elicitation-sep1034-defaults, elicitation-sep1330-enums: Elicitation is listed as "Unsupported" in the SDK (README.md L235);
Transport#send_request does not exist
These are scope gaps (unimplemented features) rather than quality problems in existing functionality.
Baseline Comparison
| Expected Failure (expected_failures.yml) |
Observed |
Match? |
| tools-call-sampling |
FAIL |
Yes |
| tools-call-elicitation |
FAIL |
Yes |
| elicitation-sep1034-defaults |
FAIL |
Yes |
| elicitation-sep1330-enums |
FAIL |
Yes |
| tools-call-with-progress |
FAIL |
Yes |
All 5 expected failures are confirmed. No unexpected failures detected.
Client Conformance Details
Skipped — no client command was provided. The Ruby SDK has a basic HTTP client (MCP::Client::HTTP), but no conformance client has been implemented yet.
Issue Triage Details
Analysis period: Last 7 issues
Labels present: bug, enhancement, question, needs confirmation, needs repro, ready for work, good first issue, help wanted, P0, P1, P2, P3 (12/12)
| Metric |
Value |
T1 Req |
T2 Req |
Verdict |
| Compliance rate |
100% |
>= 90% |
>= 80% |
PASS |
| Triaged within SLA |
7 |
-- |
-- |
-- |
| Exceeding SLA |
0 |
-- |
-- |
-- |
| Median hours |
0h |
-- |
-- |
-- |
| Open P0s |
0 |
0 |
0 |
PASS |
Documentation Coverage
SDK path: /Users/koic/src/github.com/modelcontextprotocol/ruby-sdk
Documentation locations found:
- README.md: Main SDK documentation (~1040 lines) with prose and code examples
- examples/README.md: Example script descriptions
- examples/stdio_server.rb, http_server.rb, http_client.rb, streamable_http_server.rb, streamable_http_client.rb: Runnable examples
- docs/index.md: Jekyll redirect to RubyDoc.info — no substantive content
Feature Documentation Table
| # |
Feature |
Documented? |
Where |
Has Examples? |
Verdict |
| 1 |
Tools - listing |
Yes |
README.md L46-47,L930-996 |
Yes |
PASS |
| 2 |
Tools - calling |
Yes |
README.md L47,L930-1003 |
Yes |
PASS |
| 3 |
Tools - text results |
Yes |
README.md L469-540 |
Yes |
PASS |
| 4 |
Tools - image results |
No |
— |
No |
FAIL |
| 5 |
Tools - audio results |
No |
— |
No |
FAIL |
| 6 |
Tools - embedded resources |
No |
— |
No |
FAIL |
| 7 |
Tools - error handling |
Yes |
README.md L710-734 |
Yes |
PASS |
| 8 |
Tools - change notifications |
Yes |
README.md L105-126,L207-220 |
Yes |
PASS |
| 9 |
Resources - listing |
Yes |
README.md L51,L874-895 |
Yes |
PASS |
| 10 |
Resources - reading text |
Yes |
README.md L878-909 |
Yes |
PASS |
| 11 |
Resources - reading binary |
No |
— |
No |
FAIL |
| 12 |
Resources - templates |
Yes |
README.md L911-928 |
Yes |
PASS |
| 13 |
Resources - template reading |
No |
— |
No |
FAIL |
| 14 |
Resources - subscribing |
No |
Listed as unsupported (README.md L231) |
No |
FAIL |
| 15 |
Resources - unsubscribing |
No |
Listed as unsupported (README.md L231) |
No |
FAIL |
| 16 |
Resources - change notifications |
Yes |
README.md L105-126 |
Yes |
PASS |
| 17 |
Prompts - listing |
Yes |
README.md L736-870 |
Yes |
PASS |
| 18 |
Prompts - getting simple |
Yes |
README.md L744-776 |
Yes |
PASS |
| 19 |
Prompts - getting with arguments |
Yes |
README.md L784-811 |
Yes |
PASS |
| 20 |
Prompts - embedded resources |
No |
— |
No |
FAIL |
| 21 |
Prompts - image content |
No |
— |
No |
FAIL |
| 22 |
Prompts - change notifications |
Yes |
README.md L105-126 |
Yes |
PASS |
| 23 |
Sampling - creating messages |
No |
Not implemented |
No |
FAIL |
| 24 |
Elicitation - form mode |
No |
Not implemented (README.md L235) |
No |
FAIL |
| 25 |
Elicitation - URL mode |
No |
Not implemented |
No |
FAIL |
| 26 |
Elicitation - schema validation |
No |
Not implemented |
No |
FAIL |
| 27 |
Elicitation - default values |
No |
Not implemented |
No |
FAIL |
| 28 |
Elicitation - enum values |
No |
Not implemented |
No |
FAIL |
| 29 |
Elicitation - complete notification |
No |
Not implemented |
No |
FAIL |
| 30 |
Roots - listing |
No |
— |
No |
FAIL |
| 31 |
Roots - change notifications |
No |
— |
No |
FAIL |
| 32 |
Logging - sending log messages |
Yes |
README.md L127-206 |
Yes |
PASS |
| 33 |
Logging - setting level |
Yes |
README.md L163-170 |
Yes |
PASS |
| 34 |
Completions - resource argument |
No |
Not implemented (README.md L234) |
No |
FAIL |
| 35 |
Completions - prompt argument |
No |
Not implemented (README.md L234) |
No |
FAIL |
| 36 |
Ping |
Yes |
README.md L45,L306 |
Yes |
PASS |
| 37 |
Streamable HTTP transport (client) |
Yes |
README.md L971-1003 |
Yes |
PASS |
| 38 |
Streamable HTTP transport (server) |
Yes |
README.md L207-229,L238-262 |
Yes |
PASS |
| 39 |
SSE transport - legacy (client) |
No |
Not implemented |
No |
FAIL |
| 40 |
SSE transport - legacy (server) |
No |
Not implemented |
No |
FAIL |
| 41 |
stdio transport (client) |
No |
Not implemented |
No |
FAIL |
| 42 |
stdio transport (server) |
Yes |
README.md L264-309 |
Yes |
PASS |
| 43 |
Progress notifications |
No |
— |
No |
FAIL |
| 44 |
Cancellation |
No |
— |
No |
FAIL |
| 45 |
Pagination |
No |
— |
No |
FAIL |
| 46 |
Capability negotiation |
Partial |
README.md L35 (brief mention) |
No |
PARTIAL |
| 47 |
Protocol version negotiation |
Yes |
README.md L427-447 |
Yes |
PASS |
| 48 |
JSON Schema 2020-12 support |
No |
— |
No |
FAIL |
| — |
Tasks - get (experimental) |
No |
— |
No |
INFO |
| — |
Tasks - result (experimental) |
No |
— |
No |
INFO |
| — |
Tasks - cancel (experimental) |
No |
— |
No |
INFO |
| — |
Tasks - list (experimental) |
No |
— |
No |
INFO |
| — |
Tasks - status notifications (experimental) |
No |
— |
No |
INFO |
Summary
- Total non-experimental features: 48
- PASS (documented with examples): 22/48 (45.8%)
- PARTIAL (documented, no examples): 1/48 (2.1%)
- FAIL (not documented): 25/48 (52.1%)
All features documented with examples: 22/48 (45.8%)
Tier 1 (all non-experimental features documented with examples): FAIL — 25 features missing, 1 partial
Tier 2 (basic docs covering core features): PASS — tools, resources, prompts, logging, transports documented with examples
Note: Resources subscribing/unsubscribing are listed as "unsupported" in README.md L231, yet the conformance tests resources-subscribe and resources-unsubscribe both PASS. This documentation-vs-implementation disconnect should be resolved.
Policy Evaluation
1. Dependency Update Policy: PASS
| File |
Exists |
Content Verdict |
| DEPENDENCY_POLICY.md |
No |
N/A |
| docs/dependency-policy.md |
No |
N/A |
| .github/dependabot.yml |
Yes |
Configured (GitHub Actions ecosystem, weekly) |
| .github/renovate.json |
No |
N/A |
| renovate.json |
No |
N/A |
Verdict: PASS — Dependabot v2 configured for GitHub Actions. Note: does not cover Ruby gem (bundler) ecosystem. The gemspec declares a runtime dependency on json-schema (>= 4.1) that is not auto-updated.
2. Roadmap: FAIL
| File |
Exists |
Content Verdict |
| ROADMAP.md |
No |
N/A |
| docs/roadmap.md |
No |
N/A |
The only roadmap-adjacent content is the "Unsupported Features" subsection in README.md L231-235, listing "Resource subscriptions, Completions, Elicitation" as "to be implemented in future versions." This lacks timelines, milestones, or prioritization.
Verdict:
- Tier 1: FAIL — No published roadmap
- Tier 2: FAIL — No plan toward Tier 1
3. Versioning Policy: FAIL
| File |
Exists |
Content Verdict |
| VERSIONING.md |
No |
N/A |
| docs/versioning.md |
No |
N/A |
| BREAKING_CHANGES.md |
No |
N/A |
| CONTRIBUTING.md |
No |
N/A |
CHANGELOG.md declares adherence to Semantic Versioning and uses Keep a Changelog format. However, the project is pre-1.0 (v0.8.0), meaning under SemVer conventions, breaking changes can occur in any minor version bump without explicit policy.
Verdict:
- Tier 1: FAIL — No documented breaking change policy
- Tier 2: N/A
Overall Policy Summary
| Policy Area |
Tier 1 |
Tier 2 |
| Dependency Update Policy |
PASS |
PASS |
| Roadmap |
FAIL |
FAIL |
| Versioning Policy |
FAIL |
N/A |
MCP SDK Tier Audit: modelcontextprotocol/ruby-sdk
SDK: modelcontextprotocol/ruby-sdk
Date: 2026-03-03
Branch: main
Requested tier: Tier 3
Auditor: mcp-sdk-tier-audit skill (automated + subagent evaluation)
Tier Assessment: Tier 3
The Ruby SDK is at Tier 3 (Experimental). Server conformance passes the Tier 2 threshold at 83.3%, all 12 labels are present, triage is at 100%, and core documentation passes Tier 2. However, two requirements block advancement: no stable release (v0.8.0) and no published roadmap. All 5 server conformance failures are documented in
conformance/expected_failures.yml.Requirements Summary
Tier Determination
Server Conformance Details
Pass rate: 83.3% (25/30)
By Spec Version
Scenario Results
Failed Scenario Analysis
All 5 failures are documented in
conformance/expected_failures.ymland concentrated in sampling, elicitation, and progress features:_meta.progressTokenfrom tool call requests or delivernotifications/progressto toolsTransport#send_request(server-to-client request), required for the sampling flowTransport#send_requestdoes not existThese are scope gaps (unimplemented features) rather than quality problems in existing functionality.
Baseline Comparison
All 5 expected failures are confirmed. No unexpected failures detected.
Client Conformance Details
Skipped — no client command was provided. The Ruby SDK has a basic HTTP client (
MCP::Client::HTTP), but no conformance client has been implemented yet.Issue Triage Details
Analysis period: Last 7 issues
Labels present: bug, enhancement, question, needs confirmation, needs repro, ready for work, good first issue, help wanted, P0, P1, P2, P3 (12/12)
Documentation Coverage
SDK path: /Users/koic/src/github.com/modelcontextprotocol/ruby-sdk
Documentation locations found:
Feature Documentation Table
Summary
All features documented with examples: 22/48 (45.8%)
Tier 1 (all non-experimental features documented with examples): FAIL — 25 features missing, 1 partial
Tier 2 (basic docs covering core features): PASS — tools, resources, prompts, logging, transports documented with examples
Note: Resources subscribing/unsubscribing are listed as "unsupported" in README.md L231, yet the conformance tests
resources-subscribeandresources-unsubscribeboth PASS. This documentation-vs-implementation disconnect should be resolved.Policy Evaluation
1. Dependency Update Policy: PASS
Verdict: PASS — Dependabot v2 configured for GitHub Actions. Note: does not cover Ruby gem (bundler) ecosystem. The gemspec declares a runtime dependency on
json-schema(>= 4.1) that is not auto-updated.2. Roadmap: FAIL
The only roadmap-adjacent content is the "Unsupported Features" subsection in README.md L231-235, listing "Resource subscriptions, Completions, Elicitation" as "to be implemented in future versions." This lacks timelines, milestones, or prioritization.
Verdict:
3. Versioning Policy: FAIL
CHANGELOG.md declares adherence to Semantic Versioning and uses Keep a Changelog format. However, the project is pre-1.0 (v0.8.0), meaning under SemVer conventions, breaking changes can occur in any minor version bump without explicit policy.
Verdict:
Overall Policy Summary