Add experimental-API enforcement tooling for TS, Python, Go, and Rust by stephentoub · Pull Request #1719 · github/copilot-sdk

stephentoub · 2026-06-18T13:47:56Z

Why

The C# and Java SDKs mark unstable APIs with [Experimental] / @CopilotExperimental, and the toolchain produces a real diagnostic that a consumer must explicitly suppress to opt in. The TypeScript, Python, Go, and Rust SDKs only had doc comments, so nothing actually stopped a user from depending on an experimental API by accident. This brings all four languages up to parity with a mechanism that the compiler or linter legitimately flags.

Approach

Each language gets the idiomatic equivalent of [Experimental], and codegen emits the marker/gate onto generated experimental methods so the surface stays in sync with the schema:

TypeScript - new standalone @github/eslint-plugin-copilot-sdk package with a type-aware no-experimental-api rule that flags references to any symbol whose JSDoc carries @experimental. Opt in per call site with // eslint-disable-next-line @github/copilot-sdk/no-experimental-api.
Python - an @experimental decorator that raises ExperimentalWarning under a configurable warn / error / ignore policy (also driven by the COPILOT_EXPERIMENTAL env var), plus allow_experimental() and set_experimental_policy(). Codegen applies it to generated experimental methods.
Go - a go/analysis analyzer (go/copilotexperimental/, nested module) that flags references to Experimental:-documented symbols via exported analysis Facts. Suppress with a same-line //nolint:copilotexperimental.
Rust - a new experimental Cargo feature. Codegen gates experimental RPC methods by conditional visibility: pub when the feature is enabled, and a pub(crate) fallback (identical body) when it is not. External consumers hit a hard E0624 compile error unless they opt in with features = ["experimental"], while the SDK's own internal callers keep compiling.

Notes for reviewers

The two large generated diffs (rust/src/generated/rpc.rs, python/copilot/generated/rpc.py) are produced by the codegen changes in scripts/codegen/*.ts, not hand-edited.
Rust CI (.github/workflows/rust-sdk-tests.yml) now passes test-support,experimental to the clippy and test steps. The SDK's own tests and the manual_tool_resume example exercise experimental RPCs directly, so they need the feature enabled, just as the C#/Java test projects suppress the experimental diagnostic. cargo doc --all-features already covers it, and the manual_tool_resume example is declared with required-features = ["experimental"].
Verified: Rust lib compiles with the feature both on and off, the gate produces the intended E0624 for an external consumer, and the regenerated file passes rustfmt. Go, Python, and the TS plugin test suites all pass.

@experimental

Bring TypeScript, Python, Go, and Rust in line with the C# ([Experimental]) and Java (@CopilotExperimental) SDKs, where consuming an experimental API produces a real diagnostic the user must explicitly suppress. Previously these four languages only had doc comments. - TypeScript: new @github/eslint-plugin-copilot-sdk package with a type-aware no-experimental-api rule keyed off the @experimental JSDoc tag. - Python: @experimental decorator emitting ExperimentalWarning with a configurable warn/error/ignore policy (COPILOT_EXPERIMENTAL); emitted by codegen onto generated experimental methods. - Go: go/analysis analyzer (go/copilotexperimental) that flags references to Experimental:-documented symbols; suppress with //nolint:copilotexperimental. - Rust: experimental Cargo feature; codegen gates experimental RPC methods via conditional visibility (pub with the feature, pub(crate) fallback without) so external callers get a hard compile error while internal SDK code keeps working. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot

Pull request overview

This PR adds enforceable “experimental API” opt-in mechanisms for the TypeScript, Python, Go, and Rust SDKs (bringing them closer to the C# / Java [Experimental] / @CopilotExperimental behavior) by introducing language-idiomatic enforcement and updating codegen so experimental markers stay in sync with the schema.

Changes:

Rust: Gate experimental generated RPC methods behind a new experimental Cargo feature via conditional visibility, and enable that feature in Rust CI for lint/tests.
Python: Add a runtime @experimental decorator + policy controls and have codegen decorate experimental generated RPC methods; add unit tests and public re-exports.
TypeScript & Go: Introduce a new ESLint plugin rule for TS consumers and a go/analysis vettool analyzer for Go consumers; document usage.

Show a summary per file

File	Description
scripts/codegen/rust.ts	Emits feature-gated conditional visibility for experimental generated Rust RPC methods.
scripts/codegen/python.ts	Imports and applies `@experimental` to generated Python RPC methods when marked experimental.
rust/README.md	Documents the `experimental` feature and how to opt in to experimental Rust APIs.
rust/Cargo.toml	Adds `experimental` feature and marks example(s) requiring it.
python/test_experimental.py	Adds unit tests validating runtime experimental gating behavior.
python/copilot/generated/rpc.py	Regenerated RPC client with `@experimental` decorators on experimental methods.
python/copilot/experimental.py	Implements Python runtime experimental decorator, warning type, and policy controls.
python/copilot/init.py	Re-exports experimental gating utilities in the public `copilot` package API.
nodejs/README.md	Documents experimental API enforcement via the companion ESLint plugin rule.
nodejs/eslint-plugin/test/no-experimental-api.test.js	Adds tests for the ESLint rule behavior against fixture consumers.
nodejs/eslint-plugin/test/fixtures/tsconfig.json	TS project config used by the rule’s type-aware tests.
nodejs/eslint-plugin/test/fixtures/sdk.ts	Fixture SDK surface with `@experimental` JSDoc tags.
nodejs/eslint-plugin/test/fixtures/re-export.ts	Fixture re-export used by consumer aliasing test scenarios.
nodejs/eslint-plugin/test/fixtures/consumer-suppressed.ts	Fixture demonstrating per-use suppression.
nodejs/eslint-plugin/test/fixtures/consumer-stable.ts	Fixture verifying stable API usage is not flagged.
nodejs/eslint-plugin/test/fixtures/consumer-experimental.ts	Fixture verifying experimental usages are flagged.
nodejs/eslint-plugin/test/fixtures/consumer-aliased.ts	Fixture verifying aliased experimental references are flagged.
nodejs/eslint-plugin/rules/no-experimental-api.js	Implements the type-aware ESLint rule that flags `@experimental` symbol references.
nodejs/eslint-plugin/README.md	Documents installation/configuration and suppression for the ESLint plugin.
nodejs/eslint-plugin/package.json	Defines the new standalone ESLint plugin package metadata/deps.
nodejs/eslint-plugin/package-lock.json	Locks dependencies for the standalone ESLint plugin package.
nodejs/eslint-plugin/index.js	Exposes plugin entry point, rule registration, and recommended configs.
nodejs/eslint-plugin/eslint.config.js	Flat-config used for plugin’s own tests/fixtures.
nodejs/eslint-plugin/.gitignore	Ignores local node_modules for the plugin package.
go/README.md	Documents running the new `copilotexperimental` analyzer via `go vet -vettool`.
go/copilotexperimental/testdata/src/sdk/sdk.go	Analyzer testdata: “SDK” declarations marked experimental via doc markers.
go/copilotexperimental/testdata/src/consumer/consumer.go	Analyzer testdata: consumer references + suppression directive.
go/copilotexperimental/README.md	Documents analyzer install/run and suppression behavior.
go/copilotexperimental/go.sum	Dependency checksums for the analyzer’s nested module.
go/copilotexperimental/go.mod	Nested module definition for the analyzer.
go/copilotexperimental/experimental.go	Implements the analyzer using exported analysis Facts + suppression scanning.
go/copilotexperimental/experimental_test.go	Runs analysistest suite for the analyzer.
go/copilotexperimental/cmd/copilotexperimental/main.go	Provides a `go vet`-compatible singlechecker entry point.
.github/workflows/rust-sdk-tests.yml	Enables `experimental` feature during Rust clippy and test runs.

Copilot's findings

Files not reviewed (1)

nodejs/eslint-plugin/package-lock.json: Generated file

Files reviewed: 31/35 changed files
Comments generated: 2

The standalone @github/eslint-plugin-copilot-sdk package ships its own node:test suite (run via its package's npm test). The SDK root 'vitest run' was globbing eslint-plugin/test/no-experimental-api.test.js and failing to collect it (No test suite found), breaking Node.js SDK Tests on all platforms. Exclude eslint-plugin/** from the root vitest config so the plugin's tests run only under their own runner. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

…anchor - go/test.sh now explicitly runs the copilotexperimental nested module's tests, which the repo-root `go test ./...` does not descend into, so analyzer regressions are caught in CI. - nodejs/eslint-plugin/README.md adds a Rules section with a `no-experimental-api` heading so the rule's generated docs URL fragment (#no-experimental-api) resolves to a real anchor. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

github-actions · 2026-06-18T15:56:53Z

Cross-SDK Consistency Review

This PR does exactly what it says: brings TypeScript, Python, Go, and Rust up to parity with the existing .NET and Java experimental-API enforcement. The overall design is consistent — each language gets an idiomatic enforcement mechanism, and codegen is updated to emit the markers in the two languages (Python, Rust) that were missing them.

✅ What's consistent across all 6 SDKs

SDK	Enforcement mechanism	Opt-in suppression	Codegen integration
.NET	`[Experimental]` compiler attribute (error/warning)	`#pragma warning disable`	✅ pre-existing
Java	`@CopilotExperimental` annotation processor (compile-time)	`@AllowCopilotExperimental`	✅ pre-existing
TypeScript	ESLint `no-experimental-api` rule (lint-time)	`// eslint-disable-next-line ...`	✅ pre-existing (`@experimental` JSDoc)
Go	`go/analysis` analyzer (analysis-time)	`//nolint:copilotexperimental`	✅ pre-existing (`Experimental:` in doc comments)
Python	`@experimental` decorator (runtime `ExperimentalWarning`)	`with allow_experimental():`	✅ added by this PR
Rust	`experimental` Cargo feature gate (compile error)	`features = ["experimental"]`	✅ added by this PR

All per-call-site suppression directives are inline annotations/comments — consistent across TypeScript and Go. The Java and .NET declaration-level opt-ins are analogous and idiomatic for those languages.

⚠️ Minor gaps worth noting

1. Python's since parameter has no Java counterpart

Python's @experimental(since="1.2") introduces a version parameter not present in Java's @CopilotExperimental (which takes no fields). If version tracking becomes part of generated metadata in the future, Java would need a matching since field. Today the codegen emits bare @experimental without a since argument, so this is only visible in manually-decorated non-generated code, and is low urgency — but worth tracking.

2. COPILOT_EXPERIMENTAL env var is Python-only

Python consumers can set COPILOT_EXPERIMENTAL=error in CI to make any accidental experimental-API call raise an exception. TypeScript/Go consumers need to configure their respective linters explicitly. This is an inherent difference between a runtime language and compiled/lint-time enforcement — not a bug, but worth calling out in cross-SDK docs so consumers know the CI story for each language.

3. is_experimental() runtime check is Python-only

Python uniquely exposes is_experimental(obj) -> bool for runtime introspection. This is appropriate since other SDKs handle detection at build/lint time, but documenting this Python-specific capability in the cross-SDK README would help consumers understand the difference.

💡 Suggestion for the cross-SDK README

The top-level README.md could add a brief table (analogous to what already exists for other features) summarising the enforcement mechanism and opt-in syntax per language, so SDK consumers can find the right suppression pattern without having to read six separate READMEs.

Overall this PR is a solid, well-thought-out addition that genuinely improves cross-SDK consistency. The observations above are minor and non-blocking.

Generated by SDK Consistency Review Agent for issue #1719 · sonnet46 2.5M · ◷

stephentoub requested a review from a team as a code owner June 18, 2026 13:47

Copilot AI review requested due to automatic review settings June 18, 2026 13:47

Copilot started reviewing on behalf of stephentoub June 18, 2026 13:48 View session

github-advanced-security AI found potential problems Jun 18, 2026

View reviewed changes

Comment thread python/copilot/experimental.py

Comment thread python/copilot/experimental.py

Comment thread python/test_experimental.py

Copilot AI reviewed Jun 18, 2026

View reviewed changes

Comment thread go/copilotexperimental/go.mod

Comment thread nodejs/eslint-plugin/rules/no-experimental-api.js

This comment has been minimized.

Sign in to view

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add experimental-API enforcement tooling for TS, Python, Go, and Rust#1719

Add experimental-API enforcement tooling for TS, Python, Go, and Rust#1719
stephentoub wants to merge 3 commits into
mainfrom
stephentoub/super-bassoon

stephentoub commented Jun 18, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

This comment has been minimized.

This comment has been minimized.

github-actions Bot commented Jun 18, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

stephentoub commented Jun 18, 2026

Why

Approach

Notes for reviewers

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Copilot's findings

Uh oh!

Uh oh!

Uh oh!

This comment has been minimized.

This comment has been minimized.

github-actions Bot commented Jun 18, 2026

Cross-SDK Consistency Review

✅ What's consistent across all 6 SDKs

⚠️ Minor gaps worth noting

💡 Suggestion for the cross-SDK README

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants