Skip to content

Speed up targeted opencode tests#27812

Open
kitlangton wants to merge 19 commits into
anomalyco:devfrom
kitlangton:autoresearch/test-suite-speed
Open

Speed up targeted opencode tests#27812
kitlangton wants to merge 19 commits into
anomalyco:devfrom
kitlangton:autoresearch/test-suite-speed

Conversation

@kitlangton
Copy link
Copy Markdown
Contributor

@kitlangton kitlangton commented May 16, 2026

Summary

  • Add lightweight benchmark/profile scripts for packages/opencode test-suite research.
  • Reduce avoidable test setup across slow plugin, provider, workspace, SDK, config, HTTP listener, prompt, and processor-effect tests.
  • Preserve coverage where review showed setup mattered: prompt shell queue/cancel tests still use git-backed fixtures, and SDK read-route parity still exercises git-backed VCS output.
  • Record kept/discarded measurements and the safety-review correction in perf/test-suite.md.

Measurements

  • Initial full-suite benchmark: 250.922s.
  • Current reviewed full-suite benchmark: 202.317s via bun run bench:test.
  • Total measured savings: 48.605s, about 19.4% faster.
  • An intermediate unsafe benchmark reached 186.729s, but review found that removed coverage from several prompt shell tests and SDK VCS parity, so those parts were restored.

Focused wins include:

  • test/session/prompt.test.ts: 26.930s -> 19.610s targeted after restoring shell queue/cancel coverage.
  • test/session/processor-effect.test.ts: 12.500s -> 9.230s targeted.
  • test/server/httpapi-listen.test.ts: 10.554s -> 7.818s, then 7.051s -> 6.170s targeted.
  • test/server/httpapi-provider.test.ts: 7.905s -> 2.980s targeted.
  • test/cli/tui/plugin-lifecycle.test.ts: 7.330s -> 1.507s targeted.
  • test/server/httpapi-sdk.test.ts: 8.011s -> 5.180s targeted, with VCS parity restored for read routes.
  • test/control-plane/workspace.test.ts: 12.949s -> 8.305s targeted.
  • test/tool/skill.test.ts: 2.320s -> 1.425s targeted.

Safety Review

  • Parallel review found no public API compatibility issues; new timeout parameters are optional and preserve production defaults.
  • Prompt shell queue/cancel tests were failing without their original git-backed fixture and deterministic busy wait; restored both.
  • SDK instance read-route parity no longer lost VCS coverage; it now opts into a git fixture and asserts branch-shaped VCS data.
  • Benchmark script now rejects invalid BENCH_RUNS=0 / negative warmup inputs instead of crashing.

Verification

  • bun run test -- test/session/prompt.test.ts: 53 pass, 0 fail.
  • bun run test -- test/server/httpapi-sdk.test.ts: 16 pass, 0 fail.
  • bun run test -- test/server/httpapi-listen.test.ts test/server/httpapi-provider.test.ts test/server/httpapi-sdk.test.ts: 24 pass, 0 fail.
  • bun run bench:test: 2609 tests, 0 fail, 202.317s.
  • bun typecheck from packages/opencode.
  • bunx prettier --check ... for changed files.
  • File-scoped bunx oxlint ...: 0 errors; existing warnings remain in legacy tests.
  • Pre-push bun turbo typecheck passed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant