Repository navigation
Conversation
Mrtenz
force-pushed
the
mrtenz/test-sharding
branch
from
November 2, 2026 13:43
676a083 to
d789f46
Compare
Mrtenz
force-pushed
the
mrtenz/test-sharding
branch
from
November 2, 2026 13:47
d789f46 to
6e186f0
Compare
Mrtenz
force-pushed
the
mrtenz/test-sharding
branch
2 times, most recently
from
November 2, 2026 20:20
47b8e37 to
393627a
Compare
4 tasks
Mrtenz
force-pushed
the
mrtenz/test-sharding
branch
from
November 3, 2026 09:55
393627a to
33ac410
Compare
Mrtenz
force-pushed
the
mrtenz/test-sharding
branch
2 times, most recently
from
November 3, 2026 19:55
5aa7016 to
e9fd789
Compare
Run the full suite through `vitest --shard` across 10 shards and every supported Node version, replacing the per-package `test-22`, `test-24`, and `test-26` jobs. The shard count and Node versions come from a renamed `prepare-environment` job, so the count is defined in one place. Each shard uploads its blob report. A `merge-test-reports` job for each Node version combines them with `vitest --merge-reports` into a single result. This also adds `.vitest/` to `.gitignore`.
Pass `--silent` to `vitest --merge-reports` so the `merge-test-reports` job does not replay console output from the tests. The shard jobs already print it.
The package isn't tested in CI and its tests currently fail. The `test` job now runs `vitest --shard` from the root, so it would pick up the package and fail the run. Leave it out of `projects`.
Set `VITEST_BLOB_LABEL` to the Node version in the `test` job. The label goes into each blob's filename and tags its test files, so reports from different Node versions no longer overwrite each other when merged. A single `merge-test-reports` job can now combine every blob, and the merged output shows which Node version a failure came from.
Add the `github-actions` reporter to the `vitest --merge-reports` step. Vitest enables it automatically only when no `--reporter` is passed, so our explicit `--reporter default` turned it off. The reporter writes a test report to `GITHUB_STEP_SUMMARY` and adds inline annotations for failures.
Each job spends about 25 seconds on setup, and the whole suite takes about 9 minutes of test time, measured on a sharded run. Doubling the shard count cuts the slowest shard from roughly 80 to 55 seconds. That is still about six times fewer jobs than the old one-job-per-package layout.
When a PR doesn't change any root files, the `test` job now writes a temporary Vitest config that lists only the changed packages and their dependants, and runs the shards against that. Full runs still use the root config. The `test` job is skipped when no packages changed, and `merge-test-reports` is skipped with it. `--passWithNoTests` stops shards that end up with no test files from failing.
Running 20 shards for a PR that touches one package leaves most of them with no test files, and each still pays for checkout and setup. Full runs keep 20 shards. Other runs use `min(changed packages, 18) + 2` shards, so between 3 and 20. The shard step moves from `prepare-environment` to `prepare`, because it needs the changed paths. The workflow also passes `--silent` to the test step.
Coverage thresholds are configured per package, and a package's files are split over shards, so they can only be checked once the shard reports are merged. The shards now run with `--coverage`, and the merge job runs `--merge-reports --coverage`, which checks each package's thresholds against the merged data. `createRootConfig` in `scripts/lib/create-vitest-config.ts` builds the root config for this. It loads each package's own Vitest config, prefixes its coverage `include`, `exclude`, and `thresholds` with the package directory, and only applies thresholds when merging. The root config and the temporary config for changed-only runs both use it. Workspace paths come from Yarn through `getAllWorkspaces`, which gains an `includePrivate` option so `package-template` stays in the root projects.
Mrtenz
force-pushed
the
mrtenz/test-sharding
branch
from
November 3, 2026 20:01
e9fd789 to
cafccdb
Compare
Vitest loads `scripts/lib/create-vitest-config.ts` as part of a config, and a config doesn't get the `tsconfig` path mapping that tests get. The import of `@metamask/node-utils` therefore resolved to its build output, and failed with "Failed to resolve entry for package" when the package wasn't built. The helper now checks whether a config file exists with `access` from `node:fs/promises`, and the tests mock that instead.
With coverage enabled, the slowest of the 20 shards takes about 170s while the average takes about 95s, so the slowest shard sets how long the test phase lasts. More shards shrink the slowest one. Full runs now use 30 shards, and runs that touch only some packages use between 3 and 30.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Explanation
The
testworkflow ran one job per package for each of Node 22, 24, and 26 (test-22,test-24,test-26). The jobs were duplicated to stay under GitHub's 256-job matrix limit, and every package ran regardless of how long it took.This replaces them with a single
testjob that runsvitest --shardfor every supported Node version. Each shard uploads its Vitest blob report. For each Node version, amerge-test-reportsjob combines the reports withvitest --merge-reportsinto one result, including failure details.The shard count is defined once, in the
prepare-environmentjob (renamed fromget-node-versions), and is used for both the matrix and the--shardargument. This also adds.vitest/to.gitignore, where Vitest writes the blob reports.References
None.
Checklist