feat(grouped-tests): root test_groups.toml → CI matrix (closes #76); fix #75 self-test regression#78
Merged
ChrisRackauckas merged 1 commit intoJun 8, 2026
Conversation
…self-test regression Implements SciML#76: let a monorepo root (or any single package) declare its group x version matrix once in a root test/test_groups.toml and run it via a new reusable grouped-tests.yml, so CI.yml is a thin caller instead of a hand-maintained matrix. - scripts/compute_affected_sublibraries.jl: add a --root-matrix mode that reads <repo>/test/test_groups.toml (no diff filter, no lib/ required) and emits {group, version, runner, timeout, num_threads, continue_on_error}. Adds the continue_on_error per-group field (maps to tests.yml continue-on-error; used for non-fatal groups like OrdinaryDiffEq's Downstream). Default when absent: a single Core group on [lts,1,pre]. - .github/workflows/grouped-tests.yml: new reusable workflow; detect job runs --root-matrix, test job fans out to tests.yml@v1 (project '.') per cell. Mirrors sublibrary-project-tests.yml's proven passthrough. - Additive / backward compatible: existing callers and the sublibrary JSON output are unchanged. - Fix SciML#75 regression: the default-groups test still expected the old [lts,1.11,1,pre]/[1] matrix, so script-tests went red on master after SciML#75; update the expectation and two stale [lts,1.11,1,pre] comments. - Tests: build_root_matrix defaults/fields/continue_on_error, a faithful 46-cell reproduction of OrdinaryDiffEq's embedded root matrix, and a --root-matrix CLI test. README documents the model, grouped-tests.yml, the continue_on_error field, and the new output mode. Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com> Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implements #76 — let a monorepo root (or any single package) declare its
group × versionmatrix once in a roottest/test_groups.toml, run via a new reusablegrouped-tests.yml, soCI.ymlbecomes a thin caller instead of a hand-maintained matrix.What's here
scripts/compute_affected_sublibraries.jl— new--root-matrixmode: reads<repo>/test/test_groups.toml(no diff filter, nolib/required → works for non-monorepos), emits{group, version, runner, timeout, num_threads, continue_on_error}. Adds thecontinue_on_errorper-group field (maps totests.yml'scontinue-on-error; for non-fatal groups like ODE'sDownstream). Default when absent: a singleCoregroup on[lts,1,pre]..github/workflows/grouped-tests.yml— new reusable workflow:detectruns--root-matrix,testfans out totests.yml@v1(project: '.') per cell, dispatching the group viagroup-env-name. Mirrorssublibrary-project-tests.yml's proven passthrough.[lts,1.11,1,pre]/[1]matrix, soscript-testswent red onmasterafter Default version matrix -> Core [lts,1,pre], QA [lts,1] #75 merged. Updated the expectation + two stale comments. (v1currently ships this red test — this greens it.)build_root_matrixdefaults/fields/continue_on_error, a faithful 46-cell reproduction of OrdinaryDiffEq's embedded root matrix, and a--root-matrixCLI test. Verified locally: all 50 pass;actionlintclean on the new workflow.grouped-tests.yml, thecontinue_on_errorfield, and the new output mode.After merge +
v1retagMigrate each monorepo's root
CI.ymlto the thin caller + author its roottest/test_groups.toml(ODE first as reference; the 46-cell test shows the translation). Non-monorepos can opt in for a near-zero-lineCI.yml.Ignore until reviewed by @ChrisRackauckas.