Skip to content

feat(grouped-tests): root test_groups.toml → CI matrix (closes #76); fix #75 self-test regression#78

Merged
ChrisRackauckas merged 1 commit into
SciML:masterfrom
ChrisRackauckas-Claude:root-test-groups-matrix
Jun 8, 2026
Merged

feat(grouped-tests): root test_groups.toml → CI matrix (closes #76); fix #75 self-test regression#78
ChrisRackauckas merged 1 commit into
SciML:masterfrom
ChrisRackauckas-Claude:root-test-groups-matrix

Conversation

@ChrisRackauckas-Claude
Copy link
Copy Markdown

Implements #76 — let a monorepo root (or any single package) declare its group × version matrix once in a root test/test_groups.toml, run via a new reusable grouped-tests.yml, so CI.yml becomes a thin caller instead of a hand-maintained matrix.

What's here

  • scripts/compute_affected_sublibraries.jl — new --root-matrix mode: reads <repo>/test/test_groups.toml (no diff filter, no lib/ required → works for non-monorepos), emits {group, version, runner, timeout, num_threads, continue_on_error}. Adds the continue_on_error per-group field (maps to tests.yml's continue-on-error; for non-fatal groups like ODE's Downstream). Default when absent: a single Core group on [lts,1,pre].
  • .github/workflows/grouped-tests.yml — new reusable workflow: detect runs --root-matrix, test fans out to tests.yml@v1 (project: '.') per cell, dispatching the group via group-env-name. Mirrors sublibrary-project-tests.yml's proven passthrough.
  • Additive / backward compatible — existing callers and the sublibrary JSON output are unchanged.
  • Fixes the Default version matrix -> Core [lts,1,pre], QA [lts,1] #75 regression: the default-groups self-test still expected the old [lts,1.11,1,pre]/[1] matrix, so script-tests went red on master after Default version matrix -> Core [lts,1,pre], QA [lts,1] #75 merged. Updated the expectation + two stale comments. (v1 currently ships this red test — this greens it.)
  • Tests: build_root_matrix defaults/fields/continue_on_error, a faithful 46-cell reproduction of OrdinaryDiffEq's embedded root matrix, and a --root-matrix CLI test. Verified locally: all 50 pass; actionlint clean on the new workflow.
  • README documents the model, grouped-tests.yml, the continue_on_error field, and the new output mode.

After merge + v1 retag

Migrate each monorepo's root CI.yml to the thin caller + author its root test/test_groups.toml (ODE first as reference; the 46-cell test shows the translation). Non-monorepos can opt in for a near-zero-line CI.yml.

Ignore until reviewed by @ChrisRackauckas.

…self-test regression

Implements SciML#76: let a monorepo root (or any single package)
declare its group x version matrix once in a root test/test_groups.toml and
run it via a new reusable grouped-tests.yml, so CI.yml is a thin caller
instead of a hand-maintained matrix.

- scripts/compute_affected_sublibraries.jl: add a --root-matrix mode that reads
  <repo>/test/test_groups.toml (no diff filter, no lib/ required) and emits
  {group, version, runner, timeout, num_threads, continue_on_error}. Adds the
  continue_on_error per-group field (maps to tests.yml continue-on-error; used
  for non-fatal groups like OrdinaryDiffEq's Downstream). Default when absent:
  a single Core group on [lts,1,pre].
- .github/workflows/grouped-tests.yml: new reusable workflow; detect job runs
  --root-matrix, test job fans out to tests.yml@v1 (project '.') per cell.
  Mirrors sublibrary-project-tests.yml's proven passthrough.
- Additive / backward compatible: existing callers and the sublibrary JSON
  output are unchanged.
- Fix SciML#75 regression: the default-groups test still expected the old
  [lts,1.11,1,pre]/[1] matrix, so script-tests went red on master after SciML#75;
  update the expectation and two stale [lts,1.11,1,pre] comments.
- Tests: build_root_matrix defaults/fields/continue_on_error, a faithful
  46-cell reproduction of OrdinaryDiffEq's embedded root matrix, and a
  --root-matrix CLI test. README documents the model, grouped-tests.yml, the
  continue_on_error field, and the new output mode.

Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@ChrisRackauckas ChrisRackauckas marked this pull request as ready for review June 8, 2026 10:40
@ChrisRackauckas ChrisRackauckas merged commit 711d973 into SciML:master Jun 8, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants