Uniformize monorepo structure to OrdinaryDiffEq canonical style#611
Merged
ChrisRackauckas merged 10 commits intoJun 8, 2026
Merged
Conversation
Align RecursiveArrayTools.jl with the OrdinaryDiffEq.jl canonical monorepo layout: - Add [sources] to each sublibrary pointing at the umbrella root via ../.. (true leaf->root dependency direction; the 3 sublibs depend on the registered root today). No cyclic root->leaf [sources] are added because the root does not depend on its sublibs. - Remove the redundant double-dispatch of the sublibraries: the root Tests.yml no longer GROUP-dispatches RaggedArrays/ArrayPartitionAnyAll/ ShorthandConstructors, and the root test/runtests.jl no longer activates and Pkg.tests them. Sublibraries are now covered exclusively by SublibraryCI's per-project model. - Rewrite root test/runtests.jl as a _detect_sublibrary_group dispatcher mirroring OrdinaryDiffEq: a GROUP naming lib/<X> activates that sublib, develops its [sources] path deps on Julia < 1.11, sets the sub group env, and Pkg.tests it; otherwise the root runs its own @safetestset groups. - SublibraryCI.yml: thread group-env-name RECURSIVEARRAYTOOLS_TEST_GROUP and check-bounds auto. - DowngradeSublibraries.yml: add group-env-name/group-env-value Core, expand the skip list to the siblings (root + 3 sublibs) plus stdlibs; auto-discovers lib/* (sublibrary-downgrade.yml@v1 has no allow-reresolve input). - Each sublibrary test/runtests.jl now reads get(ENV, "RECURSIVEARRAYTOOLS_TEST_GROUP", "Core") and gates its tests on the Core group. - Add test/test_groups.toml to each sublibrary declaring only [Core] on [lts, 1.11, 1, pre], since none of them have QA-group tests (the default expansion would otherwise emit an unsupported QA group on [1]). Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com> Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Match the SciML/OrdinaryDiffEq.jl convention: the catch-all test group
is canonical Title-case All, not all-caps ALL. The sublib runtests.jl
gates compared TEST_GROUP against "ALL" while the default and the
test_groups.toml key are already canonical ("Core"). Align the gate
string casing so all group-name strings are consistent.
Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- TagBot.yml: adopt canonical OrdinaryDiffEq structure — top-level permissions block, workflow_dispatch.inputs.lookback default "3", named root TagBot job, and a TagBot-Subpackages matrix (fail-fast: false) over the three registered sublibraries, each invoking TagBot@v1 with subdir: lib/<package>. - .gitignore: complete to canonical set (docs/build, LocalPreferences.toml, *.jl.*.mem, *.DS_Store, profile.pb.gz, .*.swp, .claude/, etc.). - Casing: rename the special test group "nopre" -> "NoPre" on both sides of the workflow<->runtests contract (Tests.yml matrix.group + exclude, and runtests.jl GROUP comparison) so it is canonical Title-case and case-consistent. Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com> Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ors test dep
- Sublibrary READMEs: prepend the canonical OrdinaryDiffEq-style badge
block (Zulip, Global Docs -> RecursiveArrayTools, ColPrac, SciML Code
Style) and the "<Name>.jl is a component of the RecursiveArrayTools.jl
monorepo. <role>. While completely independent and usable on its own,
users wanting the full functionality should use RecursiveArrayTools.jl."
wording, substituting names. Existing content preserved below.
- Project.toml: declare RecursiveArrayToolsShorthandConstructors as a
proper test dependency — added to [extras], [targets].test, [compat]
("1"), and a [sources] path entry so the Core test group resolves it
from the in-repo lib/ on Julia >= 1.11 (runtests.jl Pkg.develop
fallback still covers Julia < 1.11 where [sources] is ignored).
Verified: project resolves and `VA[...]`/`AP[...]` from the sublibrary
load in a Core-test-like environment.
Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Move the inline Aqua quality-assurance test out of the main test environment and into a dedicated test/qa/ project, matching the canonical OrdinaryDiffEq sublib layout (lib/*/test/qa/). - Add test/qa/Project.toml: Aqua + RecursiveArrayTools (via [sources]) + Pkg + Test, with [compat] for each dep and julia. - Move test/qa.jl -> test/qa/qa.jl (Aqua testset + the existing downstream-package manifest sentinel; behavior unchanged). - runtests.jl: add activate_qa_env() and, in the Core/All branch, activate the qa env, run the QA safetestset, then restore the main test project so the remaining Core functional tests keep their full dependency set. - Remove Aqua from the main Project.toml [extras], [targets].test, and [compat]; it now lives only in the qa env. JET (NoPre group) is already isolated in test/nopre/ and is left as-is. The three lib/* sublibraries run no QA today and are left untouched. Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com> Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
… downstream-only NLsolve The branch had added RecursiveArrayToolsShorthandConstructors to the umbrella root [sources]. It is a test-only dependency (the VA[]/AP[] shorthand tests), not a dependency of the RecursiveArrayTools package itself, so sourcing it at the package level makes Pkg.test add it to the Core test env as a declared package dependency. Aqua.test_stale_deps then flags it as a stale dep of RecursiveArrayTools (the module never imports it), failing the QA group. master never had this [sources] entry: its runtests.jl Pkg.develops the sublibrary into the test env directly, which the current runtests.jl still does. Removing the [sources] entry restores master's green QA behavior (verified: Core+QA Pkg.test now passes — Quality Assurance 9 pass / 1 broken (expected ambiguities @test_broken) / 0 fail; previously 1 fail). Also drop NLsolve from the main [extras]/[targets].test/[compat]: it is used only by test/downstream/odesolve.jl and is already declared in test/downstream/Project.toml, so it does not belong in the base test env. Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com> Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ject.tomls
Replace the root Tests.yml + bespoke GPU.yml with a single canonical CI.yml
(matching the OrdinaryDiffEq umbrella layout):
- CI.yml carries the version matrix for the root base groups
(Core/SymbolicIndexingInterface/Downstream/NoPre on [lts, 1, pre], NoPre
excluded on pre) via SciML/.github tests.yml@v1.
- The GPU tests become a dep-adding group on the self-hosted GPU runner: a
dedicated CI.yml job passes group=GPU and
runner=[self-hosted, Linux, X64, gpu] to tests.yml@v1 (which sets
GROUP=GPU; runtests.jl then activates test/gpu and runs the GPU
safetestsets). This reproduces the former GPU.yml (same runner labels,
60-min timeout, src,ext coverage) so GPU.yml is deleted.
- Give the dep-adding root group Project.tomls the canonical
[sources] = {path=../..} (+ sibling sublib paths) plus the package and Test
deps so they resolve on Julia >= 1.11 against the PR-branch code:
- test/gpu: add Adapt/ArrayInterface/RecursiveArrayTools/Test/Zygote +
[sources] for the root and RecursiveArrayToolsArrayPartitionAnyAll.
- test/nopre: add RecursiveArrayTools + [sources].
- test/downstream: add Test + [sources] for root and
RecursiveArrayToolsShorthandConstructors.
Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…olders) Make the root test layout match the OrdinaryDiffEq canonical pattern: every test lives in exactly one group, each group in its own capitalized test/<Group>/ folder, dep-adding groups have their own Project.toml (excluded from the light main env), and the version matrix is consistent between CI and test_groups.toml. - Split QA (Aqua) out of the Core branch into a standalone QA group keyed on GROUP == "QA" || GROUP == "All", running in its dep-adding test/QA env. Add a dedicated QA matrix row to CI.yml restricted to [lts, 1] (excluded on pre), so QA no longer runs inside the functional Core job. - Normalize sublib test_groups.toml [Core] versions ["lts","1.11","1","pre"] -> ["lts","1","pre"] for all three sublibraries. - Move the loose root Core test files into test/Core/, including the previously orphaned partitions_and_static_arrays.jl which was never wired into runtests and is now part of the Core group. Capitalize the remaining group folders (downstream->Downstream, gpu->GPU, nopre->NoPre, qa->QA) and route the SymbolicIndexingInterface and NoPre tests into their own group folders so all group folder names match their CI GROUP names. Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com> Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
QA is its own dep-adding test group (test/QA/Project.toml) and is run as a separate CI matrix entry, so it should not also fire under the "All" group. CI never runs "All", and a local "All" run should not pull the QA tooling into the functional test pass. Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com> Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Replace TagBot.yml with the canonical SciML/.github tagbot.yml@v1 thin caller, plus a tagbot-subpackages matrix over the registered monorepo sublibraries (RecursiveArrayToolsArrayPartitionAnyAll, RecursiveArrayToolsRaggedArrays, RecursiveArrayToolsShorthandConstructors). Drop hand-maintained skip lists and pinned julia-version floors from the downgrade callers; the centralized workflow now auto-populates skip (stdlibs union in-repo sublibs union caller package) and defaults to the LTS floor. Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com> Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
30d494c to
86dbcc1
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Uniformizes the RecursiveArrayTools.jl monorepo to the OrdinaryDiffEq.jl canonical structure.
Canonicalization done
[sources]path graph (leaf->root). Each sublibrary (RecursiveArrayToolsArrayPartitionAnyAll,RecursiveArrayToolsRaggedArrays,RecursiveArrayToolsShorthandConstructors) now declaresRecursiveArrayTools = {path = "../.."}in[sources], so CI tests them against the PR-branch root rather than the registered release. This preserves the true dependency direction (the sublibs depend on the registered root). The root does not add cyclicroot->leaf[sources]because it does not depend on its sublibs — Julia 1.11 rejects[sources]entries that are not in[deps]/[extras](verified locally).Tests.ymlno longer GROUP-dispatchesRaggedArrays/ArrayPartitionAnyAll/ShorthandConstructors, and the roottest/runtests.jlno longerPkg.activate/Pkg.tests them. Sublibraries are now covered exclusively bySublibraryCI's per-project model.test/runtests.jlis now a_detect_sublibrary_groupdispatcher mirroring OrdinaryDiffEq: aGROUPnaminglib/<X>activates that sublib, develops its[sources]path deps on Julia < 1.11, setsRECURSIVEARRAYTOOLS_TEST_GROUP, andPkg.tests it; otherwise the root runs its own@safetestsetgroups (Core / SymbolicIndexingInterface / Downstream / GPU / nopre).SublibraryCI.ymlnow passesgroup-env-name: RECURSIVEARRAYTOOLS_TEST_GROUPandcheck-bounds: autotosublibrary-project-tests.yml@v1.DowngradeSublibraries.ymlnow passesgroup-env-name/group-env-value: Core, expands theskiplist to the siblings (root + 3 sublibs) plus stdlibs, and continues to auto-discoverlib/*.sublibrary-downgrade.yml@v1has noallow-reresolveinput, so none is passed.test/runtests.jlnow readsget(ENV, "RECURSIVEARRAYTOOLS_TEST_GROUP", "Core")and gates its tests on theCoregroup — the env name is threaded consistently from every caller.test/test_groups.tomladded to each sublibrary declaring only[Core]on["lts", "1.11", "1", "pre"]. None of the sublibs have QA-group tests, so the default group expansion (which would also emitQAon[1]) is overridden.No
[compat]floors were lowered; no tests/tolerances were loosened, skipped, or broken. There are notest/Project.tomlfiles (the root'stest/{downstream,gpu,nopre}/Project.tomlare nested Pkg-environment subprojects, matching OrdinaryDiffEq'stest/qa/test/gpuenv subprojects — not test-target Project.tomls). No@testitem/ReTestItems were present in this repo, so item D is a no-op.GPU.ymlis left as a root self-hosted job (the GPU tests are a root-level concern using the ArrayPartitionAnyAll env, not a sublibrary group).Verified locally (isolated depot, Julia 1.11.9)
Project.toml; every[targets].testdep resolves from[extras]or[weakdeps](ReverseDiff/Tablescome from[weakdeps], as on master). YAML-parsed all changed workflows.RecursiveArrayToolsfrom the local../..path (is_tracking_path=true) via[sources].RecursiveArrayToolsArrayPartitionAnyAllCore group via the new@testset/group dispatch:Optimized any 5/5,Optimized all 4/4,Matches AbstractArray default results 4/4— all pass.RecursiveArrayToolsShorthandConstructorsCore group:VA[...] shorthand 6/6,AP[...] shorthand 6/6— all pass.Pkg.test()withGROUP=RecursiveArrayToolsArrayPartitionAnyAll:_detect_sublibrary_groupactivated the sublib, set the group env, andPkg.tested it — "RecursiveArrayTools tests passed" (exit 0). This exercises the dispatcher,[sources]resolution, and env threading together._detect_sublibrary_groupmapping (X->(X, Core),X_QA->(X, QA), root group names fall through) and the group gate (default -> Core runs,QA-> skips).@safetestsetfiles (named_array_partition_tests.jlpassed;utils_test.jl7/8 — the single error was a missing dep in the hand-built minimal env, present in the real[targets].test)..jl;--checkis clean.What CI must confirm
Core/Downstream/nopregroups on the[1, lts, pre]matrix (heavy deps: Mooncake, Aqua, ModelingToolkit, OrdinaryDiffEq, JET — not run locally).SublibraryCIdiscovery + theRaggedArraysCore group (large suite, not fully run locally).DowngradeSublibrariesresolution on lts with the new skip list and group env.[sources]develop branch — only the >= 1.11 native path was exercised locally).