Skip to content

Unify CI caches (reduce total cache size by 75% ±)#7610

Merged
youknowone merged 3 commits intoRustPython:mainfrom
ShaharNaveh:unify-ci-caches
Apr 15, 2026
Merged

Unify CI caches (reduce total cache size by 75% ±)#7610
youknowone merged 3 commits intoRustPython:mainfrom
ShaharNaveh:unify-ci-caches

Conversation

@ShaharNaveh
Copy link
Copy Markdown
Contributor

@ShaharNaveh ShaharNaveh commented Apr 14, 2026 •
edited by coderabbitai bot
Loading

ATM we have separate cache for a matrix of operating systems * job number (the actual number is less, but not by that much). And the cache of all those jobs is 99% the same.

This PR makes it so we have a single cache for an ${{ runner.os }}-${{ matrix.target }}-${{ matrix.toolchain }}, which is 3 in total for now. each of them weight 820-900MB.

The issue with the large number of cache size is that once we reach our limit GH starts to purge old caches, so sometimes some cargo check jobs didn't have a cache hit:/

Caches on main
Screenshot_2026-04-14-17-24-19-02_e4424258c8b8649f6e67d283a50a2cbc

Caches after change (there will be more entries with more invocations):

Screenshot_2026-04-14-17-23-14-68_e4424258c8b8649f6e67d283a50a2cbc

more targets & toolchains can be easily added if needed

Summary by CodeRabbit

  • Chores
    • Standardized and expanded CI caching to improve cache hit rates and build speed (broader restore fallbacks and unified keying).
    • Disabled debug-info generation for build profiles to reduce artifact size.
    • Removed restrictive cache-restore guards to allow more consistent cache usage.
    • Added an automated workflow to refresh and maintain build caches across supported OSes.

@ShaharNaveh ShaharNaveh changed the title Unify CI caches (reduce total size to about a 10th) Unify CI caches (reduce total size by 75% ±) Apr 14, 2026
@ShaharNaveh ShaharNaveh requested a review from fanninpm April 14, 2026 15:29
@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai bot commented Apr 14, 2026 •
edited
Loading

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yml

Review profile: CHILL

Plan: Pro

Run ID: bfe7695e-a891-44b2-8b5e-701e8d87c143

📥 Commits

Reviewing files that changed from the base of the PR and between 50f42b6 and ea1db89.

📒 Files selected for processing (1)
  • .github/workflows/ci.yaml
🚧 Files skipped from review as they are similar to previous changes (1)
  • .github/workflows/ci.yaml
📝 Walkthrough

Walkthrough

Replaces the Swatinem/rust-cache usage with explicit actions/cache/restore steps across CI, standardizes cache keys using ${{ runner.os }} and hashFiles('**/Cargo.toml'), removes branch-based restore guards, adds env vars disabling Cargo debug info, and introduces a new workflow to pre-warm caches across macOS, Ubuntu, and Windows.

Changes

Cohort / File(s) Summary
CI Cache Modernization
.​github/workflows/ci.yaml		 Replaced Swatinem/rust-cache with explicit actions/cache/restore steps and updated cache key/restore-keys to use ${{ runner.os }} and Cargo manifest hashes; removed if: github.ref != 'refs/heads/main' guards in restore steps; added CARGO_PROFILE_*_DEBUG=0 env vars; adjusted cargo_check key format and removed its explicit save step.
Cache Pre-warming Workflow
.​github/workflows/update-caches.yml		 Added new "Update Actions Caches" workflow (manual and push to main) that builds on macos-latest, ubuntu-latest, and windows-latest with the stable toolchain, runs builds for dev/test/release profiles with shared CARGO_ARGS, and saves Cargo registry/git and target/ artifacts to the Actions cache keyed by OS, toolchain, target, manifest hashes, and commit SHA.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Poem

🐇✨
I hopped through workflows, swift and spry,
Pre-warmed caches under every sky,
No more waiting, builds now fleet,
CI hums on nimble feet,
🥕🚀

🚥 Pre-merge checks | ✅ 3
✅ Passed checks (3 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title 'Unify CI caches (reduce total cache size by 75% ±)' accurately summarizes the main change: consolidating fragmented CI caches into unified ones, with the quantified benefit of reducing total cache size.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

ShaharNaveh
ShaharNaveh commented Apr 14, 2026
View reviewed changes
.github/workflows/update-caches.yml
Copy link
Copy Markdown
Contributor Author

@ShaharNaveh ShaharNaveh Apr 14, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

stole the entire thing from https://github.com/bevyengine/bevy/blob/4553e66916b882ba76be9423f401b79ebc1d7eec/.github/workflows/update-caches.yml

coderabbitai[bot]
coderabbitai bot reviewed Apr 14, 2026
View reviewed changes
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents 
Verify each finding against the current code and only fix it if needed.

Inline comments:
In @.github/workflows/ci.yaml:
- Around line 578-583: Restore keys use the wrong target triple string; replace
occurrences of "wasm-wasip1" with the correct "wasm32-wasip1" in the restore key
templates (specifically the entries like ${{ runner.os }}-stable-wasm-wasip1-${{
hashFiles('**/Cargo.toml') }}- and ${{ runner.os }}-stable-wasm-wasip1-) so the
key matches the installed target "wasm32-wasip1" and can be restored correctly.
- Around line 177-183: The cargo_check job's restore-keys don't match the cache
save key so caches never restore; update either the save key or the restore-keys
so they share the same prefix: locate the cargo_check job and its restore-keys
and change the restore patterns to include the "cargo-check-" prefix (e.g. start
with cargo-check-${{ runner.os }}-...) to match the save key used when caching,
or alternatively change the save key to use the shared `${{ runner.os
}}-stable-...` family; ensure the restore-keys and the key used in the save step
(the save block) are consistent for the same cache family.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yml

Review profile: CHILL

Plan: Pro

Run ID: df8e8056-50c5-4dca-99c6-fc911fe5a66f

📥 Commits

Reviewing files that changed from the base of the PR and between 898da7f and 50f42b6.

📒 Files selected for processing (2)
  • .github/workflows/ci.yaml
  • .github/workflows/update-caches.yml

@ShaharNaveh ShaharNaveh changed the title Unify CI caches (reduce total size by 75% ±) Unify CI caches (reduce total cache size by 75% ±) Apr 14, 2026
@youknowone youknowone merged commit a2c3e65 into RustPython:main Apr 15, 2026
20 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@coderabbitai coderabbitai[bot] coderabbitai[bot] left review comments

@youknowone youknowone youknowone approved these changes

@fanninpm fanninpm Awaiting requested review from fanninpm

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@ShaharNaveh @youknowone