ci: cache poetry venv keyed on lockfile and cython sources#1704
Merged
Conversation
Caches the poetry-managed .venv plus the built Cython artefacts (src/zeroconf/**/*.so) for both the test matrix and benchmark jobs, keyed on runner OS + image version + python version + extension mode + hash of poetry.lock/pyproject.toml/build_ext.py + hash of src/zeroconf/**/*.py and **/*.pxd. On a cache hit, the Install Dependencies step is skipped. Pure Python and Cython source changes both invalidate the cache, so tests never run against stale .so files. POETRY_VIRTUALENVS_IN_PROJECT="true" is set at the job level so poetry creates .venv inside the repo and the cache can target a single path.
Three trims:
1. Drop ${{ env.ImageOS }}-${{ env.ImageVersion }}- from both
cache keys. env context only resolves workflow-declared env
vars, not runner process env vars like ImageOS / ImageVersion,
so these interpolations were silently empty (visible in
dbus-fast's post-merge logs of the same pattern).
2. Merge the two hashFiles() calls into one covering poetry.lock,
pyproject.toml, build_ext.py, src/zeroconf/**/*.py, and
src/zeroconf/**/*.pxd. Concatenated either way; one call is
easier to read.
3. Promote POETRY_VIRTUALENVS_IN_PROJECT to workflow-level env:
instead of duplicating on the test and benchmark jobs.
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #1704 +/- ##
=======================================
Coverage 99.76% 99.76%
=======================================
Files 33 33
Lines 3410 3410
Branches 464 464
=======================================
Hits 3402 3402
Misses 5 5
Partials 3 3 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Cache the poetry-managed
.venvplus built Cython artefacts (src/zeroconf/**/*.so) across the test matrix and benchmark jobs. Mirrors dbus-fast#675 and the follow-up cleanup in dbus-fast#679, adapted to this repo's matrix shape.Details
Cache key components:
runner.os+matrix.python-version+matrix.extension(skip_cython/use_cython)hashFiles('poetry.lock', 'pyproject.toml', 'build_ext.py', 'src/zeroconf/**/*.py', 'src/zeroconf/**/*.pxd')On a cache hit, the
Install Dependenciesstep is skipped viaif: steps.cache-venv.outputs.cache-hit != 'true'. Pure-Python and Cython-source changes both invalidate the cache, so we never run tests against stale.sofiles.POETRY_VIRTUALENVS_IN_PROJECT: "true"is set at workflow-levelenv:so poetry creates.venvinside the repo (cacheable as a single path) and the env var lives in one place.Why the cache key looks the way it does:
extensionaxis is in the key becauseskip_cythonanduse_cythonproduce different venvs (one has.sofiles, one doesn't)..cfiles are intentionally not cached. They're regenerated from.py/.pxdby Cython, and only needed when we rebuild — and on a hit we skip the rebuild.pyproject.tomlandbuild_ext.pyare in the hash so changes toTO_CYTHONIZE, extension flags, or runtime deps invalidate the cache.The matrix here is ~30 cells (7 Python × 3 OS × 2 extension modes, minus exclusions) plus the benchmark job, so unchanged-deps PRs that only touch tests or docs should now skip ~30 poetry installs and ~13 Cython rebuilds.
Test plan
src/zeroconf/**/*.pyor*.pxdshould hit the cache and skip installactions/cachestep succeeds across Linux / macOS / Windowsuse_cythoncells continue to exercise the Cython extension (CodSpeed signal is the canary)3.14tentry still passes