Repository navigation
Skip re-downloading zigpy-ota index with unchanged version file - #1876
Draft
TheJulianJES wants to merge 2 commits into
Draft
TheJulianJES wants to merge 2 commits into
TheJulianJES wants to merge 2 commits into
Conversation
…anged The zigpy-ota provider fetches its index in two steps: a tiny version file on the `release/version` branch and the full index it points to. Remember the version file's `zigpy_v2` entry from the last successful load and skip the full index download (keeping the cached images) when it has not changed — the index URL is release-tag-pinned, so an unchanged entry means an unchanged index. This makes both the 24-hour background refresh and any forced cache invalidation nearly free when no new zigpy-ota release was published.
The `zigpy_v2` version marker was remembered at the end of the index generator, but the index is only accepted by the OTA manager after `load_index()` returns. A load failing in between — e.g. the fetch timeout cancelling `load_index()` while its HTTP session is being closed — would mark the index as loaded without the manager ever receiving it, permanently skipping every future index download. Stage the version info during the load instead and commit it via the new `_index_loaded()` hook, called past all suspension points, and add a regression test pinning the invariant. Also mention the freshness probe in the `invalidate_provider_caches()` docstring.
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## dev #1876 +/- ##
=======================================
Coverage 99.50% 99.50%
=======================================
Files 59 59
Lines 12412 12427 +15
=======================================
+ Hits 12350 12365 +15
Misses 62 62 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
DRAFT. I probably want to revisit and merge some/one of the other OTA PRs open first: At least the "Retry failed OTA index downloads with exponential backoff" commit from #1845, as this would impact this PR / the second "fix" commit here.
AI PR description: Important part / proposed change
The zigpy-ota provider fetches its index in two steps: a tiny version file on the
release/versionbranch and the full index it points to. This PR remembers the version file'szigpy_v2entry from the last successful load and skips the full index download — keeping the cached images — when it has not changed.Rest of detailed PR description
This is safe because the
zigpy_v2index URL is release-tag-pinned (e.g..../releases/download/0.0.20/zigpy_v2_ota.json): the index content cannot change without the version file changing. A change to an unrelated schema entry (e.g.z2m_v1) does not trigger a re-download either, since only thezigpy_v2entry is compared.With this, the 24-hour background index refresh and any forced cache invalidation (e.g. a user-initiated "check for updates" via
invalidate_provider_caches(), see #1843) become nearly free — a single ~200-byte download — unless a new zigpy-ota release was actually published.The second commit makes the version marker commit only once
load_index()has fully succeeded, past all suspension points: without it, the fetch timeout cancelling a load while its HTTP session is being closed (the index generator already finished, but the OTA manager never received the index) would have marked the index as loaded and permanently skipped every future index download until a new zigpy-ota release. A regression test pins the invariant.Notes
invalidate_provider_caches()semantics for the zigpy-ota provider: a forced invalidation re-reads only the version file when nothing was published, keeping the cached images (which are then still current). The docstring is updated accordingly.release/version. Re-uploading a release asset under the same tag would be invisible to this check (and was already unsafe before, since clients cache the index for 24 hours anyway)._index_last_updatedin the pre-existingfinallyblock, locking the provider out for up to 24 hours — unchanged here; Improve OTA provider index refresh resilience #1845 replaces that with retry backoff.Follow-up
Extracted from a larger branch (
zigpy-bot/ota-install-revalidation, prompted by the pulled zigpy-ota 2026.8.1 Sonoff images): a follow-up PR on top of this one re-checks the index right before installing an image, so a withdrawn image is refused instead of installed from a stale cache. The cheap version-file probe added here is what makes that install-time re-check nearly free.