Skip to content

Support OTA updates for devices exposing OTA as a server cluster - #1915

Draft
TheJulianJES wants to merge 3 commits into
zigpy:devfrom
TheJulianJES:tjj/ota-server-cluster-fallback
Draft

TheJulianJES wants to merge 3 commits into
zigpy:devfrom
TheJulianJES:tjj/ota-server-cluster-fallback

Conversation

@TheJulianJES

Copy link
Copy Markdown
Contributor

DRAFT.

Proposed change

Some devices (e.g. the Third Reality 3RSS007Z) list the OTA cluster as an input (server) cluster in their simple descriptor, even though they run a normal OTA client and send spec-correct client-to-server query_next_image commands. zigpy already routes those frames to the server cluster via the direction-flip fallback (see #1640), and ZHA creates an update entity for them (FirmwareUpdateServerEntity, zigpy/zha#425), so updates are offered. Installing one has never worked, though:

  1. OTAManager.__init__ and the post-OTA cleanup in Device.update_firmware look up the OTA cluster with find_cluster(..., ClusterType.Client), which raises ValueError: Cluster 0x0019 not found on these devices. The install fails before any transfer starts.
  2. Even with the right cluster object, Cluster.request()/reply() derive the frame direction from the cluster type, so everything sent through a server OTA cluster went out client-to-server. The device's OTA client doesn't recognize those frames, e.g. image_notify (client command 0x00) arrives as an undefined server command. This also affected the existing NO_IMAGE_AVAILABLE reply zigpy sends to these devices.

This PR:

  • adds Device.find_ota_cluster(), which prefers a client OTA cluster on any endpoint and falls back to a server one, and uses it in OTAManager and the post-OTA cleanup.
  • adds a Cluster.send_direction property (unchanged behavior for every cluster) that request()/reply() use for the frame direction and the disable_default_response default. Ota overrides it to always send server-to-client, since zigpy only implements the OTA server role and all OTA attributes are client-side (ZCL R8 §11.10).
  • keeps the join-time OTA lookup in _discover client-only. The coordinator's own endpoint lists OTA as a server cluster (register_endpoints), and bellows/zigpy-znp initialize the coordinator device on every startup, so falling back there would make every coordinator read OTA attributes from itself and fail init after a timeout. A regression test covers this.

Known limitation

After a successful update, update_firmware() re-interviews the device, and _device_reinterviewed() does not carry over the OTA attribute cache or last_query_cmd. Since _discover stays client-only, a server-cluster OTA device shows no installed version after the update until it sends its next query_next_image. Handling the join-time path for these devices safely (coordinator guard, guarded attribute read) is left for a follow-up.

Related

Testing

  • New tests: full OTA run against a device with a server-only OTA cluster (asserts every sent frame is server-to-client and that the post-OTA cleanup runs on that cluster), find_ota_cluster() lookup order incl. multi-endpoint, send_direction for request/reply on regular and OTA clusters, and coordinator startup sending no OTA traffic. They fail without the fix; the integration test also fails with only the cluster fallback, since the direction fix is needed as well.
  • zigpy test suite passes; zha and zha-quirks test suites pass with this branch installed.
  • Not yet tested on real hardware.

Devices that expose their OTA client as a server cluster still expect OTA server
commands, so the direction of sent frames can't be derived from the cluster type.
The coordinator's own endpoint lists OTA as a server cluster, so falling back to it
during initialization made every coordinator try to read OTA attributes from itself.
@codecov

codecov Bot commented Oct 15, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 99.52%. Comparing base (2fdc4ab) to head (237a01a).

Additional details and impacted files
@@           Coverage Diff           @@
##              dev    #1915   +/-   ##
=======================================
  Coverage   99.51%   99.52%           
=======================================
  Files          61       61           
  Lines       12777    12790   +13     
=======================================
+ Hits        12715    12729   +14     
+ Misses         62       61    -1     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@TheJulianJES

TheJulianJES commented Oct 15, 2026 •

Copy link
Copy Markdown
Contributor Author

@zigpy-review-bot Alternatively, think about if we should just always swap a server OTA cluster to a client cluster on all devices zigpy-sided, since none should really have server ones in the first place. (Note there's an ongoing review session for this PR, so possibly communicate with that session: review <=> process mentions.)

@zigpy-review-bot zigpy-review-bot left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No blocking findings. The server-cluster fallback and the Ota.send_direction override are correct and narrowly scoped (unchanged for every non-OTA cluster and for client OTA clusters), and the two regression tests (test_update_device_firmware_server_ota_cluster, test_initialize_coordinator_no_ota_traffic) fail when their fix is reverted. What's left is the hardware test the description already lists, so this is a comment rather than an approval while the PR is a draft.

Notes

  • The direction-flip fallback now carries OTA for these devices. On a server-only OTA device, every inbound query_next_image / image_block / upgrade_end reaches the server Ota cluster, and so the OTA manager's callbacks, only through the fallback in _find_zcl_cluster (zigpy/device.py:1073). Without it, _match_packet_endpoint_cluster drops the frame as an unknown cluster. That fallback's comment still says "This will be removed in 2025.9.0." Optional: reword it so it says OTA on these devices depends on it, so whoever removes it later doesn't silently break these installs (inline note on find_ota_cluster).
  • Overlap with draft #1849. It rewrites the post-OTA block in update_firmware that this PR touches and keeps find_cluster(..., ClusterType.Client) there. Whichever lands second needs find_ota_cluster() in the rewritten block, otherwise server-only devices hit the same ValueError after a successful transfer.
  • For the hardware test: the post-OTA read_attributes(..., retries=10) at zigpy/device.py:1359 is not guarded. If the 3RSS007Z doesn't answer a server-to-client read on the OTA cluster it lists as input, update_firmware raises after the transfer has completed, and the image_notify and reinterview() are skipped. This is existing behaviour for every device, but it is the untested part of the new direction on these devices, so it's worth watching (confirmed with a variant of the new integration test whose device fails the read: DeliveryError propagates, the cache-cleared event has fired, reinterview is not called).
  • ZHA follow-up, besides _read_current_firmware_version: OtaCurrentFileVersionCache (zha/application/platforms/virtual.py) only matches client OTA clusters, so on these devices current_file_version isn't refreshed from query_next_image. The update entity still gets its installed version from OtaImageAvailableEvent, so this only affects the attribute cache.
Verified (7 checks)
  • zigpy suite at 237a01a: 1362 passed.
  • zha suite (dev 0ea5e1ad) with this branch installed editable: 1458 passed.
  • send_direction is the old is_client expression for every cluster except Ota, and Ota already sent server-to-client from a client cluster, so only server Ota clusters change. No zha / zha-quirks code subclasses Ota or overrides request/reply.
  • Response matching (device.request keys on the flipped request direction) lines up with the new direction: a server-to-client read expects the client-to-server response the device's OTA client sends. Automatic default responses on a server Ota cluster now also go server-to-client, which is correct for frames from the device's client.
  • Reverting only the Ota.send_direction override makes test_update_device_firmware_server_ota_cluster fail; switching _discover to find_ota_cluster() makes test_initialize_coordinator_no_ota_traffic fail. Both pin the behaviour the description claims.
  • Z2M has no ota: true for the 3RSS007Z (zigbee-herdsman-converters third_reality.ts), and herdsman's OTA picks endpoints via supportsOutputCluster("genOta"), so there's no prior art for this device either way.
  • Copilot second opinion (two models): no findings.

Comment thread zigpy/device.py
"""Find the OTA cluster, falling back to a server cluster.

OTA is a client cluster on the device, but some devices expose it as a server
cluster instead. Runtime routing also falls back to the server cluster for

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Optional: this makes the direction-flip fallback in _find_zcl_cluster (zigpy/device.py:1080-1081) load-bearing for OTA on these devices, but its comment still says "This will be removed in 2025.9.0." Consider rewording that comment so the dependency is visible to whoever eventually removes it.

@zigpy-review-bot

Copy link
Copy Markdown
Collaborator

I looked into it. In short, a blanket swap wouldn't be a simpler replacement for this PR. The descriptor it would change is what quirks match on, so it needs quirk-side care and more zigpy code than find_ota_cluster() + send_direction. I'd keep the PR's approach.

What a swap would gain

  • Incoming OTA frames from these devices would hit the client cluster directly, so OTA would no longer depend on the direction-flip fallback in _find_zcl_cluster (the one whose comment says it will be removed).
  • ZHA would use the normal FirmwareUpdateEntity, and FirmwareUpdateServerEntity plus the server-side diagnostics handling could go away later. The update entity's unique ID is <ieee>-<ep>-25-firmware_update for both classes, so no registry orphans.

What it breaks or needs

  • Quirk matching. v1 signatures compare cluster sets exactly (set(a) == set(b), zhaquirks/legacy/__init__.py:91), and 12 v1 quirk files list OTA as an input cluster. That includes the 3RSS007Z's own quirk (zhaquirks/thirdreality/switch.py) and several Xiaomi devices (cube, sensor_ht, magnet, motion, …). A swap before matching stops those quirks from matching. Swapping between matching and replacement doesn't work either, because the replacements add OTA back as an input cluster. In v2, the Candeo quirks call .removes(Ota.cluster_id), which defaults to the server cluster, so a pre-quirk swap would make that removal miss. Those devices would then get an update entity their quirk deliberately drops.
  • So it has to run after quirk resolution, on the final device, in both paths: _finalize_device for joins/re-interviews and appdb.load() after _resolve_device. original_signature and the DB rows then keep the reported descriptor. Attribute-cache rows are keyed by cluster type, so cached OTA attributes from the server cluster are skipped on load unless they're remapped.
  • It doesn't remove the _discover part. _discover runs on the raw device before resolution, so the join/re-interview OTA read still needs its own fallback, with the coordinator guard. Without that, the PR's "Known limitation" stays the same with or without the swap.
  • Coordinator. zigpy registers OTA as an input cluster on its own endpoint (zigpy/application.py:1016), and bellows/znp initialize the coordinator on startup. It has to be exempt, or it reads OTA attributes from itself, which is the failure this PR already has a regression test for.
  • OTA in both directions. Some devices list OTA as both input and output (Eurotronic SPZB0001, Xiaomi cube). Only server-only endpoints should be touched. Where both exist, the PR's client-first lookup already picks the client.
  • Object. It needs a fresh client Ota instance rather than moving the server one (cluster type is fixed at construction), plus the endpoint attribute/DB listener registration that add_output_cluster does.
  • "None should have server ones" holds for end devices. A remote device that really runs an OTA server (another gateway on the network) would be misrepresented. That's rare, but a blanket rule can't tell the difference.

If you want it anyway, the minimal safe version is a post-resolution normalizer for non-coordinator devices. It would only touch server-only OTA endpoints and create a fresh client Ota, with signature and DB rows left as reported, plus the _discover fallback. That's more code than this PR, and its main payoff is not depending on the direction-flip fallback. I'd only go that route once that fallback is actually being removed.

What I checked
  • zigpy at the PR head (237a01a): Endpoint.initialize adds clusters straight from the simple descriptor; appdb.load() runs _load_clusters → original_signature = get_signature() → _resolve_device → attribute-cache re-populate keyed by cluster_type; _save_clusters inserts with ON CONFLICT DO NOTHING (old rows are not removed); _load_ota_query_cache already restores client-first, then server.
  • zha-device-handlers (d6fcec59): INPUT_CLUSTERS blocks containing OTA in 12 files (eurotronic/spzb0001, thirdreality/switch, and 10 Xiaomi files); the 3RSS007Z signature and replacement both list Ota.cluster_id as input; QuirkBuilder.removes() defaults to ClusterType.Server (zhaquirks/builder/builder.py:508), used for OTA in zhaquirks/candeo/rotary_dimmer_switches.py.
  • zha (0ea5e1ad): FirmwareUpdateServerEntity subclasses FirmwareUpdateEntity and keeps the firmware_update suffix; the unique ID is {ieee}-{endpoint}-{cluster_id}-{suffix} (zha/application/platforms/__init__.py:575-580), with no direction in it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bugfix This PR fixes a bug

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants