Tear down a removed device after its leave request completes - #1892

Proposed change

_remove_device() popped the device from the device table once the ZDO leave request finished, but never tore the Device object down. For a device that is not yet initialized, its own Mgmt_Leave_rsp goes through the uninitialized-device branch of packet_received(), which re-schedules initialization right before the pop. The orphaned initialize task then keeps sending Node_Desc_req to a device that has already left (three attempts of 28 s each on dev, five on older releases) and emits device_init_failure for a device that is no longer registered. This is the zigpy side of #1802.

This PR calls Device.on_remove() after the leave exchange, so the removed device's tasks and callbacks are cancelled the same way they are on shutdown and on re-interview swaps. on_remove() was never called for a device removed through remove(), so removed devices also leaked the PollControl check-in listener registered in Device.__init__. If a distinct object was registered under the same IEEE while the leave request was in flight (a re-interview shadow, or an application re-adding the device from its own database), that object is popped as before and now torn down as well. The task done-callback registered by Device.create_task() is changed from set.remove to set.discard, because on_remove() clears _tasks while the cancelled tasks are still completing, and their callbacks then raised KeyError in the event loop (on_remove() is also called on shutdown, so that was reachable before this change too).

Related: #1816 adds the same teardown for the original device when a quirk replaces it, and makes the identical remove to discard change in Device.create_task() (with a different comment), so whichever of the two merges second needs a trivial rebase on that hunk. The _remove_device() change itself does not overlap with it.

Removing a device while it is being re-interviewed used to re-register it: reinterview() restored the old object after a failed discovery, or handed the shadow to _device_reinterviewed() for finalization, regardless of the removal. With the teardown that would have registered a torn-down object, so both paths now check that the shadow is still the registered device (reinterview() after discovery, successful or not, and _device_reinterviewed() after the awaited DB delete) and otherwise discard it, tear down the old device as well (in that order remove() picks up the shadow, so nothing else would have torn the old object down) and emit device_reinterview_failure so callers get a completion signal.

Behaviour note for remove(..., rejoin=True): if the Mgmt_Leave_rsp is lost and the device rejoins and announces within the leave timeout, handle_join() reuses the still-registered object and schedules its initialization, which the teardown now cancels; previously that orphaned initialization could complete after the pop and silently re-register the device through _finalize_device(). The device now comes back through unknown-device discovery on its next packet. zha never passes rejoin=True.

Additional information

• Fixes the zigpy part of Removing an uninitialized device restarts its initialization on the Mgmt_Leave_rsp #1802 (the second, interleaved initialization loop in that log is the plugin re-adding the device from its own database after zigpy popped it).
• Tests: test_remove_tears_down_device_after_leave_response reproduces the issue log (uninitialized device, remove(), Mgmt_Leave_rsp fed through packet_received()) and asserts no event-loop callback errors; test_remove_tears_down_replacement_device covers a distinct object registered during removal; test_reinterview_discards_shadow_when_device_removed, test_reinterview_failure_after_device_removed, test_device_reinterviewed_skips_device_removed_during_swap and test_device_reinterviewed_keeps_rejoined_device_during_swap cover the re-interview races. All fail on dev without the fix. The existing direct callers of _device_reinterviewed() in the tests now register the shadow first, as reinterview() does.

Checklist

• The code change is tested and works locally
• Tests have been added to verify that the new code works