Do not report a successful connection when the transport dropped - #2
Open
evnchn wants to merge 1 commit into
Open
Do not report a successful connection when the transport dropped#2evnchn wants to merge 1 commit into
evnchn wants to merge 1 commit into
Conversation
If the Engine.IO transport dies while `connect()` is still awaiting the namespace handshake, `_handle_eio_disconnect()` runs while `connected` is still `False`, so its cleanup is skipped, and the resuming `connect()` then sets `connected = True` on a transport that is gone. The client cannot recover from that state: `disconnect()` reaches `eio.disconnect()`, whose body is guarded by `if self.state == 'connected'`, so the disconnect event that would reset the flag is never emitted; `shutdown()` delegates to `disconnect()`; and a fresh `connect()` refuses with "Already connected". Track whether the transport dropped during the attempt and raise `ConnectionError` instead of reporting success.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Opened by Claude Code on evnchn's behalf.
TL;DR:
AsyncClient/Clientcan be left permanentlyconnected == Trueon a dead transport, with no supported way to recover —disconnect(),shutdown()and a freshconnect()all fail.The gap is an ordering one:
connect()assignsself.connected = Trueas its last statement, after awaiting the namespace handshake._handle_eio_disconnect()guards its namespace teardown andself.connected = Falsebehindif self.connected:.False— andconnect()then sets it anyway, on a socket that is gone.This PR tracks whether the transport dropped during the attempt and raises
ConnectionErrorinstead of reporting success. +47 lines, of which 38 are tests; the behaviour change is 9 lines across three files.Full suite 661 passed, flake8 clean. Both new tests were confirmed to fail without the fix (
DID NOT RAISE ConnectionError).The change
base_client.py— initialiseself._connection_dropped = False.client.py/async_client.py— reset it at the start of eachconnect(), set it in_handle_eio_disconnect(), and raiseConnectionError('Connection dropped')instead of assigningconnected = Truewhen it is set.Why a flag rather than checking
self.eio.state. My first attempt gated the assignment onself.eio.state != 'connected'. It works at runtime but breaks 16 existing tests, because they drive a realengineio.Clientwith onlyeio.connectmocked, sostatenever leaves'disconnected'. Narrowing to== 'disconnected'did not help for the same reason. The flag records the invariant we actually care about — did a disconnect land during this attempt — and leaves the test doubles alone.A fix that did not work, recorded so nobody retries it: moving
self.namespaces = {}out of theif self.connected:guard, soconnect()'s existingset(self.namespaces) != set(self.connection_namespaces)check would fire. It fails 25/25, because_receive_packetdispatches withrun_async=True, so_handle_connectruns after the disconnect cleanup and repopulatesnamespaces.Reproduction, and before/after
Self-contained, socketio + aiohttp only, no monkeypatching and no writes to any client attribute. The server accepts the namespace and then immediately closes.
Before:
After:
Deterministic — the wedge reproduced 25/25 on the unpatched build (macOS 15 arm64 and Linux
python:3.12-slim), and a control that adds one 0.1 s server-side delay before closing reproduced 0/25.What this proves: the wedged state is reachable and unrecoverable, and the fix removes it. What it does not prove: that a cooperative server produces it unaided — see below.
What I could NOT reproduce (so nobody re-walks it)
A plain cooperative
AsyncServercallingsio.disconnect(sid)from its ownconnecthandler does not reproduce, at any delay from 0 to 5 ms (0/45 attempts). With no proxy and no client-sideawait, the client always wins the race — the window is narrower than a network round-trip.So the trigger above is fair to call staged. Two things widen the window in practice:
_receive_packetdispatches withrun_async=True, so the message handler runs in a separate task, decoupled from the read loop._handle_connectpopulatesself.namespaces[ns]and then awaits the user'sconnecthandler before_connect_event.set(). Anyawaitin that handler — an emit, a DB call, an HTTP request — widens it to the handler's duration.A variant using a genuine
AsyncServerwith an ordinary asyncconnecthandler, severed in the network by a small TCP proxy, reproduces identically with a real server-issued sid innamespaces. That is what a load balancer, proxy or restarting pod does. Happy to add it if useful.Precision note:
_handle_eio_disconnect()does clearself.callbacks,self._binary_packetandself.sidunconditionally. Only the namespace teardown andself.connected = Falsesit inside the guard — which is enough.Left alone deliberately
disconnect()still cannot clearconnectedwhen the transport is already gone, becauseeio.disconnect()is guarded byif self.state == 'connected'. This PR removes the known path into that state rather than making the state escapable. Makingdisconnect()always resetconnectedwould be a good belt-and-braces follow-up, but it is a separate behavioural change and I did not want to bundle it.Found while investigating a downstream report of devices becoming permanently unreachable after a network change, with
connected == Trueand no TCP connection in the process's socket list.Environment: reproduced on python-socketio 5.16.3 / python-engineio 4.13.3 (current PyPI latest, so not a stale pin), aiohttp 3.14.3, websocket transport, Python 3.12, macOS 15 arm64 and Linux aarch64. Untested: Windows, and the polling transport.