Skip to content

Fix network_retry_loop stop_event handling: blocking shutdown, orphaned tasks, and runaway repeat loop - #5310

Open
xsneh wants to merge 1 commit into
python-telegram-bot:masterfrom
xsneh:networkloop-stop-event-sleep
Open

Fix network_retry_loop stop_event handling: blocking shutdown, orphaned tasks, and runaway repeat loop#5310
xsneh wants to merge 1 commit into
python-telegram-bot:masterfrom
xsneh:networkloop-stop-event-sleep

Conversation

@xsneh

@xsneh xsneh commented Jul 27, 2026

Copy link
Copy Markdown
Contributor

Summary

Fixes three related bugs in networkloop.py caused by stop_event being partially ignored during shutdown:

  • Shutdown hangs: network_retry_loop used a bare asyncio.sleep during its retry backoff, ignoring stop_event entirely. This caused shutdown routines to hang for up to 30 seconds during network outages. Replaced with asyncio.wait_for(stop_event.wait(), timeout=...).
  • Orphaned tasks: Pending tasks (such as in-flight HTTP calls) were cancelled but never awaited before do_action returned, leading to "Task was destroyed but it is pending!" garbage collector warnings. Added asyncio.gather(*pending, return_exceptions=True) to ensure clean teardown.
  • Runaway loop: When repeat_on_success=True, the loop failed to break if stop_event was set during an action. The loop would immediately restart and spin indefinitely. Added explicit stop_event.is_set() exit checks to break the outer loop safely.

Adds three regression tests to prevent recurrence.

Tests

uv run --group tests pytest tests/ext/_utils/test_networkloop.py -v

Results:

tests/ext/_utils/test_networkloop.py::TestNetworkRetryLoop::test_stop_event_interrupts_backoff_sleep PASSED [ 87%]
tests/ext/_utils/test_networkloop.py::TestNetworkRetryLoop::test_stop_event_breaks_repeat_on_success_loop PASSED [ 93%]
tests/ext/_utils/test_networkloop.py::TestNetworkRetryLoop::test_no_pending_tasks_after_stop_event PASSED [100%]

============================= 16 passed in 13.90s ==============================

Live verification:

Ran a custom asyncio verification script locally simulating network outages. Confirmed the event loop exits promptly on shutdown and no pending task warnings are emitted during teardown.

@xsneh
xsneh force-pushed the networkloop-stop-event-sleep branch 4 times, most recently from b140442 to f85a252 Compare July 27, 2026 11:07
…twork outages

Bug 1: Replace bare asyncio.sleep(cur_interval) with an event-aware
asyncio.wait_for(stop_event.wait(), timeout=cur_interval) so that the
backoff sleep is interrupted immediately when the stop_event is set.
Previously, a bot experiencing a network outage could hang for up to
30 seconds on graceful shutdown while the backoff sleep ran uninterrupted.

Bug 2: After calling .cancel() on pending asyncio tasks inside do_action(),
await asyncio.gather(*pending, return_exceptions=True) to ensure cancelled
tasks are fully cleaned up before returning. Without this, merely requesting
cancellation left tasks in a pending state, causing 'Task was destroyed but
it is pending!' warnings from the garbage collector when the tasks were
eventually collected.
@xsneh
xsneh force-pushed the networkloop-stop-event-sleep branch from f85a252 to fa2bd5f Compare July 27, 2026 11:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant