Skip to content

RemoteUnwinder.get_async_stack_trace() crashes with a C stack overflow on a deeply nested awaited_by graph instead of raising #151535

@tonghuaroot

Description

@tonghuaroot

Bug description

_remote_debugging.RemoteUnwinder.get_async_stack_trace() reconstructs the async
call graph of a target process by recursing up the awaited_by relation. The
recursion is a three-function cycle with no depth limit, cycle detection, or
_Py_EnterRecursiveCall guard:

process_task_and_waitersprocess_task_awaited_byprocess_waiter_task
process_task_and_waiters (…)

(On main these are in Modules/_remote_debugging/asyncio.c; on 3.14 they are in
the single-file Modules/_remote_debugging_module.c.) Each level also stack-allocates
a char task_obj[SIZEOF_TASK_OBJ] (SIZEOF_TASK_OBJ == 4096), so ~1900 levels
exhaust a default 8 MiB stack. When the target's running task sits at the bottom of
a sufficiently deep awaited_by chain, the debugger process (the one calling
get_async_stack_trace()) overflows its C stack and dies with SIGSEGV.

This is asymmetric with the iterative sibling path: get_all_awaited_by /
append_awaited_by_for_thread bounds its walk with MAX_ITERATIONS = 2 << 15.
Only the recursive get_async_stack_trace path is unbounded. The module already
treats the target's tables as untrusted input (debug_offsets_validation.h) and the
thread-list walk already has explicit "corrupted remote memory" cycle detection, so
bounding this traversal is consistent with the module's existing invariants.

The same pattern (C recursion converted to RecursionError instead of a segfault)
was treated as a bug in #137894.

Reproducer

A target with a deep linear awaited_by chain whose leaf is the running task; a
second process attaches and calls get_async_stack_trace():

# target.py <N>:  tN await t(N-1) await ... await leaf;  leaf busy-spins (= running task)
# attacker.py <pid>:
from _remote_debugging import RemoteUnwinder
RemoteUnwinder(int(pid)).get_async_stack_trace()
  • N = 10 → returns a stack trace cleanly (exit 0).
  • N >= ~2000 → attacker process SIGSEGV (exit 139). gdb shows ~1884 stacked
    process_task_awaited_by frames terminating at a guard-page fault.

(Full PoC scripts available on request.)

Reproduced on

  • CPython 3.14.6 (GA, python:3.14 image, aarch64 Linux) — crashes.
  • CPython 3.16 main (local --with-pydebug build) — same unguarded recursion in source.

Cross-process attach uses the normal Linux ptrace/process_vm_readv path
(--cap-add=SYS_PTRACE); the crash is in the debugger, driven by the target's
graph shape. A privileged profiler/observability tool attaching to an untrusted (or
just legitimately deeply nested) workload is the realistic setting.

Expected behavior

A bounded traversal — raise/propagate an error (as the iterative path does on hitting
its limit), not crash the debugger process.

Proposed fix

Bound the process_task_and_waitersprocess_waiter_task recursion, matching the
iterative sibling. I have a PR ready that adds an explicit recursion-depth cap
(MAX_TASK_AWAITED_BY_DEPTH, mirroring the existing MAX_ITERATIONS /
MAX_SET_TABLE_SIZE constants in the module) and raises a RuntimeError on
overflow, which also handles a cyclic awaited_by graph from corrupted remote
memory. Happy to open it.

Linked PRs

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions