Skip to content

Fix deadlock in c-api when a thread enters a stop_the_world#8355

Merged
youknowone merged 2 commits into
RustPython:mainfrom
bschoenmaeckers:fix-c-api-deadloack
Jul 23, 2026
Merged

Fix deadlock in c-api when a thread enters a stop_the_world#8355
youknowone merged 2 commits into
RustPython:mainfrom
bschoenmaeckers:fix-c-api-deadloack

Conversation

@bschoenmaeckers

@bschoenmaeckers bschoenmaeckers commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

Store a lock-free pointer to the main interpreter so Py_IsInitialized & PyInterpreterState_Get/PyEval_SaveThread do not have to lock the main Mutex.

Tested that while true; cargo test --lib; end; does not deadlock anymore.

Summary by CodeRabbit

  • Bug Fixes
    • Improved reliability of interpreter initialization status checks.
    • Updated interpreter state retrieval to avoid failures when the main interpreter is not yet available.
  • Performance
    • Enabled faster, lock-free probing of interpreter initialization state.

@coderabbitai

coderabbitai Bot commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

📝 Walkthrough

Walkthrough

The C API adds an atomic main-interpreter pointer, publishes it during initialization, and uses it for initialization checks and interpreter-state retrieval.

Changes

Main interpreter pointer

Layer / File(s) Summary
Publish the main interpreter pointer
crates/capi/src/pylifecycle.rs, crates/capi/src/lib.rs
Defines the null-initialized atomic pointer and stores the initialized interpreter pointer during main interpreter setup.
Read the atomic pointer
crates/capi/src/pylifecycle.rs, crates/capi/src/pystate.rs
Py_IsInitialized and PyInterpreterState_Get load the pointer with Acquire ordering instead of using the mutex-backed lookup.

Estimated code review effort: 2 (Simple) | ~10 minutes

Sequence Diagram(s)

sequenceDiagram
  participant Py_InitializeEx
  participant init_main_interpreter
  participant MAIN_INTERP_PTR
  participant C_API_queries
  Py_InitializeEx->>init_main_interpreter: initialize main interpreter
  init_main_interpreter->>MAIN_INTERP_PTR: publish interpreter pointer
  C_API_queries->>MAIN_INTERP_PTR: load pointer with Acquire ordering
  MAIN_INTERP_PTR-->>C_API_queries: initialization status or interpreter state
Loading

Possibly related PRs

  • RustPython/RustPython#8133: Modifies the interpreter initialization flow in crates/capi/src/pylifecycle.rs, overlapping with pointer publication during initialization.

Suggested reviewers: shaharnaveh, youknowone

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately reflects the main change: fixing a C-API deadlock related to stop_the_world by removing locking from interpreter access.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@crates/capi/src/lib.rs`:
- Around line 65-68: Update the MAIN_INTERP_PTR.store call in
init_main_interpreter to use Release ordering instead of Relaxed, preserving the
existing pointer publication while synchronizing it with Acquire readers.

In `@crates/capi/src/pystate.rs`:
- Around line 69-70: Update PyInterpreterState_Get to preserve its non-null
contract by checking the result of MAIN_INTERP_PTR.load(Ordering::Acquire) and
invoking the crate’s existing fatal-error path when it is null; otherwise return
the loaded interpreter pointer unchanged.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yml

Review profile: CHILL

Plan: Pro Plus

Run ID: 7a8375bc-4e7d-4a73-b2ae-1d2cdd420bfb

📥 Commits

Reviewing files that changed from the base of the PR and between 9062f02 and 2e71298.

📒 Files selected for processing (3)
  • crates/capi/src/lib.rs
  • crates/capi/src/pylifecycle.rs
  • crates/capi/src/pystate.rs

Comment thread crates/capi/src/lib.rs
Comment thread crates/capi/src/pystate.rs Outdated

@youknowone youknowone left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@youknowone
youknowone merged commit 28454cc into RustPython:main Jul 23, 2026
26 checks passed
@bschoenmaeckers
bschoenmaeckers deleted the fix-c-api-deadloack branch July 23, 2026 18:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants