Skip to content

gh-113433: Automatically Clean Up Subinterpreters in Py_Finalize() #121060

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
ericsnowcurrently merged 8 commits into from
Jun 26, 2024
Merged

gh-113433: Automatically Clean Up Subinterpreters in Py_Finalize() #121060

Changes from 1 commit
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
df63d15
Factor out resolve_final_tstate() and _Py_Finalize().
ericsnowcurrently Jun 25, 2024
f6df8be
Make sure the main interpreter is active in _Py_Finalize().
ericsnowcurrently Jun 25, 2024
071d270
Add a NEWS entry.
ericsnowcurrently Jun 26, 2024
ccb3aa2
Clean up all remaining subinterpreters.
ericsnowcurrently Jun 19, 2024
6b26e0a
Finalize subinterpreters *after* running pending calls and atexit hooks.
ericsnowcurrently Jun 26, 2024
74f4ed0
Make sure there's only one tstate left.
ericsnowcurrently Jun 26, 2024
02a969a
Add a NEWS entry.
ericsnowcurrently Jun 26, 2024
86cd3dd
Clean up subinterpreters only after modules (incl. __main__) have bee…
ericsnowcurrently Jun 26, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Clean up all remaining subinterpreters.
  • Loading branch information
@ericsnowcurrently
ericsnowcurrently committed Jun 26, 2024
commit ccb3aa263924e6b6a5064d857114e2d07b846c52
45 changes: 45 additions & 0 deletions Python/pylifecycle.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -74,6 +74,7 @@ static PyStatus init_sys_streams(PyThreadState *tstate);
static PyStatus init_android_streams(PyThreadState *tstate);
#endif
static void wait_for_thread_shutdown(PyThreadState *tstate);
static void finalize_subinterpreters(void);
static void call_ll_exitfuncs(_PyRuntimeState *runtime);

/* The following places the `_PyRuntime` structure in a location that can be
Expand Down Expand Up @@ -1982,6 +1983,8 @@ _Py_Finalize(_PyRuntimeState *runtime)
// Wrap up existing "threading"-module-created, non-daemon threads.
wait_for_thread_shutdown(tstate);

finalize_subinterpreters();

// Make any remaining pending calls.
_Py_FinishPendingCalls(tstate);

Expand Down Expand Up @@ -2415,6 +2418,48 @@ _Py_IsInterpreterFinalizing(PyInterpreterState *interp)
return finalizing != NULL;
}

static void
finalize_subinterpreters(void)
{
PyInterpreterState *main_interp = _PyInterpreterState_Main();
assert(_PyInterpreterState_GET() == main_interp);
_PyRuntimeState *runtime = main_interp->runtime;
struct pyinterpreters *interpreters = &runtime->interpreters;

/* Clean up all remaining subinterpreters. */
HEAD_LOCK(runtime);
PyInterpreterState *interp = interpreters->head;
if (interp == main_interp) {
interp = interp->next;
}
HEAD_UNLOCK(runtime);
if (interp != NULL) {
(void)PyErr_WarnEx(
PyExc_RuntimeWarning,
"remaining subinterpreters; "
"destroy them with _interpreters.destroy()",
0);
}
while (interp != NULL) {
/* Destroy the subinterpreter. */
assert(!_PyInterpreterState_IsRunningMain(interp));
PyThreadState *tstate =
_PyThreadState_NewBound(interp, _PyThreadState_WHENCE_FINI);
PyThreadState *save_tstate = PyThreadState_Swap(tstate);
Py_EndInterpreter(tstate);
(void)PyThreadState_Swap(save_tstate);

/* Advance to the next interpreter. */
HEAD_LOCK(runtime);
interp = interpreters->head;
if (interp == main_interp) {
interp = interp->next;
}
HEAD_UNLOCK(runtime);
}
}


/* Add the __main__ module */

static PyStatus
Expand Down