Skip to content

gh-80406: Finalise subinterpreters in Py_FinalizeEx() #17575

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
LewisGaul wants to merge 21 commits into
base: main
Choose a base branch
from
Open

gh-80406: Finalise subinterpreters in Py_FinalizeEx() #17575

Changes from 1 commit
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
23af5f5
Add test suggested by ncoghlan
LewisGaul Nov 21, 2019
433663c
Finalise sub-interpreters in Py_FinalizeEx()
LewisGaul Dec 11, 2019
48e1cfc
Improve test name
LewisGaul Dec 13, 2019
0400634
Switch back to main threadstate in test_audit_subinterpreter before c…
LewisGaul Dec 13, 2019
b79649c
📜🤖 Added by blurb_it.
blurb-it[bot] Dec 14, 2019
8b1e7d9
Markups including: switch from 'finalizing' flag to 'allow_new', add …
LewisGaul Jan 21, 2020
fd6073a
Merge branch 'finalise-subinterps' of github.com:LewisGaul/cpython in…
LewisGaul Jan 21, 2020
4bbd58f
Merge branch 'master' into finalise-subinterps
LewisGaul Oct 20, 2020
1095e66
Use '_' for unused variable in test_embed.py
LewisGaul Oct 20, 2020
675285d
Fix struct position of 'allow_new' flag
LewisGaul Oct 22, 2020
8e21788
Add handling for unsupported case of calling Py_Finalize() from a sub…
LewisGaul Oct 22, 2020
606c068
Emit resource warning when calling Py_Finalize() with unfinalized sub…
LewisGaul Oct 22, 2020
e0789b0
Update Py_FinalizeEx() docs
LewisGaul Oct 22, 2020
dda99ce
Update test for resource warning when implicitly finalizing subinterp…
LewisGaul Oct 23, 2020
847e8d2
Tidy up test_finalize_subinterps() testcase
LewisGaul Oct 23, 2020
a2fb0fc
Add testcase for calling Py_Finalize() from a subinterpreter
LewisGaul Oct 23, 2020
d234528
Tweak subinterpreters still running ResourceWarning handling
LewisGaul Nov 23, 2020
46a8619
Make calling PyFinalizeEx() from a subinterpreter a Py_FatalError
LewisGaul Nov 23, 2020
c89c0e5
Acquire interpreters mutex before setting allow_new=0 in PyFinalizeEx()
LewisGaul Nov 23, 2020
c285f52
Merge remote-tracking branch 'upstream/master' into finalise-subinterps
LewisGaul Nov 23, 2020
95cbfd4
Add back in the 'interp' variable to PyFinalizeEx() to fix the build
LewisGaul Nov 23, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Acquire interpreters mutex before setting allow_new=0 in PyFinalizeEx()
  • Loading branch information
@LewisGaul
LewisGaul committed Nov 23, 2020
commit c89c0e55c32fa9edb6ad479c4cbcacb13b089b03
2 changes: 2 additions & 0 deletions Python/pylifecycle.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1359,7 +1359,9 @@ Py_FinalizeEx(void)
}

// Finalize sub-interpreters.
PyThread_acquire_lock(runtime->interpreters.mutex, WAIT_LOCK);
runtime->interpreters.allow_new = 0;
Copy link
Copy Markdown
Member

@vstinner vstinner Oct 27, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It may be safer to acquire _PyRuntime.interpreters.mutex beforing setting this variable. It may be better to move this code into pystate.c, since this file control the list of interpreters.

Copy link
Copy Markdown
Contributor Author

@LewisGaul LewisGaul Nov 23, 2020 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I presume you're suggesting refactoring the 'finalize subinterpreters' logic? It would seem to me a function that does 'finalizing' belongs better in pylifecycle than pystate? I believe this refactoring was referred to by Eric above, where he says this can be addressed separately.

I've added in an acquisition of the lock here for now.

PyThread_release_lock(runtime->interpreters.mutex);
PyInterpreterState *curr_interp = PyInterpreterState_Head();
PyInterpreterState *next_interp;
int64_t num_destroyed = 0;
Expand Down