Skip to content

gh-120837: Update _Py_DumpExtensionModules to be async-signal-safe #121051

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
corona10 merged 8 commits into from
Jun 27, 2024
Merged

gh-120837: Update _Py_DumpExtensionModules to be async-signal-safe #121051

Changes from 1 commit
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
8430508
gh-120837: Update _Py_DumpExtensionModules to be async-signal-safe
corona10 Jun 26, 2024
07962e5
nit
corona10 Jun 26, 2024
1f56fcc
fix
corona10 Jun 26, 2024
e7f24af
Address code review
corona10 Jun 27, 2024
ed9bb3b
Address code review
corona10 Jun 27, 2024
136e8d1
nit
corona10 Jun 27, 2024
61de424
Address code review
corona10 Jun 27, 2024
2d31780
Address code review
corona10 Jun 27, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Next Next commit
gh-120837: Update _Py_DumpExtensionModules to be async-signal-safe
  • Loading branch information
@corona10
corona10 committed Jun 26, 2024
commit 84305083c7f69f46013c729faf69588469935162
116 changes: 76 additions & 40 deletions Python/pylifecycle.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2897,6 +2897,30 @@ fatal_error_exit(int status)
}
}

static inline int
acquire_dict_lock_for_dump(PyObject *obj) {
Comment thread
colesbury marked this conversation as resolved.
Outdated
#ifdef Py_GIL_DISABLED
PyMutex *mutex = &obj->ob_mutex;
if (_PyMutex_LockTimed(mutex, 0, 0) == PY_LOCK_ACQUIRED) {
return 0;
Comment thread
colesbury marked this conversation as resolved.
Outdated
}
return -1;
#else
return 0;
#endif
}

static inline void
release_dict_lock_for_dump(PyObject *obj) {
Comment thread
colesbury marked this conversation as resolved.
Outdated
#ifdef Py_GIL_DISABLED
PyMutex *mutex = &obj->ob_mutex;
uint8_t expected = _Py_UNLOCKED;
// Do not wake up other threads.
Comment thread
corona10 marked this conversation as resolved.
Outdated
_Py_atomic_compare_exchange_uint8(&mutex->_bits, &expected, _Py_UNLOCKED);
Copy link
Copy Markdown
Member Author

@corona10 corona10 Jun 26, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we have to run for loop to prevent CAS failure?

Copy link
Copy Markdown
Contributor

@colesbury colesbury Jun 26, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this should be a _Py_atomic_store_uint8(&mutex->_bits, _Py_UNLOCKED)

#else
return;
Copy link
Copy Markdown
Contributor

@colesbury colesbury Jun 27, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The #else return; doesn't do anything. I would delete it, but up to you.

Copy link
Copy Markdown
Member Author

@corona10 corona10 Jun 27, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done!

#endif
}

// Dump the list of extension modules of sys.modules, excluding stdlib modules
// (sys.stdlib_module_names), into fd file descriptor.
Expand Down Expand Up @@ -2924,12 +2948,18 @@ _Py_DumpExtensionModules(int fd, PyInterpreterState *interp)
PyObject *stdlib_module_names = NULL;
if (interp->sysdict != NULL) {
pos = 0;
while (PyDict_Next(interp->sysdict, &pos, &key, &value)) {
if (PyUnicode_Check(key)
&& PyUnicode_CompareWithASCIIString(key, "stdlib_module_names") == 0) {
stdlib_module_names = value;
break;
if (acquire_dict_lock_for_dump(interp->sysdict) == 0) {
while (_PyDict_Next(interp->sysdict, &pos, &key, &value, NULL)) {
if (PyUnicode_Check(key)
&& PyUnicode_CompareWithASCIIString(key, "stdlib_module_names") == 0) {
stdlib_module_names = value;
break;
}
}
release_dict_lock_for_dump(interp->sysdict);
}
else {
return;
}
}
// If we failed to get sys.stdlib_module_names or it's not a frozenset,
Expand All @@ -2942,46 +2972,52 @@ _Py_DumpExtensionModules(int fd, PyInterpreterState *interp)
int header = 1;
Py_ssize_t count = 0;
pos = 0;
while (PyDict_Next(modules, &pos, &key, &value)) {
if (!PyUnicode_Check(key)) {
continue;
}
if (!_PyModule_IsExtension(value)) {
continue;
}
// Use the module name from the sys.modules key,
// don't attempt to get the module object name.
if (stdlib_module_names != NULL) {
int is_stdlib_ext = 0;

Py_ssize_t i = 0;
PyObject *item;
Py_hash_t hash;
// if stdlib_module_names is not NULL, it is always a frozenset.
while (_PySet_NextEntry(stdlib_module_names, &i, &item, &hash)) {
if (PyUnicode_Check(item)
&& PyUnicode_Compare(key, item) == 0)
{
is_stdlib_ext = 1;
break;
}
if (acquire_dict_lock_for_dump(modules) == 0) {
Comment thread
corona10 marked this conversation as resolved.
Outdated
while (_PyDict_Next(modules, &pos, &key, &value, NULL)) {
if (!PyUnicode_Check(key)) {
continue;
}
if (is_stdlib_ext) {
// Ignore stdlib extension
if (!_PyModule_IsExtension(value)) {
continue;
}
}
// Use the module name from the sys.modules key,
// don't attempt to get the module object name.
if (stdlib_module_names != NULL) {
int is_stdlib_ext = 0;

Py_ssize_t i = 0;
PyObject *item;
Py_hash_t hash;
// if stdlib_module_names is not NULL, it is always a frozenset.
while (_PySet_NextEntry(stdlib_module_names, &i, &item, &hash)) {
if (PyUnicode_Check(item)
&& PyUnicode_Compare(key, item) == 0)
{
is_stdlib_ext = 1;
break;
}
}
if (is_stdlib_ext) {
// Ignore stdlib extension
continue;
}
}

if (header) {
PUTS(fd, "\nExtension modules: ");
header = 0;
}
else {
PUTS(fd, ", ");
}
if (header) {
PUTS(fd, "\nExtension modules: ");
header = 0;
}
else {
PUTS(fd, ", ");
}

_Py_DumpASCII(fd, key);
count++;
_Py_DumpASCII(fd, key);
count++;
}
release_dict_lock_for_dump(modules);
}
else {
return;
}

if (count) {
Expand Down