Skip to content

gh-117953: Track Extra Details in Global Extensions Cache #118532

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
ericsnowcurrently merged 24 commits into from
May 4, 2024
+578 −138
Merged

gh-117953: Track Extra Details in Global Extensions Cache #118532

Changes from 1 commit
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
1fc1149
Store the cached def in a wrapping struct rather than directly.
ericsnowcurrently Apr 30, 2024
254e1f1
Pass the cached value around where appropriate.
ericsnowcurrently May 2, 2024
0694318
Pass the cached value to check_singlephase().
ericsnowcurrently May 1, 2024
9ca330b
Add extensions_cache_value helpers.
ericsnowcurrently Apr 30, 2024
7e0c4ee
Add get_cached_m_dict().
ericsnowcurrently Apr 30, 2024
c1c1379
Track ext module origin.
ericsnowcurrently May 1, 2024
8485fb5
Look up core module dicts in get_cached_m_dict().
ericsnowcurrently May 1, 2024
abbddde
Pass m_dict to _extensions_cache_set().
ericsnowcurrently May 1, 2024
12bcd96
Pass m_init to _extensions_cache_set().
ericsnowcurrently May 1, 2024
2d56ada
Add some comments.
ericsnowcurrently May 2, 2024
9c70d53
Use del_cached_m_dict() in set_cached_m_dict().
ericsnowcurrently May 2, 2024
0098057
Add m_init and m_dict fields to extensions_cache_value.
ericsnowcurrently May 1, 2024
57aaba2
Track which interpreter was used to create m_dict.
ericsnowcurrently May 1, 2024
96932eb
Track the interpreter ID.
ericsnowcurrently May 1, 2024
a4cb134
Look up core module dicts in get_cached_m_dict().
ericsnowcurrently May 1, 2024
c4d3be1
Make sure we only set m_dict for non-isolated interpreters.
ericsnowcurrently May 1, 2024
d87584d
Simplify.
ericsnowcurrently May 3, 2024
336d1da
Fix a comment.
ericsnowcurrently May 3, 2024
bd1e134
Separate the def logic from the rest of the extensions cache code.
ericsnowcurrently May 3, 2024
e42e797
Add the per-interpreter cache index to each global cache entry.
ericsnowcurrently May 3, 2024
faa4254
Fix an assert.
ericsnowcurrently May 4, 2024
5276981
Properly sync the cached m_index and the def m_index.
ericsnowcurrently May 3, 2024
f9a2a05
Merge branch 'main' into extensions-cache-better-tracking
ericsnowcurrently May 4, 2024
7d3dbc1
Fix make smelly.
ericsnowcurrently May 4, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Look up core module dicts in get_cached_m_dict().
  • Loading branch information
@ericsnowcurrently
ericsnowcurrently committed May 3, 2024
commit 8485fb5c24f7cc1b89b2b45e4536d1fab4081466
29 changes: 17 additions & 12 deletions Python/import.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -968,13 +968,24 @@ free_extensions_cache_value(struct extensions_cache_value *value)
PyMem_RawFree(value);
}

static PyObject * get_core_module_dict(
PyInterpreterState *interp, PyObject *name, PyObject *path);

static PyObject *
get_cached_m_dict(struct extensions_cache_value *value)
get_cached_m_dict(struct extensions_cache_value *value,
PyObject *name, PyObject *path)
{
assert(value != NULL);
PyInterpreterState *interp = _PyInterpreterState_GET();
/* It might be a core module (e.g. sys & builtins),
for which we don't cache m_dict. */
if (value->origin == _Py_ext_module_origin_CORE) {
return get_core_module_dict(interp, name, path);
}
assert(value->def != NULL);
Py_XINCREF(value->def->m_base.m_copy);
return value->def->m_base.m_copy;
PyObject *m_dict = value->def->m_base.m_copy;
Py_XINCREF(m_dict);
return m_dict;
}

static int
Expand Down Expand Up @@ -1505,16 +1516,10 @@ reload_singlephase_extension(PyThreadState *tstate,
// XXX Copying the cached dict may break interpreter isolation.
// We could solve this by temporarily acquiring the original
// interpreter's GIL.
PyObject *m_copy = get_cached_m_dict(cached);
PyObject *m_copy = get_cached_m_dict(cached, info->name, info->path);
if (m_copy == NULL) {
/* It might be a core module (e.g. sys & builtins),
for which we don't set m_copy. */
m_copy = get_core_module_dict(
tstate->interp, info->name, info->path);
if (m_copy == NULL) {
assert(!PyErr_Occurred());
return NULL;
}
assert(!PyErr_Occurred());
return NULL;
}
mod = import_add_module(tstate, info->name);
if (mod == NULL) {
Expand Down