Skip to content

gh-91048: Refactor and optimize remote debugging module #134652

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
pablogsal merged 20 commits into from
May 25, 2025
Merged

gh-91048: Refactor and optimize remote debugging module #134652

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
20 commits
Select commit Hold shift + click to select a range
38f9ad0
gh-91048: Refactor and optimize remote debugging module
pablogsal May 25, 2025
a6fed05
Fix free threading
pablogsal May 25, 2025
8c01234
Fix lint
pablogsal May 25, 2025
1130f27
Simplify stuff and fix free threading caching
pablogsal May 25, 2025
e112a4a
Fix clinic
pablogsal May 25, 2025
559d44f
fix windows
pablogsal May 25, 2025
5b4d67d
Fix arm
pablogsal May 25, 2025
9033b20
Fix lint
pablogsal May 25, 2025
db35f6e
Small fix
pablogsal May 25, 2025
1773d78
Simplofy cleanup
pablogsal May 25, 2025
e2ad532
Small fix
pablogsal May 25, 2025
eb61f38
Fix race in test (two threads need two ready markers!)
ambv May 25, 2025
5da486c
Fix windows
pablogsal May 25, 2025
d6d0c7e
Fix windows
pablogsal May 25, 2025
5bdfec1
Fix windows
pablogsal May 25, 2025
f463436
Regen files
pablogsal May 25, 2025
416d96b
MOAR WINDOWS
pablogsal May 25, 2025
681ab63
Regen morefiles
pablogsal May 25, 2025
14a5f58
More windows stuff
pablogsal May 25, 2025
89311c9
Regen frozenmain
ambv May 25, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Fix windows
  • Loading branch information
@pablogsal
pablogsal committed May 25, 2025
commit 5da486ccda56314171a0cb6eea91dd73d3f85d58
34 changes: 21 additions & 13 deletions Modules/_remote_debugging_module.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -41,29 +41,37 @@
#define GET_MEMBER(type, obj, offset) (*(type*)((char*)(obj) + (offset)))

/* Size macros for opaque buffers */
#define SIZEOF_UNICODE_OBJ sizeof(PyUnicodeObject)
#define SIZEOF_BYTES_OBJ sizeof(PyBytesObject)
#define SIZEOF_INTERP_FRAME sizeof(_PyInterpreterFrame)
#define SIZEOF_CODE_OBJ sizeof(PyCodeObject)
#define SIZEOF_GEN_OBJ sizeof(PyGenObject)
#define SIZEOF_INTERP_FRAME sizeof(_PyInterpreterFrame)
#define SIZEOF_LLIST_NODE sizeof(struct llist_node)
#define SIZEOF_PAGE_CACHE_ENTRY sizeof(page_cache_entry_t)
#define SIZEOF_PYOBJECT sizeof(PyObject)
#define SIZEOF_SET_OBJ sizeof(PySetObject)
#define SIZEOF_TYPE_OBJ sizeof(PyTypeObject)
#define SIZEOF_TASK_OBJ 4096
#define SIZEOF_PYOBJECT sizeof(PyObject)
#define SIZEOF_LLIST_NODE sizeof(struct llist_node)
#define SIZEOF_THREAD_STATE sizeof(PyThreadState)
#define SIZEOF_CODE_OBJ sizeof(PyCodeObject)
#define SIZEOF_TYPE_OBJ sizeof(PyTypeObject)
#define SIZEOF_UNICODE_OBJ sizeof(PyUnicodeObject)

// Calculate the minimum buffer size needed to read interpreter state fields
// We need to read code_object_generation and potentially tlbc_generation
#ifndef MAX
#define _MAX(a, b) ((a) > (b) ? (a) : (b))
#else
#define _MAX(a, b) MAX(a, b)
#endif

#ifdef Py_GIL_DISABLED
#define INTERP_STATE_MIN_SIZE MAX(MAX(offsetof(PyInterpreterState, _code_object_generation) + sizeof(uint64_t), \
#define INTERP_STATE_MIN_SIZE _MAX(_MAX(offsetof(PyInterpreterState, _code_object_generation) + sizeof(uint64_t), \
offsetof(PyInterpreterState, tlbc_indices.tlbc_generation) + sizeof(uint32_t)), \
offsetof(PyInterpreterState, threads.head) + sizeof(void*))
#else
#define INTERP_STATE_MIN_SIZE MAX(offsetof(PyInterpreterState, _code_object_generation) + sizeof(uint64_t), \
#define INTERP_STATE_MIN_SIZE _MAX(offsetof(PyInterpreterState, _code_object_generation) + sizeof(uint64_t), \
offsetof(PyInterpreterState, threads.head) + sizeof(void*))
#endif
#define INTERP_STATE_BUFFER_SIZE MAX(INTERP_STATE_MIN_SIZE, 256)
#define INTERP_STATE_BUFFER_SIZE _MAX(INTERP_STATE_MIN_SIZE, 256)

Check failure on line 73 in Modules/_remote_debugging_module.c

View workflow job for this annotation

GitHub Actions / Hypothesis tests on Ubuntu 

implicit declaration of function ‘_MAX’; did you mean ‘MAX’? [-Werror=implicit-function-declaration]

Check failure on line 73 in Modules/_remote_debugging_module.c

View workflow job for this annotation

GitHub Actions / Ubuntu (free-threading) / build and test (ubuntu-24.04) 

implicit declaration of function ‘_MAX’; did you mean ‘MAX’? [-Werror=implicit-function-declaration]

Check failure on line 73 in Modules/_remote_debugging_module.c

View workflow job for this annotation

GitHub Actions / Ubuntu / build and test (ubuntu-24.04) 

implicit declaration of function ‘_MAX’; did you mean ‘MAX’? [-Werror=implicit-function-declaration]

Check failure on line 73 in Modules/_remote_debugging_module.c

View workflow job for this annotation

GitHub Actions / Cross build Linux 

implicit declaration of function ‘_MAX’; did you mean ‘MAX’? [-Werror=implicit-function-declaration]

Check failure on line 73 in Modules/_remote_debugging_module.c

View workflow job for this annotation

GitHub Actions / Address sanitizer (ubuntu-24.04) 

implicit declaration of function ‘_MAX’; did you mean ‘MAX’? [-Werror=implicit-function-declaration]

Check failure on line 73 in Modules/_remote_debugging_module.c

View workflow job for this annotation

GitHub Actions / Ubuntu / build and test (ubuntu-24.04-arm) 

implicit declaration of function ‘_MAX’; did you mean ‘MAX’? [-Werror=implicit-function-declaration]

Check failure on line 73 in Modules/_remote_debugging_module.c

View workflow job for this annotation

GitHub Actions / Ubuntu (bolt) / build and test (ubuntu-24.04) 

implicit declaration of function ‘_MAX’; did you mean ‘MAX’? [-Werror=implicit-function-declaration]
#undef _MAX



Expand Down Expand Up @@ -392,7 +400,7 @@
unsigned int shift = PYLONG_BITS_IN_DIGIT;

// Read the entire PyLongObject at once
char long_obj[unwinder->debug_offsets.long_object.size];
char long_obj[SIZEOF_LONG_OBJ];

Check failure on line 403 in Modules/_remote_debugging_module.c

View workflow job for this annotation

GitHub Actions / Hypothesis tests on Ubuntu 

‘SIZEOF_LONG_OBJ’ undeclared (first use in this function); did you mean ‘SIZEOF_LONG_LONG’?

Check warning on line 403 in Modules/_remote_debugging_module.c

View workflow job for this annotation

GitHub Actions / Hypothesis tests on Ubuntu 

unused variable ‘long_obj’ [-Wunused-variable]

Check failure on line 403 in Modules/_remote_debugging_module.c

View workflow job for this annotation

GitHub Actions / Ubuntu (free-threading) / build and test (ubuntu-24.04) 

‘SIZEOF_LONG_OBJ’ undeclared (first use in this function); did you mean ‘SIZEOF_LONG_LONG’?

Check warning on line 403 in Modules/_remote_debugging_module.c

View workflow job for this annotation

GitHub Actions / Ubuntu (free-threading) / build and test (ubuntu-24.04) 

unused variable ‘long_obj’ [-Wunused-variable]

Check failure on line 403 in Modules/_remote_debugging_module.c

View workflow job for this annotation

GitHub Actions / Ubuntu / build and test (ubuntu-24.04) 

‘SIZEOF_LONG_OBJ’ undeclared (first use in this function); did you mean ‘SIZEOF_LONG_LONG’?

Check warning on line 403 in Modules/_remote_debugging_module.c

View workflow job for this annotation

GitHub Actions / Ubuntu / build and test (ubuntu-24.04) 

unused variable ‘long_obj’ [-Wunused-variable]

Check failure on line 403 in Modules/_remote_debugging_module.c

View workflow job for this annotation

GitHub Actions / Cross build Linux 

‘SIZEOF_LONG_OBJ’ undeclared (first use in this function); did you mean ‘SIZEOF_LONG_LONG’?

Check warning on line 403 in Modules/_remote_debugging_module.c

View workflow job for this annotation

GitHub Actions / Cross build Linux 

unused variable ‘long_obj’ [-Wunused-variable]

Check failure on line 403 in Modules/_remote_debugging_module.c

View workflow job for this annotation

GitHub Actions / Address sanitizer (ubuntu-24.04) 

‘SIZEOF_LONG_OBJ’ undeclared (first use in this function); did you mean ‘SIZEOF_LONG_LONG’?

Check warning on line 403 in Modules/_remote_debugging_module.c

View workflow job for this annotation

GitHub Actions / Address sanitizer (ubuntu-24.04) 

unused variable ‘long_obj’ [-Wunused-variable]

Check failure on line 403 in Modules/_remote_debugging_module.c

View workflow job for this annotation

GitHub Actions / Windows / Build and test (x64) 

'SIZEOF_LONG_OBJ': undeclared identifier [D:\a\cpython\cpython\PCbuild\_remote_debugging.vcxproj]

Check failure on line 403 in Modules/_remote_debugging_module.c

View workflow job for this annotation

GitHub Actions / Windows / Build and test (x64) 

expected constant expression [D:\a\cpython\cpython\PCbuild\_remote_debugging.vcxproj]

Check failure on line 403 in Modules/_remote_debugging_module.c

View workflow job for this annotation

GitHub Actions / Windows / Build and test (x64) 

cannot allocate an array of constant size 0 [D:\a\cpython\cpython\PCbuild\_remote_debugging.vcxproj]

Check failure on line 403 in Modules/_remote_debugging_module.c

View workflow job for this annotation

GitHub Actions / Windows / Build and test (x64) 

'long_obj': unknown size [D:\a\cpython\cpython\PCbuild\_remote_debugging.vcxproj]

Check failure on line 403 in Modules/_remote_debugging_module.c

View workflow job for this annotation

GitHub Actions / Windows (free-threading) / Build and test (x64) 

'SIZEOF_LONG_OBJ': undeclared identifier [D:\a\cpython\cpython\PCbuild\_remote_debugging.vcxproj]

Check failure on line 403 in Modules/_remote_debugging_module.c

View workflow job for this annotation

GitHub Actions / Windows (free-threading) / Build and test (x64) 

expected constant expression [D:\a\cpython\cpython\PCbuild\_remote_debugging.vcxproj]

Check failure on line 403 in Modules/_remote_debugging_module.c

View workflow job for this annotation

GitHub Actions / Windows (free-threading) / Build and test (x64) 

cannot allocate an array of constant size 0 [D:\a\cpython\cpython\PCbuild\_remote_debugging.vcxproj]

Check failure on line 403 in Modules/_remote_debugging_module.c

View workflow job for this annotation

GitHub Actions / Windows (free-threading) / Build and test (x64) 

'long_obj': unknown size [D:\a\cpython\cpython\PCbuild\_remote_debugging.vcxproj]

Check failure on line 403 in Modules/_remote_debugging_module.c

View workflow job for this annotation

GitHub Actions / Ubuntu / build and test (ubuntu-24.04-arm) 

‘SIZEOF_LONG_OBJ’ undeclared (first use in this function); did you mean ‘SIZEOF_LONG_LONG’?

Check warning on line 403 in Modules/_remote_debugging_module.c

View workflow job for this annotation

GitHub Actions / Ubuntu / build and test (ubuntu-24.04-arm) 

unused variable ‘long_obj’ [-Wunused-variable]

Check failure on line 403 in Modules/_remote_debugging_module.c

View workflow job for this annotation

GitHub Actions / Ubuntu (bolt) / build and test (ubuntu-24.04) 

‘SIZEOF_LONG_OBJ’ undeclared (first use in this function); did you mean ‘SIZEOF_LONG_LONG’?

Check warning on line 403 in Modules/_remote_debugging_module.c

View workflow job for this annotation

GitHub Actions / Ubuntu (bolt) / build and test (ubuntu-24.04) 

unused variable ‘long_obj’ [-Wunused-variable]
int bytes_read = _Py_RemoteDebug_PagedReadRemoteMemory(
&unwinder->handle,
address,
Expand Down Expand Up @@ -531,7 +539,7 @@
uintptr_t task_address
) {
// Read the entire TaskObj at once
char task_obj[unwinder->async_debug_offsets.asyncio_task_object.size];
char task_obj[SIZEOF_TASK_OBJ];
int err = _Py_RemoteDebug_PagedReadRemoteMemory(
&unwinder->handle,
task_address,
Expand Down Expand Up @@ -594,7 +602,7 @@
int recurse_task
) {
// Read the entire TaskObj at once
char task_obj[unwinder->async_debug_offsets.asyncio_task_object.size];
char task_obj[SIZEOF_TASK_OBJ];
if (_Py_RemoteDebug_PagedReadRemoteMemory(&unwinder->handle, task_address,
unwinder->async_debug_offsets.asyncio_task_object.size,
task_obj) < 0) {
Expand Down Expand Up @@ -745,7 +753,7 @@
PyObject* result = NULL;
PyObject *call_stack = NULL;
PyObject *tn = NULL;
char task_obj[unwinder->async_debug_offsets.asyncio_task_object.size];
char task_obj[SIZEOF_TASK_OBJ];
uintptr_t coro_addr;

result = PyList_New(0);
Expand Down Expand Up @@ -2302,7 +2310,7 @@
{
PyObject* result = NULL;
// Read interpreter state into opaque buffer
char interp_state_buffer[INTERP_STATE_BUFFER_SIZE];

Check warning on line 2313 in Modules/_remote_debugging_module.c

View workflow job for this annotation

GitHub Actions / Windows / Build and test (x64) 

'_MAX' undefined; assuming extern returning int [D:\a\cpython\cpython\PCbuild\_remote_debugging.vcxproj]

Check failure on line 2313 in Modules/_remote_debugging_module.c

View workflow job for this annotation

GitHub Actions / Windows / Build and test (x64) 

expected constant expression [D:\a\cpython\cpython\PCbuild\_remote_debugging.vcxproj]

Check failure on line 2313 in Modules/_remote_debugging_module.c

View workflow job for this annotation

GitHub Actions / Windows / Build and test (x64) 

cannot allocate an array of constant size 0 [D:\a\cpython\cpython\PCbuild\_remote_debugging.vcxproj]

Check failure on line 2313 in Modules/_remote_debugging_module.c

View workflow job for this annotation

GitHub Actions / Windows / Build and test (x64) 

'interp_state_buffer': unknown size [D:\a\cpython\cpython\PCbuild\_remote_debugging.vcxproj]

Check warning on line 2313 in Modules/_remote_debugging_module.c

View workflow job for this annotation

GitHub Actions / Windows (free-threading) / Build and test (x64) 

'_MAX' undefined; assuming extern returning int [D:\a\cpython\cpython\PCbuild\_remote_debugging.vcxproj]

Check failure on line 2313 in Modules/_remote_debugging_module.c

View workflow job for this annotation

GitHub Actions / Windows (free-threading) / Build and test (x64) 

expected constant expression [D:\a\cpython\cpython\PCbuild\_remote_debugging.vcxproj]

Check failure on line 2313 in Modules/_remote_debugging_module.c

View workflow job for this annotation

GitHub Actions / Windows (free-threading) / Build and test (x64) 

cannot allocate an array of constant size 0 [D:\a\cpython\cpython\PCbuild\_remote_debugging.vcxproj]

Check failure on line 2313 in Modules/_remote_debugging_module.c

View workflow job for this annotation

GitHub Actions / Windows (free-threading) / Build and test (x64) 

'interp_state_buffer': unknown size [D:\a\cpython\cpython\PCbuild\_remote_debugging.vcxproj]
if (_Py_RemoteDebug_PagedReadRemoteMemory(
&self->handle,
self->interpreter_addr,
Expand Down
2 changes: 1 addition & 1 deletion Python/remote_debug.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -106,7 +106,7 @@ typedef struct {
HANDLE hProcess;
#endif
page_cache_entry_t pages[MAX_PAGES];
int page_size;
Py_ssize_t page_size;
} proc_handle_t;

static void
Expand Down
Loading