From 2199f40a7d57b8ee131996201ad8f5c8032e6301 Mon Sep 17 00:00:00 2001 From: Pablo Galindo Date: Fri, 20 Oct 2017 04:44:37 -0400 Subject: [PATCH 01/14] Explain real behaviour of sys.settrace and sys.setprofile --- Doc/library/sys.rst | 57 +++++++++++++++++++++++++++++++++++---------- 1 file changed, 45 insertions(+), 12 deletions(-) diff --git a/Doc/library/sys.rst b/Doc/library/sys.rst index 853e91b93de5d5..31030b06cf87c5 100644 --- a/Doc/library/sys.rst +++ b/Doc/library/sys.rst @@ -1060,6 +1060,49 @@ always available. so it does not make sense to use this in the presence of multiple threads. Also, its return value is not used, so it can simply return ``None``. + Profile functions should have three arguments: *frame*, *event*, and + *arg*. *frame* is the current stack frame. *event* is a string: ``'call'``, + ``'line'``, ``'return'``, ``'exception'``, ``'c_call'``, ``'c_return'``, or + ``'c_exception'``, ``'opcode'``. *arg* depends on the event type. + + The events have the following meaning: + + ``'call'`` + A function is called (or some other code block entered). The + global trace function is called; *arg* is ``None``; the return value + specifies the local trace function. + + ``'return'`` + A function (or other code block) is about to return. The local trace + function is called; *arg* is the value that will be returned, or ``None`` + if the event is caused by an exception being raised. The trace function's + return value is ignored. + + ``'exception'`` + An exception has occurred. The local trace function is called; *arg* is a + tuple ``(exception, value, traceback)``; the return value specifies the + new local trace function. + + ``'c_call'`` + A C function is about to be called. This may be an extension function or + a built-in. *arg* is the C function object. + + ``'c_return'`` + A C function has returned. *arg* is the C function object. + + ``'c_exception'`` + A C function has raised an exception. *arg* is the C function object. + + ``'opcode'`` + The interpreter is about to execute a new opcode (see :mod:`dis` for + opcode details). The local trace function is called; *arg* is + ``None``; the return value specifies the new local trace function. + Per-opcode events are not emitted by default: they must be explicitly + requested by setting :attr:`f_trace_opcodes` to :const:`True` on the + frame. + + Note that as an exception is propagated down the chain of callers, an + ``'exception'`` event is generated at each level. .. function:: setrecursionlimit(limit) @@ -1106,8 +1149,8 @@ always available. Trace functions should have three arguments: *frame*, *event*, and *arg*. *frame* is the current stack frame. *event* is a string: ``'call'``, - ``'line'``, ``'return'``, ``'exception'``, ``'c_call'``, ``'c_return'``, or - ``'c_exception'``, ``'opcode'``. *arg* depends on the event type. + ``'line'``, ``'return'``, ``'exception'`` or ``'opcode'``. *arg* depends on + the event type. The trace function is invoked (with *event* set to ``'call'``) whenever a new local scope is entered; it should return a reference to a local trace @@ -1144,16 +1187,6 @@ always available. tuple ``(exception, value, traceback)``; the return value specifies the new local trace function. - ``'c_call'`` - A C function is about to be called. This may be an extension function or - a built-in. *arg* is the C function object. - - ``'c_return'`` - A C function has returned. *arg* is the C function object. - - ``'c_exception'`` - A C function has raised an exception. *arg* is the C function object. - ``'opcode'`` The interpreter is about to execute a new opcode (see :mod:`dis` for opcode details). The local trace function is called; *arg* is From 62918e4ddfbe3bddd0f884e25bdcb45117600782 Mon Sep 17 00:00:00 2001 From: Pablo Galindo Date: Mon, 22 Jan 2018 21:12:35 +0000 Subject: [PATCH 02/14] Add documentation in the c-api section --- Doc/c-api/init.rst | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/Doc/c-api/init.rst b/Doc/c-api/init.rst index 7792058683da0c..017039ca08cf7d 100644 --- a/Doc/c-api/init.rst +++ b/Doc/c-api/init.rst @@ -1091,6 +1091,10 @@ Python-level trace functions in previous versions. | :const:`PyTrace_C_RETURN` | Function object being called. | +------------------------------+--------------------------------------+ + Any trace function registered using :c:func:`PyEval_SetTrace` will not recieve + :const:`PyTrace_C_CALL`, :const:`PyTrace_C_EXCEPTION`, :const:`PyTrace_C_RETURN` + or :const:`PyTrace_LINE` as a value for the *what* parameter. + .. c:var:: int PyTrace_CALL @@ -1156,7 +1160,7 @@ Python-level trace functions in previous versions. Set the tracing function to *func*. This is similar to :c:func:`PyEval_SetProfile`, except the tracing function does receive line-number - events. + events or any event related to C function objects being called. .. _advanced-debugging: From 778c0983aa240a8d056c0beb5ed70330996367bd Mon Sep 17 00:00:00 2001 From: Pablo Galindo Date: Mon, 22 Jan 2018 21:14:05 +0000 Subject: [PATCH 03/14] Add News entry --- .../next/Documentation/2018-01-22-21-13-46.bpo-17799.rdZ-Vk.rst | 2 ++ 1 file changed, 2 insertions(+) create mode 100644 Misc/NEWS.d/next/Documentation/2018-01-22-21-13-46.bpo-17799.rdZ-Vk.rst diff --git a/Misc/NEWS.d/next/Documentation/2018-01-22-21-13-46.bpo-17799.rdZ-Vk.rst b/Misc/NEWS.d/next/Documentation/2018-01-22-21-13-46.bpo-17799.rdZ-Vk.rst new file mode 100644 index 00000000000000..5f9e1eb6a6da0b --- /dev/null +++ b/Misc/NEWS.d/next/Documentation/2018-01-22-21-13-46.bpo-17799.rdZ-Vk.rst @@ -0,0 +1,2 @@ +Explain real behaviour of sys.settrace regarding C function object calls. +Patch by Pablo Galindo Salgado. From 18b88ec7bc97e049c7caa43b020782e2c4b51c60 Mon Sep 17 00:00:00 2001 From: Pablo Galindo Date: Mon, 22 Jan 2018 21:49:47 +0000 Subject: [PATCH 04/14] Correct typo --- Doc/c-api/init.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Doc/c-api/init.rst b/Doc/c-api/init.rst index 017039ca08cf7d..edf1da9ab863c1 100644 --- a/Doc/c-api/init.rst +++ b/Doc/c-api/init.rst @@ -1091,7 +1091,7 @@ Python-level trace functions in previous versions. | :const:`PyTrace_C_RETURN` | Function object being called. | +------------------------------+--------------------------------------+ - Any trace function registered using :c:func:`PyEval_SetTrace` will not recieve + Any trace function registered using :c:func:`PyEval_SetTrace` will not receive :const:`PyTrace_C_CALL`, :const:`PyTrace_C_EXCEPTION`, :const:`PyTrace_C_RETURN` or :const:`PyTrace_LINE` as a value for the *what* parameter. From a9f381f2deffbed4c01a3a5752c46cd206b058b4 Mon Sep 17 00:00:00 2001 From: Pablo Galindo Date: Tue, 23 Jan 2018 16:05:14 +0000 Subject: [PATCH 05/14] Update init.rst --- Doc/c-api/init.rst | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/Doc/c-api/init.rst b/Doc/c-api/init.rst index edf1da9ab863c1..ad79507f906a08 100644 --- a/Doc/c-api/init.rst +++ b/Doc/c-api/init.rst @@ -1092,8 +1092,8 @@ Python-level trace functions in previous versions. +------------------------------+--------------------------------------+ Any trace function registered using :c:func:`PyEval_SetTrace` will not receive - :const:`PyTrace_C_CALL`, :const:`PyTrace_C_EXCEPTION`, :const:`PyTrace_C_RETURN` - or :const:`PyTrace_LINE` as a value for the *what* parameter. + :const:`PyTrace_C_CALL`, :const:`PyTrace_C_EXCEPTION` or :const:`PyTrace_C_RETURN` + as a value for the *what* parameter. .. c:var:: int PyTrace_CALL @@ -1160,7 +1160,7 @@ Python-level trace functions in previous versions. Set the tracing function to *func*. This is similar to :c:func:`PyEval_SetProfile`, except the tracing function does receive line-number - events or any event related to C function objects being called. + events and does not receive any event related to C function objects being called. .. _advanced-debugging: From 3887c6cdd8655d835c860b33abdf8e6b8338525a Mon Sep 17 00:00:00 2001 From: Pablo Galindo Date: Tue, 23 Jan 2018 16:12:11 +0000 Subject: [PATCH 06/14] Update sys.rst --- Doc/library/sys.rst | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/Doc/library/sys.rst b/Doc/library/sys.rst index 31030b06cf87c5..cea361f1847820 100644 --- a/Doc/library/sys.rst +++ b/Doc/library/sys.rst @@ -1062,15 +1062,14 @@ always available. Profile functions should have three arguments: *frame*, *event*, and *arg*. *frame* is the current stack frame. *event* is a string: ``'call'``, - ``'line'``, ``'return'``, ``'exception'``, ``'c_call'``, ``'c_return'``, or + ``'return'``, ``'exception'``, ``'c_call'``, ``'c_return'``, or ``'c_exception'``, ``'opcode'``. *arg* depends on the event type. The events have the following meaning: ``'call'`` A function is called (or some other code block entered). The - global trace function is called; *arg* is ``None``; the return value - specifies the local trace function. + profile function is called; *arg* is ``None``. ``'return'`` A function (or other code block) is about to return. The local trace From 076a702c08613e4bcbf837829999d6d4e153b0e2 Mon Sep 17 00:00:00 2001 From: Pablo Galindo Date: Tue, 23 Jan 2018 16:48:54 +0000 Subject: [PATCH 07/14] Remove local/global references in setprofile. Remove events that are not being received. --- Doc/library/sys.rst | 20 +++----------------- 1 file changed, 3 insertions(+), 17 deletions(-) diff --git a/Doc/library/sys.rst b/Doc/library/sys.rst index cea361f1847820..59f0ff5372b40f 100644 --- a/Doc/library/sys.rst +++ b/Doc/library/sys.rst @@ -1063,7 +1063,7 @@ always available. Profile functions should have three arguments: *frame*, *event*, and *arg*. *frame* is the current stack frame. *event* is a string: ``'call'``, ``'return'``, ``'exception'``, ``'c_call'``, ``'c_return'``, or - ``'c_exception'``, ``'opcode'``. *arg* depends on the event type. + ``'c_exception'``. *arg* depends on the event type. The events have the following meaning: @@ -1072,15 +1072,9 @@ always available. profile function is called; *arg* is ``None``. ``'return'`` - A function (or other code block) is about to return. The local trace + A function (or other code block) is about to return. The profile function is called; *arg* is the value that will be returned, or ``None`` - if the event is caused by an exception being raised. The trace function's - return value is ignored. - - ``'exception'`` - An exception has occurred. The local trace function is called; *arg* is a - tuple ``(exception, value, traceback)``; the return value specifies the - new local trace function. + if the event is caused by an exception being raised. ``'c_call'`` A C function is about to be called. This may be an extension function or @@ -1092,14 +1086,6 @@ always available. ``'c_exception'`` A C function has raised an exception. *arg* is the C function object. - ``'opcode'`` - The interpreter is about to execute a new opcode (see :mod:`dis` for - opcode details). The local trace function is called; *arg* is - ``None``; the return value specifies the new local trace function. - Per-opcode events are not emitted by default: they must be explicitly - requested by setting :attr:`f_trace_opcodes` to :const:`True` on the - frame. - Note that as an exception is propagated down the chain of callers, an ``'exception'`` event is generated at each level. From 3aa12670ebe5f9164c6e0389ac0f9f0d521722d0 Mon Sep 17 00:00:00 2001 From: Pablo Galindo Date: Tue, 23 Jan 2018 16:49:58 +0000 Subject: [PATCH 08/14] Merge sections about trace functions --- Doc/c-api/init.rst | 11 ++++------- 1 file changed, 4 insertions(+), 7 deletions(-) diff --git a/Doc/c-api/init.rst b/Doc/c-api/init.rst index ad79507f906a08..fcd6a65a458c83 100644 --- a/Doc/c-api/init.rst +++ b/Doc/c-api/init.rst @@ -1091,11 +1091,6 @@ Python-level trace functions in previous versions. | :const:`PyTrace_C_RETURN` | Function object being called. | +------------------------------+--------------------------------------+ - Any trace function registered using :c:func:`PyEval_SetTrace` will not receive - :const:`PyTrace_C_CALL`, :const:`PyTrace_C_EXCEPTION` or :const:`PyTrace_C_RETURN` - as a value for the *what* parameter. - - .. c:var:: int PyTrace_CALL The value of the *what* parameter to a :c:type:`Py_tracefunc` function when a new @@ -1160,8 +1155,10 @@ Python-level trace functions in previous versions. Set the tracing function to *func*. This is similar to :c:func:`PyEval_SetProfile`, except the tracing function does receive line-number - events and does not receive any event related to C function objects being called. - + events and does not receive any event related to C function objects being called.Any + trace function registered using :c:func:`PyEval_SetTrace` will not receive + :const:`PyTrace_C_CALL`, :const:`PyTrace_C_EXCEPTION` or :const:`PyTrace_C_RETURN` + as a value for the *what* parameter. .. _advanced-debugging: From d0dce9798681171dfb433aeb12cce76aeb7a9963 Mon Sep 17 00:00:00 2001 From: Pablo Galindo Date: Tue, 23 Jan 2018 16:57:36 +0000 Subject: [PATCH 09/14] Corrected typo in PyEval_SetTrace section --- Doc/c-api/init.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Doc/c-api/init.rst b/Doc/c-api/init.rst index fcd6a65a458c83..0c7074af078c59 100644 --- a/Doc/c-api/init.rst +++ b/Doc/c-api/init.rst @@ -1155,7 +1155,7 @@ Python-level trace functions in previous versions. Set the tracing function to *func*. This is similar to :c:func:`PyEval_SetProfile`, except the tracing function does receive line-number - events and does not receive any event related to C function objects being called.Any + events and does not receive any event related to C function objects being called. Any trace function registered using :c:func:`PyEval_SetTrace` will not receive :const:`PyTrace_C_CALL`, :const:`PyTrace_C_EXCEPTION` or :const:`PyTrace_C_RETURN` as a value for the *what* parameter. From 9301d5666bf201ad8a543527cf59338507716439 Mon Sep 17 00:00:00 2001 From: Pablo Galindo Date: Tue, 23 Jan 2018 16:58:40 +0000 Subject: [PATCH 10/14] Remove exception event from profile paragraph --- Doc/library/sys.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/Doc/library/sys.rst b/Doc/library/sys.rst index 59f0ff5372b40f..01cd0c8630ffa0 100644 --- a/Doc/library/sys.rst +++ b/Doc/library/sys.rst @@ -1062,8 +1062,8 @@ always available. Profile functions should have three arguments: *frame*, *event*, and *arg*. *frame* is the current stack frame. *event* is a string: ``'call'``, - ``'return'``, ``'exception'``, ``'c_call'``, ``'c_return'``, or - ``'c_exception'``. *arg* depends on the event type. + ``'return'``, ``'c_call'``, ``'c_return'``, or ``'c_exception'``. *arg* depends + on the event type. The events have the following meaning: From b4531a91bde825d6c8ab4845d6eba66c87af4df2 Mon Sep 17 00:00:00 2001 From: Pablo Galindo Date: Tue, 23 Jan 2018 17:27:00 +0000 Subject: [PATCH 11/14] Update PyEval_SetProfile to indicate that exception events are not received. --- Doc/c-api/init.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/Doc/c-api/init.rst b/Doc/c-api/init.rst index 0c7074af078c59..954f98b9cabd78 100644 --- a/Doc/c-api/init.rst +++ b/Doc/c-api/init.rst @@ -1147,8 +1147,8 @@ Python-level trace functions in previous versions. function as its first parameter, and may be any Python object, or *NULL*. If the profile function needs to maintain state, using a different value for *obj* for each thread provides a convenient and thread-safe place to store it. The - profile function is called for all monitored events except the line-number - events. + profile function is called for all monitored events except :const:`PyTrace_LINE` + and :const:`PyTrace_EXCEPTION`. .. c:function:: void PyEval_SetTrace(Py_tracefunc func, PyObject *obj) From 72ca9cb57c975208fcc206da2d58ccc9217ac250 Mon Sep 17 00:00:00 2001 From: Pablo Galindo Date: Tue, 23 Jan 2018 17:27:49 +0000 Subject: [PATCH 12/14] Remove line about exception events in the setprofile section. --- Doc/library/sys.rst | 3 --- 1 file changed, 3 deletions(-) diff --git a/Doc/library/sys.rst b/Doc/library/sys.rst index 01cd0c8630ffa0..61b8c6a16aacd5 100644 --- a/Doc/library/sys.rst +++ b/Doc/library/sys.rst @@ -1086,9 +1086,6 @@ always available. ``'c_exception'`` A C function has raised an exception. *arg* is the C function object. - Note that as an exception is propagated down the chain of callers, an - ``'exception'`` event is generated at each level. - .. function:: setrecursionlimit(limit) Set the maximum depth of the Python interpreter stack to *limit*. This limit From 3061991e45a17cc866d0ed3bbe8a3959e856a816 Mon Sep 17 00:00:00 2001 From: Pablo Galindo Date: Wed, 24 Jan 2018 10:19:16 +0000 Subject: [PATCH 13/14] Improve description of profile function --- Doc/library/sys.rst | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/Doc/library/sys.rst b/Doc/library/sys.rst index 61b8c6a16aacd5..748a93b7e582d7 100644 --- a/Doc/library/sys.rst +++ b/Doc/library/sys.rst @@ -1053,11 +1053,11 @@ always available. Set the system's profile function, which allows you to implement a Python source code profiler in Python. See chapter :ref:`profile` for more information on the Python profiler. The system's profile function is called similarly to the - system's trace function (see :func:`settrace`), but it isn't called for each - executed line of code (only on call and return, but the return event is reported - even when an exception has been set). The function is thread-specific, but - there is no way for the profiler to know about context switches between threads, - so it does not make sense to use this in the presence of multiple threads. Also, + system's trace function (see :func:`settrace`), but it is called with different events, + for example it isn't called for each executed line of code (only on call and return, + but the return event is reported even when an exception has been set). The function is + thread-specific, but there is no way for the profiler to know about context switches between + threads, so it does not make sense to use this in the presence of multiple threads. Also, its return value is not used, so it can simply return ``None``. Profile functions should have three arguments: *frame*, *event*, and From 426368c149da76ab75681821b65903d2b25e7ced Mon Sep 17 00:00:00 2001 From: Pablo Galindo Date: Wed, 24 Jan 2018 10:21:48 +0000 Subject: [PATCH 14/14] Update NEWS entry --- .../Documentation/2018-01-22-21-13-46.bpo-17799.rdZ-Vk.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/Misc/NEWS.d/next/Documentation/2018-01-22-21-13-46.bpo-17799.rdZ-Vk.rst b/Misc/NEWS.d/next/Documentation/2018-01-22-21-13-46.bpo-17799.rdZ-Vk.rst index 5f9e1eb6a6da0b..ccc52f60eeedac 100644 --- a/Misc/NEWS.d/next/Documentation/2018-01-22-21-13-46.bpo-17799.rdZ-Vk.rst +++ b/Misc/NEWS.d/next/Documentation/2018-01-22-21-13-46.bpo-17799.rdZ-Vk.rst @@ -1,2 +1,2 @@ -Explain real behaviour of sys.settrace regarding C function object calls. -Patch by Pablo Galindo Salgado. +Explain real behaviour of sys.settrace and sys.setprofile and their C-API counterparts +regarding which type of events are received in each function. Patch by Pablo Galindo Salgado.