Skip to content

Commit cd590a7

Browse files
vstinnervstinner
authored
bpo-1230540: Add threading.excepthook() (pythonGH-13515)
Add a new threading.excepthook() function which handles uncaught Thread.run() exception. It can be overridden to control how uncaught exceptions are handled. threading.ExceptHookArgs is not documented on purpose: it should not be used directly. * threading.excepthook() and threading.ExceptHookArgs. * Add _PyErr_Display(): similar to PyErr_Display(), but accept a 'file' parameter. * Add _thread._excepthook(): C implementation of the exception hook calling _PyErr_Display(). * Add _thread._ExceptHookArgs: structseq type. * Add threading._invoke_excepthook_wrapper() which handles the gory details to ensure that everything remains alive during Python shutdown. * Add unit tests.
1 parent 23b4b69 commit cd590a7

File tree

9 files changed

+424
-67
lines changed

9 files changed

+424
-67
lines changed

‎Doc/library/sys.rst‎

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -298,7 +298,11 @@ always available.
298298
before the program exits. The handling of such top-level exceptions can be
299299
customized by assigning another three-argument function to ``sys.excepthook``.
300300

301-
See also :func:`unraisablehook` which handles unraisable exceptions.
301+
.. seealso::
302+
303+
The :func:`sys.unraisablehook` function handles unraisable exceptions
304+
and the :func:`threading.excepthook` function handles exception raised
305+
by :func:`threading.Thread.run`.
302306

303307

304308
.. data:: __breakpointhook__

‎Doc/library/threading.rst‎

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -38,6 +38,32 @@ This module defines the following functions:
3838
returned.
3939

4040

41+
.. function:: excepthook(args, /)
42+
43+
Handle uncaught exception raised by :func:`Thread.run`.
44+
45+
The *args* argument has the following attributes:
46+
47+
* *exc_type*: Exception type.
48+
* *exc_value*: Exception value, can be ``None``.
49+
* *exc_traceback*: Exception traceback, can be ``None``.
50+
* *thread*: Thread which raised the exception, can be ``None``.
51+
52+
If *exc_type* is :exc:`SystemExit`, the exception is silently ignored.
53+
Otherwise, the exception is printed out on :data:`sys.stderr`.
54+
55+
If this function raises an exception, :func:`sys.excepthook` is called to
56+
handle it.
57+
58+
:func:`threading.excepthook` can be overridden to control how uncaught
59+
exceptions raised by :func:`Thread.run` are handled.
60+
61+
.. seealso::
62+
:func:`sys.excepthook` handles uncaught exceptions.
63+
64+
.. versionadded:: 3.8
65+
66+
4167
.. function:: get_ident()
4268

4369
Return the 'thread identifier' of the current thread. This is a nonzero
@@ -191,6 +217,10 @@ called is terminated.
191217
A thread has a name. The name can be passed to the constructor, and read or
192218
changed through the :attr:`~Thread.name` attribute.
193219

220+
If the :meth:`~Thread.run` method raises an exception,
221+
:func:`threading.excepthook` is called to handle it. By default,
222+
:func:`threading.excepthook` ignores silently :exc:`SystemExit`.
223+
194224
A thread can be flagged as a "daemon thread". The significance of this flag is
195225
that the entire Python program exits when only daemon threads are left. The
196226
initial value is inherited from the creating thread. The flag can be set

‎Doc/whatsnew/3.8.rst‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -623,6 +623,15 @@ in a standardized and extensible format, and offers several other benefits.
623623
(Contributed by C.A.M. Gerlach in :issue:`36268`.)
624624

625625

626+
threading
627+
---------
628+
629+
Add a new :func:`threading.excepthook` function which handles uncaught
630+
:meth:`threading.Thread.run` exception. It can be overridden to control how
631+
uncaught :meth:`threading.Thread.run` exceptions are handled.
632+
(Contributed by Victor Stinner in :issue:`1230540`.)
633+
634+
626635
tokenize
627636
--------
628637

‎Include/internal/pycore_pylifecycle.h‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -107,6 +107,8 @@ PyAPI_FUNC(int) _Py_HandleSystemExit(int *exitcode_p);
107107
PyAPI_FUNC(PyObject*) _PyErr_WriteUnraisableDefaultHook(PyObject *unraisable);
108108

109109
PyAPI_FUNC(void) _PyErr_Print(PyThreadState *tstate);
110+
PyAPI_FUNC(void) _PyErr_Display(PyObject *file, PyObject *exception,
111+
PyObject *value, PyObject *tb);
110112

111113
#ifdef __cplusplus
112114
}

‎Lib/test/test_threading.py‎

Lines changed: 92 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1112,6 +1112,98 @@ def run(self):
11121112
# explicitly break the reference cycle to not leak a dangling thread
11131113
thread.exc = None
11141114

1115+
1116+
class ThreadRunFail(threading.Thread):
1117+
def run(self):
1118+
raise ValueError("run failed")
1119+
1120+
1121+
class ExceptHookTests(BaseTestCase):
1122+
def test_excepthook(self):
1123+
with support.captured_output("stderr") as stderr:
1124+
thread = ThreadRunFail(name="excepthook thread")
1125+
thread.start()
1126+
thread.join()
1127+
1128+
stderr = stderr.getvalue().strip()
1129+
self.assertIn(f'Exception in thread {thread.name}:\n', stderr)
1130+
self.assertIn('Traceback (most recent call last):\n', stderr)
1131+
self.assertIn(' raise ValueError("run failed")', stderr)
1132+
self.assertIn('ValueError: run failed', stderr)
1133+
1134+
@support.cpython_only
1135+
def test_excepthook_thread_None(self):
1136+
# threading.excepthook called with thread=None: log the thread
1137+
# identifier in this case.
1138+
with support.captured_output("stderr") as stderr:
1139+
try:
1140+
raise ValueError("bug")
1141+
except Exception as exc:
1142+
args = threading.ExceptHookArgs([*sys.exc_info(), None])
1143+
threading.excepthook(args)
1144+
1145+
stderr = stderr.getvalue().strip()
1146+
self.assertIn(f'Exception in thread {threading.get_ident()}:\n', stderr)
1147+
self.assertIn('Traceback (most recent call last):\n', stderr)
1148+
self.assertIn(' raise ValueError("bug")', stderr)
1149+
self.assertIn('ValueError: bug', stderr)
1150+
1151+
def test_system_exit(self):
1152+
class ThreadExit(threading.Thread):
1153+
def run(self):
1154+
sys.exit(1)
1155+
1156+
# threading.excepthook() silently ignores SystemExit
1157+
with support.captured_output("stderr") as stderr:
1158+
thread = ThreadExit()
1159+
thread.start()
1160+
thread.join()
1161+
1162+
self.assertEqual(stderr.getvalue(), '')
1163+
1164+
def test_custom_excepthook(self):
1165+
args = None
1166+
1167+
def hook(hook_args):
1168+
nonlocal args
1169+
args = hook_args
1170+
1171+
try:
1172+
with support.swap_attr(threading, 'excepthook', hook):
1173+
thread = ThreadRunFail()
1174+
thread.start()
1175+
thread.join()
1176+
1177+
self.assertEqual(args.exc_type, ValueError)
1178+
self.assertEqual(str(args.exc_value), 'run failed')
1179+
self.assertEqual(args.exc_traceback, args.exc_value.__traceback__)
1180+
self.assertIs(args.thread, thread)
1181+
finally:
1182+
# Break reference cycle
1183+
args = None
1184+
1185+
def test_custom_excepthook_fail(self):
1186+
def threading_hook(args):
1187+
raise ValueError("threading_hook failed")
1188+
1189+
err_str = None
1190+
1191+
def sys_hook(exc_type, exc_value, exc_traceback):
1192+
nonlocal err_str
1193+
err_str = str(exc_value)
1194+
1195+
with support.swap_attr(threading, 'excepthook', threading_hook), \
1196+
support.swap_attr(sys, 'excepthook', sys_hook), \
1197+
support.captured_output('stderr') as stderr:
1198+
thread = ThreadRunFail()
1199+
thread.start()
1200+
thread.join()
1201+
1202+
self.assertEqual(stderr.getvalue(),
1203+
'Exception in threading.excepthook:\n')
1204+
self.assertEqual(err_str, 'threading_hook failed')
1205+
1206+
11151207
class TimerTests(BaseTestCase):
11161208

11171209
def setUp(self):

‎Lib/threading.py‎

Lines changed: 103 additions & 52 deletions
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,6 @@
55
import _thread
66

77
from time import monotonic as _time
8-
from traceback import format_exc as _format_exc
98
from _weakrefset import WeakSet
109
from itertools import islice as _islice, count as _count
1110
try:
@@ -27,7 +26,8 @@
2726
'enumerate', 'main_thread', 'TIMEOUT_MAX',
2827
'Event', 'Lock', 'RLock', 'Semaphore', 'BoundedSemaphore', 'Thread',
2928
'Barrier', 'BrokenBarrierError', 'Timer', 'ThreadError',
30-
'setprofile', 'settrace', 'local', 'stack_size']
29+
'setprofile', 'settrace', 'local', 'stack_size',
30+
'excepthook', 'ExceptHookArgs']
3131

3232
# Rename some stuff so "from threading import *" is safe
3333
_start_new_thread = _thread.start_new_thread
@@ -752,14 +752,6 @@ class Thread:
752752
"""
753753

754754
_initialized = False
755-
# Need to store a reference to sys.exc_info for printing
756-
# out exceptions when a thread tries to use a global var. during interp.
757-
# shutdown and thus raises an exception about trying to perform some
758-
# operation on/with a NoneType
759-
_exc_info = _sys.exc_info
760-
# Keep sys.exc_clear too to clear the exception just before
761-
# allowing .join() to return.
762-
#XXX __exc_clear = _sys.exc_clear
763755

764756
def __init__(self, group=None, target=None, name=None,
765757
args=(), kwargs=None, *, daemon=None):
@@ -802,9 +794,9 @@ class is implemented.
802794
self._started = Event()
803795
self._is_stopped = False
804796
self._initialized = True
805-
# sys.stderr is not stored in the class like
806-
# sys.exc_info since it can be changed between instances
797+
# Copy of sys.stderr used by self._invoke_excepthook()
807798
self._stderr = _sys.stderr
799+
self._invoke_excepthook = _make_invoke_excepthook()
808800
# For debugging and _after_fork()
809801
_dangling.add(self)
810802

@@ -929,47 +921,8 @@ def _bootstrap_inner(self):
929921

930922
try:
931923
self.run()
932-
except SystemExit:
933-
pass
934924
except:
935-
# If sys.stderr is no more (most likely from interpreter
936-
# shutdown) use self._stderr. Otherwise still use sys (as in
937-
# _sys) in case sys.stderr was redefined since the creation of
938-
# self.
939-
if _sys and _sys.stderr is not None:
940-
print("Exception in thread %s:\n%s" %
941-
(self.name, _format_exc()), file=_sys.stderr)
942-
elif self._stderr is not None:
943-
# Do the best job possible w/o a huge amt. of code to
944-
# approximate a traceback (code ideas from
945-
# Lib/traceback.py)
946-
exc_type, exc_value, exc_tb = self._exc_info()
947-
try:
948-
print((
949-
"Exception in thread " + self.name +
950-
" (most likely raised during interpreter shutdown):"), file=self._stderr)
951-
print((
952-
"Traceback (most recent call last):"), file=self._stderr)
953-
while exc_tb:
954-
print((
955-
' File "%s", line %s, in %s' %
956-
(exc_tb.tb_frame.f_code.co_filename,
957-
exc_tb.tb_lineno,
958-
exc_tb.tb_frame.f_code.co_name)), file=self._stderr)
959-
exc_tb = exc_tb.tb_next
960-
print(("%s: %s" % (exc_type, exc_value)), file=self._stderr)
961-
self._stderr.flush()
962-
# Make sure that exc_tb gets deleted since it is a memory
963-
# hog; deleting everything else is just for thoroughness
964-
finally:
965-
del exc_type, exc_value, exc_tb
966-
finally:
967-
# Prevent a race in
968-
# test_threading.test_no_refcycle_through_target when
969-
# the exception keeps the target alive past when we
970-
# assert that it's dead.
971-
#XXX self._exc_clear()
972-
pass
925+
self._invoke_excepthook(self)
973926
finally:
974927
with _active_limbo_lock:
975928
try:
@@ -1163,6 +1116,104 @@ def getName(self):
11631116
def setName(self, name):
11641117
self.name = name
11651118

1119+
1120+
try:
1121+
from _thread import (_excepthook as excepthook,
1122+
_ExceptHookArgs as ExceptHookArgs)
1123+
except ImportError:
1124+
# Simple Python implementation if _thread._excepthook() is not available
1125+
from traceback import print_exception as _print_exception
1126+
from collections import namedtuple
1127+
1128+
_ExceptHookArgs = namedtuple(
1129+
'ExceptHookArgs',
1130+
'exc_type exc_value exc_traceback thread')
1131+
1132+
def ExceptHookArgs(args):
1133+
return _ExceptHookArgs(*args)
1134+
1135+
def excepthook(args, /):
1136+
"""
1137+
Handle uncaught Thread.run() exception.
1138+
"""
1139+
if args.exc_type == SystemExit:
1140+
# silently ignore SystemExit
1141+
return
1142+
1143+
if _sys is not None and _sys.stderr is not None:
1144+
stderr = _sys.stderr
1145+
elif args.thread is not None:
1146+
stderr = args.thread._stderr
1147+
if stderr is None:
1148+
# do nothing if sys.stderr is None and sys.stderr was None
1149+
# when the thread was created
1150+
return
1151+
else:
1152+
# do nothing if sys.stderr is None and args.thread is None
1153+
return
1154+
1155+
if args.thread is not None:
1156+
name = args.thread.name
1157+
else:
1158+
name = get_ident()
1159+
print(f"Exception in thread {name}:",
1160+
file=stderr, flush=True)
1161+
_print_exception(args.exc_type, args.exc_value, args.exc_traceback,
1162+
file=stderr)
1163+
stderr.flush()
1164+
1165+
1166+
def _make_invoke_excepthook():
1167+
# Create a local namespace to ensure that variables remain alive
1168+
# when _invoke_excepthook() is called, even if it is called late during
1169+
# Python shutdown. It is mostly needed for daemon threads.
1170+
1171+
old_excepthook = excepthook
1172+
old_sys_excepthook = _sys.excepthook
1173+
if old_excepthook is None:
1174+
raise RuntimeError("threading.excepthook is None")
1175+
if old_sys_excepthook is None:
1176+
raise RuntimeError("sys.excepthook is None")
1177+
1178+
sys_exc_info = _sys.exc_info
1179+
local_print = print
1180+
local_sys = _sys
1181+
1182+
def invoke_excepthook(thread):
1183+
global excepthook
1184+
try:
1185+
hook = excepthook
1186+
if hook is None:
1187+
hook = old_excepthook
1188+
1189+
args = ExceptHookArgs([*sys_exc_info(), thread])
1190+
1191+
hook(args)
1192+
except Exception as exc:
1193+
exc.__suppress_context__ = True
1194+
del exc
1195+
1196+
if local_sys is not None and local_sys.stderr is not None:
1197+
stderr = local_sys.stderr
1198+
else:
1199+
stderr = thread._stderr
1200+
1201+
local_print("Exception in threading.excepthook:",
1202+
file=stderr, flush=True)
1203+
1204+
if local_sys is not None and local_sys.excepthook is not None:
1205+
sys_excepthook = local_sys.excepthook
1206+
else:
1207+
sys_excepthook = old_sys_excepthook
1208+
1209+
sys_excepthook(*sys_exc_info())
1210+
finally:
1211+
# Break reference cycle (exception stored in a variable)
1212+
args = None
1213+
1214+
return invoke_excepthook
1215+
1216+
11661217
# The timer class was contributed by Itamar Shtull-Trauring
11671218

11681219
class Timer(Thread):

‎Misc/NEWS.d/next/Library/2019-05-23-01-48-39.bpo-1230540.oKTNEQ.rst‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
Add a new :func:`threading.excepthook` function which handles uncaught
2+
:meth:`threading.Thread.run` exception. It can be overridden to control how
3+
uncaught :meth:`threading.Thread.run` exceptions are handled.

0 commit comments

Comments
 (0)