Skip to content

GH-84559: Deprecate fork being the multiprocessing default. #100618

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
gpshead merged 23 commits into from
Feb 2, 2023
Merged

GH-84559: Deprecate fork being the multiprocessing default. #100618

Changes from 1 commit
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
1cecbdb
GH-84559: Deprecate fork being the multiprocessing default.
gpshead Dec 30, 2022
df83fba
address CI test failures?
gpshead Dec 31, 2022
ec1e868
the new warning needs no pickle name mapping
gpshead Dec 31, 2022
a1064de
avoid DeprecationWarnings vs -Werror
gpshead Dec 31, 2022
223b748
prefer forkserver over fork for concurrent.futures
gpshead Dec 31, 2022
fc64626
explicitly use spawn for simple process launchers.
gpshead Dec 31, 2022
e45fab6
warn from concurrent.futures, reword things.
gpshead Dec 31, 2022
29764a2
avoid double warning in futures
gpshead Dec 31, 2022
c52b9db
Cleanup messages, test futures warning.
gpshead Dec 31, 2022
4c8cc34
cleanup warn stacklevel logic
gpshead Dec 31, 2022
7179684
Document all the things!
gpshead Dec 31, 2022
f7e3161
Have all doctests use 'spawn'.
gpshead Jan 1, 2023
9947860
Correct warning stacklevel for Manager.
gpshead Jan 1, 2023
5834f2d
more doctest cleanup.
gpshead Jan 1, 2023
7d48e86
NEWS entry.
gpshead Jan 1, 2023
28c0cfc
docs: add the implicit i
gpshead Jan 10, 2023
2820bb8
Merge branch 'main' into multiprocessing/deprecate-fork-default
gpshead Jan 28, 2023
4c16205
Use skip_file_prefixes.
gpshead Jan 28, 2023
7d84a72
use the modern warnings API, doc updates.
gpshead Jan 28, 2023
8b2029e
documentation cleanup
gpshead Jan 28, 2023
846e9a0
Remove an unused variable.
gpshead Feb 1, 2023
7078986
doc typo fix, unnecessary method change revert.
gpshead Feb 1, 2023
e3e3b3c
Add What's New entries.
gpshead Feb 2, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
cleanup warn stacklevel logic
  • Loading branch information
@gpshead
gpshead committed Dec 31, 2022
commit 4c8cc342ffc789c46180f0bdb29722085ee7edf1
25 changes: 13 additions & 12 deletions Lib/multiprocessing/pool.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -326,20 +326,21 @@ def _repopulate_pool_static(ctx, Process, processes, pool, inqueue,
wrap_exception))
w.name = w.name.replace('Process', 'PoolWorker')
w.daemon = True
if _warn := getattr(w, '_warn', None): # GH-84559
# Hacky, but BaseProcess._Popen does everything via the class
# using static or class methods with no instance. The way
# multiprocessing APIs get used, a concurrency mess here
# with a Pool() being constructed at the same time as other
# threads are calling Process.start() is highly unlikely.
# If so: The oddball application gets the warning attributed
proc_class = w.__class__
orig_level = getattr(proc_class, '_ORIG_WARNING_STACKLEVEL', None)
if orig_level:
# Direct the GH-84559 warning to a useful place.
# Hacky, but BaseProcess._Popen does everything via class
# and static methods with no instance. The way multiprocessing
# APIs get used, a concurrency mess here with a Pool() being
# constructed at the same time other threads are calling
# Process.start() is highly unlikely.
# If so: That oddball application gets the warning attributed
# to an unusual spot in the stack on occasion.
process_class = w.__class__
process_class._warning_stacklevel = 8 # blame mp.Pool()
proc_class._warning_stacklevel = 8 # blame mp.Pool()
w.start()
if _warn:
process_class._warning_stacklevel = (
process_class._ORIG_WARNING_STACKLEVEL)
if orig_level:
proc_class._warning_stacklevel = orig_level
pool.append(w)
util.debug('added worker')

Expand Down