Skip to content

GH-84559: Deprecate fork being the multiprocessing default. #100618

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
gpshead merged 23 commits into from
Feb 2, 2023
Merged

GH-84559: Deprecate fork being the multiprocessing default. #100618

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
1cecbdb
GH-84559: Deprecate fork being the multiprocessing default.
gpshead Dec 30, 2022
df83fba
address CI test failures?
gpshead Dec 31, 2022
ec1e868
the new warning needs no pickle name mapping
gpshead Dec 31, 2022
a1064de
avoid DeprecationWarnings vs -Werror
gpshead Dec 31, 2022
223b748
prefer forkserver over fork for concurrent.futures
gpshead Dec 31, 2022
fc64626
explicitly use spawn for simple process launchers.
gpshead Dec 31, 2022
e45fab6
warn from concurrent.futures, reword things.
gpshead Dec 31, 2022
29764a2
avoid double warning in futures
gpshead Dec 31, 2022
c52b9db
Cleanup messages, test futures warning.
gpshead Dec 31, 2022
4c8cc34
cleanup warn stacklevel logic
gpshead Dec 31, 2022
7179684
Document all the things!
gpshead Dec 31, 2022
f7e3161
Have all doctests use 'spawn'.
gpshead Jan 1, 2023
9947860
Correct warning stacklevel for Manager.
gpshead Jan 1, 2023
5834f2d
more doctest cleanup.
gpshead Jan 1, 2023
7d48e86
NEWS entry.
gpshead Jan 1, 2023
28c0cfc
docs: add the implicit i
gpshead Jan 10, 2023
2820bb8
Merge branch 'main' into multiprocessing/deprecate-fork-default
gpshead Jan 28, 2023
4c16205
Use skip_file_prefixes.
gpshead Jan 28, 2023
7d84a72
use the modern warnings API, doc updates.
gpshead Jan 28, 2023
8b2029e
documentation cleanup
gpshead Jan 28, 2023
846e9a0
Remove an unused variable.
gpshead Feb 1, 2023
7078986
doc typo fix, unnecessary method change revert.
gpshead Feb 1, 2023
e3e3b3c
Add What's New entries.
gpshead Feb 2, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Use skip_file_prefixes.
  • Loading branch information
@gpshead
gpshead committed Jan 28, 2023
commit 4c16205033c902ff76067f4f47f89b7ad36c92b5
2 changes: 1 addition & 1 deletion Lib/concurrent/futures/process.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -57,6 +57,7 @@
import itertools
import sys
from traceback import format_exception
import warnings


_threads_wakeups = weakref.WeakKeyDictionary()
Expand Down Expand Up @@ -652,7 +653,6 @@ def __init__(self, max_workers=None, mp_context=None,
mp_context = mp.get_context()
if (mp_context.get_start_method() == "fork" and
mp_context == mp.context._default_context._default_context):
import warnings
warnings.warn(
"The default multiprocessing start method will change "
"away from 'fork' in Python >= 3.14, per GH-84559. "
Expand Down
21 changes: 5 additions & 16 deletions Lib/multiprocessing/context.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -59,12 +59,7 @@ def Manager(self):
ctx = self.get_context()
m = SyncManager(ctx=ctx)
proc_class = ctx.Process
Comment thread
gpshead marked this conversation as resolved.
Outdated
orig_level = getattr(proc_class, '_ORIG_WARNING_STACKLEVEL', None)
if orig_level:
ctx.Process._warning_stacklevel = 6 # blame mp.Manager()
m.start()
if orig_level:
ctx.Process._warning_stacklevel = orig_level
return m

def Pipe(self, duplex=True):
Expand Down Expand Up @@ -291,9 +286,11 @@ def _Popen(process_obj):
from .popen_fork import Popen
return Popen(process_obj)

_warn_package_prefixes = (os.path.dirname(__file__),)

class _DeprecatedForkProcess(ForkProcess):
@staticmethod
def _warn(stacklevel):
@classmethod
def _Popen(cls, process_obj):
import warnings
warnings.warn(
"The default multiprocessing start method will change "
Expand All @@ -302,16 +299,8 @@ def _warn(stacklevel):
"explicitly specify it when your application requires 'fork'. "
"The safest start method is 'spawn'.",
category=DefaultForkDeprecationWarning,
stacklevel=stacklevel,
skip_file_prefixes=_warn_package_prefixes,
)

# Blame the process.start() method call.
_ORIG_WARNING_STACKLEVEL = 5
_warning_stacklevel = _ORIG_WARNING_STACKLEVEL

@classmethod
def _Popen(cls, process_obj):
cls._warn(stacklevel=cls._warning_stacklevel)
return super()._Popen(process_obj)

class SpawnProcess(process.BaseProcess):
Expand Down
10 changes: 10 additions & 0 deletions Misc/NEWS.d/next/Library/2022-12-30-01-56-24.gh-issue-84559.pFFtaE.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
The :mod:`multiprocessing` module now triggers a :exc:`DeprecationWarning`
when the (``fork``) start method is used implicitly via the default behavior
on non-macOS POSIX-like platforms.

We want application authors to explicitly specify their start method (thus
avoiding the warning) when they do need to use ``fork``. Our default will
change to something more conservative and compatible with multithreaded
processes in the Python 3.14 timeframe.

macOS and Windows already default to the ``spawn`` start method.