concurrent.futures Executor.map cancels other futures when one raises an exception

Bug report

Checklist

I am confident this is a bug in CPython, not a bug in a third-party project
I have searched the CPython issue tracker,
and am confident this bug has not been reported before

CPython versions tested on:

3.11

Operating systems tested on:

Linux

Output from running 'python -VV' on the command line:

Python 3.11.5 (main, Aug 24 2023, 12:23:19) [GCC 11.4.0]

A clear and concise description of the bug:

While using concurrent.futures.ProcessPoolExecutor.map when one of the calls to the function raises an exception, per the documentation, this exception is raised when the value is retrieved from the iterator returned by the call to concurrent.futures.ProcessPoolExecutor.map. However, it was seen that the iterator also terminates when the exception is retrieved from the iterator which is unexpected and doesn't align with what the built-in map function does. This means that other values fetched in separate calls cannot be retrieved with this method.

Reproducible example

import concurrent.futures
import contextlib

def add_one(value):
    if value == 2:
        raise ValueError()
    return value + 1

def test(executor_cm):
    with executor_cm as executor:
        values = executor.map(add_one, (1, 2, 3))
        values_it = iter(values)
        try:
            while True:
                try:
                    value = next(values_it)
                except ValueError:
                    print("value error")           
                else:
                    print(f"value: {value}")
        except StopIteration: pass

print("ProcessPoolExecutor")
test(concurrent.futures.ProcessPoolExecutor())
print("\nThreadPoolExecutor")
test(concurrent.futures.ThreadPoolExecutor())
print("\nBuilt-in map function")
class fake_executor:
    map = staticmethod(map)
test(contextlib.nullcontext(fake_executor))

The output of the example above:

ProcessPoolExecutor
value: 2
value error

ThreadPoolExecutor
value: 2
value error

Built-in map function
value: 2
value error
value: 4

The expected output for the example:

ProcessPoolExecutor
value: 2
value error
value: 4

ThreadPoolExecutor
value: 2
value error
value: 4

Built-in map function
value: 2
value error
value: 4

Linked PRs

gh-108518: Make concurrent.futures.Executor.map() consistent with built-in map() #109497

@gpshead

Caused by

cpython/Lib/concurrent/futures/_base.py

Lines 612 to 625 in d08d49d

    
           def result_iterator(): 
        
               try: 
        
                   # reverse to keep finishing order 
        
                   fs.reverse() 
        
                   while fs: 
        
                       # Careful not to keep a reference to the popped future 
        
                       if timeout is None: 
        
                           yield _result_or_cancel(fs.pop()) 
        
                       else: 
        
                           yield _result_or_cancel(fs.pop(), end_time - time.monotonic()) 
        
               finally: 
        
                   for future in fs: 
        
                       future.cancel() 
        
           return result_iterator()

You cannot return to the generator function after raising any exception inside, and current behavior obviously contradicts the documentation. The problem applies to all supported python versions.

I can imagine the solution based on making result_iterator a class with __next__ method, however, this needs some insights from concurrent.futures experts (@gpshead) first: such changes will be breaking.

gh-108518: Do not cancel remaining calls in Executor.map() on error (G…

The behavior changed in 3.16. This is a new feature.

pythongh-108518: Do not cancel remaining calls in Executor.map() on e…

gh-155852: Do not cancel remaining Executor.map calls on a callable's…

Navigation Menu

Repository navigation

concurrent.futures Executor.map cancels other futures when one raises an exception #108518

Bug report

Checklist

CPython versions tested on:

Operating systems tested on:

Output from running 'python -VV' on the command line:

A clear and concise description of the bug:

Reproducible example

Linked PRs

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

	def result_iterator():
	try:
	# reverse to keep finishing order
	fs.reverse()
	while fs:
	# Careful not to keep a reference to the popped future
	if timeout is None:
	yield _result_or_cancel(fs.pop())
	else:
	yield _result_or_cancel(fs.pop(), end_time - time.monotonic())
	finally:
	for future in fs:
	future.cancel()
	return result_iterator()

concurrent.futures Executor.map cancels other futures when one raises an exception #108518

Description

Bug report

Checklist

CPython versions tested on:

Operating systems tested on:

Output from running 'python -VV' on the command line:

A clear and concise description of the bug:

Reproducible example

Linked PRs

Activity

sterliakov commented on Aug 29, 2023

serhiy-storchaka commented on Aug 11, 2026

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions