Skip to content

itertools.accumulate() silently corrupts its running total on reentrancy #154570

Description

@PhysicistJohn

Bug description:

itertools.accumulate() silently corrupts its internal running total
if the source iterable (or the func= callable) calls back into
next() on the same accumulate object mid-step. Unlike
itertools.tee(), which explicitly detects and rejects this with
RuntimeError: cannot re-enter the tee iterator, accumulate has no
such guard, so the reentrant call fully advances total before the
outer call resumes and overwrites it, silently dropping a value from
the output instead of erroring or producing a well-defined result.

Reproducer:

import itertools

class Reentrant:
    def __init__(self, values, acc_holder):
        self.values = iter(values)
        self.acc_holder = acc_holder
        self.reentered = False
    def __iter__(self):
        return self
    def __next__(self):
        v = next(self.values)
        if v == 2 and not self.reentered:
            self.reentered = True
            next(self.acc_holder[0])  # reenters accumulate
        return v

holder = [None]
acc = itertools.accumulate(Reentrant([1, 2, 3, 4], holder))
holder[0] = acc
print(list(acc))                                # [1, 6, 10]  -- WRONG, 3 is lost
print(list(itertools.accumulate([1, 2, 3, 4])))  # [1, 3, 6, 10] -- correct

This is silent data corruption, not a crash, which makes it more
dangerous than a segfault: code can run to completion and produce a
plausible-looking but wrong result with no error or warning.

CPython versions tested on:

CPython main branch (3.16.0a0)

Operating systems tested on:

macOS (arm64)

Linked PRs

Metadata

Metadata

Assignees

No one assigned

    Labels

    extension-modulesC modules in the Modules dirpendingThe issue will be closed if no feedback is providedtype-bugAn unexpected behavior, bug, or error

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions