Skip to content

gh-155126: Implement PEP 828 - #155127

Open
ZeroIntensity wants to merge 16 commits into
python:mainfrom
ZeroIntensity:pep-828/implementation
Open

ZeroIntensity wants to merge 16 commits into
python:mainfrom
ZeroIntensity:pep-828/implementation

Conversation

@ZeroIntensity

@ZeroIntensity ZeroIntensity commented Aug 3, 2026 •

Copy link
Copy Markdown
Member

ZeroIntensity and others added 3 commits August 3, 2026 12:11
Co-authored-by: Bartosz Sławecki <bartosz@ilikepython.com>
@read-the-docs-community

read-the-docs-community Bot commented Aug 3, 2026 •

Copy link
Copy Markdown

@ZeroIntensity

Copy link
Copy Markdown
Member Author

cc @1st1

@johnslavik
johnslavik requested a review from 1st1 August 4, 2026 13:29
Comment thread Doc/reference/expressions.rst
Co-authored-by: Bartosz Sławecki <bartosz@ilikepython.com>
Comment thread Lib/test/test_yield_from_async.py Outdated
@bedevere-app bedevere-app Bot added the type-feature A feature request or enhancement label Oct 12, 2026
ZeroIntensity and others added 2 commits October 19, 2026 11:52
Treat a deleted `StopAsyncIteration.value` as `None` when handling
delegated iterator exhaustion.

Add a regression test covering immediate exhaustion and exhaustion
after yielding an item.
Comment thread Objects/genobject.c Outdated
// this was called from athrow(). This broke some tests/expected behavior
// for yield from in asyncgens. Removing the parameter didn't seem to cause
// any new test failures, nor could I reproduce any different behavior
// when experimenting with it, but we need to be careful.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Neither configuration is correct. Removing the flag breaks asynchronous cleanup; restoring it fixes that case but breaks the required close semantics for GeneratorExit.

I rebuilt and tested both configurations, restoring the original flag plumbing from main: gen_throw() passes 1, the initial aclose()/athrow() calls pass 0, and recursive calls preserve the flag.

Case Current branch, flag removed Flag restored
Delegate’s cleanup completes synchronously Works Works
Delegate’s cleanup suspends during aclose() Fails with RuntimeError; cleanup incomplete Works
Delegate’s cleanup suspends during athrow(GeneratorExit) Same failure Works
Delegate suppresses GeneratorExit during outer athrow(GeneratorExit) Correctly reraises the outer exception Incorrectly allows delegation to finish normally
Delegate yields an item while handling GeneratorExit Correctly raises RuntimeError Outer athrow() incorrectly returns that item
Delegate raises its own GeneratorExit during closing Preserves expected outer exception identity Identity test fails
Ordinary async generator’s own cleanup awaits Works Works
Custom async iterator implements aclose()/athrow() Methods skipped Methods skipped

The missing case in the tests is cleanup that actually suspends. This reproduces the current branch’s problem:

import asyncio

async def inner():
    try:
        yield 1
    finally:
        await asyncio.sleep(0)
        print("cleanup completed")

async def outer(sub):
    yield from sub

async def main():
    sub = inner()
    gen = outer(sub)
    await anext(gen)
    await gen.aclose()

asyncio.run(main())

Currently, aclose() raises:

RuntimeError: async generator ignored GeneratorExit

The cleanup never reaches the print. With the flag restored, cleanup completes and aclose() succeeds.

The cause is gen_close_iter() (Objects/genobject.c:427): the PR adds async generators to the branch that calls synchronous gen_close(). When cleanup suspends at an await, gen_close() treats the suspension as an illegal yield and raises RuntimeError.

Restoring the flag only exchanges one problem for another. It makes _gen_throw() forward GeneratorExit into the delegate. But async delegation must await the delegate’s aclose() and then reraise the original exception. Forwarding through throw lets the delegate suppress the exception or yield another item. That distinction is explicit in PEP 828’s reference expansion.

The custom iterator failures expose the same underlying mismatch: the current helpers look for synchronous .close() and .throw(), rather than calling and awaiting .aclose() and .athrow(). I confirmed this against a Python implementation of the PEP’s expansion; the reference calls those methods, while native delegation skips them in both configurations.

The fix needs to distinguish closing an async delegate from throwing into it, and provide a resumable path for awaiting its aclose(). A boolean change alone cannot provide both behaviors. I would investigate handling this in the async delegation bytecode expansion, where the operation can suspend, rather than extending synchronous gen_close_iter().

Validation:

  • Current branch: all 285 tests pass across test_yield_from_async, test_asyncgen, test_yield_from, and test_coroutines.
  • Flag restored: 4 errors and 3 failures, all in test_yield_from_async; the other suites pass.
  • Additional probes confirm the missing cleanup and custom iterator failures above.

The source and interpreter are restored to your current implementation; the working tree is clean. The comparison probe (/private/tmp/pep828-protocol-probe.py) and delegation edge-case probe (/private/tmp/pep828-delegation-probe.py) remain available for review.

@ZeroIntensity
ZeroIntensity requested a review from FFY00 as a code owner October 19, 2026 21:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

awaiting core review sprint type-feature A feature request or enhancement

Projects

Status: Todo

Development

Successfully merging this pull request may close these issues.

4 participants