zlib: return output inflate still holds after the last input byte - #8909

The decompression loop now processes empty input to retrieve pending output. It checks output-buffer fullness before treating exhausted input as complete. Rust and Python regression tests cover pending output, small output limits, and empty-stream behavior.

Changes: Pending zlib output

Layer / File(s)	Summary
Drain pending output and verify stream state crates/common/src/compression/zlib.rs	decompress_chunks calls inflate for empty input. It does not treat exhausted input as completion when the output buffer is full.
Test pending output and stream state crates/common/src/compression/zlib.rs, extra_tests/snippets/stdlib_zlib.py	Tests cover retrieving pending raw-deflate output with empty-input decompression or flush(), small output limits, and accepting later input after an empty flush.

Priority: ➖ Normal

Estimated code review effort: 2 (Simple) | ~10 minutes

Change: Bug fix · Severity of issue fixed: Medium

Suggested reviewers: youknowone

Callers relying on needs_input may stop before retrieving the last bytes of a bounded raw-deflate stream. Correct the flag before merging if those callers must be supported.

Systems and components

• observed — crates (service) was modified; 1 changed file maps to changed impact.
• observed — extra_tests (service) was modified; 1 changed file maps to changed impact.

Before / after behavior

• observed — Modified behavior in crates/common/src/compression/zlib.rs: Removed the early return that made empty input produce no output without calling inflate. Empty input now proceeds through the decompression loop, allowing pending output to be returned; the stream remains unfinished when inflate reports a buffer error.
• observed — Modified behavior in crates/common/src/compression/zlib.rs: Completion no longer follows merely from exhausted input. The loop also continues when the output buffer was filled, since inflate may have pending output; it returns as unfinished when the buffer is only partially filled and the stream has not ended.
• observed — Modified behavior in crates/common/src/compression/zlib.rs: Added tests and a helper that drains raw deflate input in 100-byte output limits until 1,100 of 1,168 bytes are produced. The tests check that empty-input decompression and flush retrieve the remaining output and reach EOF, that one-shot decompression succeeds with buffer sizes 1 and 100, and that flushing an empty stream does not set EOF before later input completes it.
• observed — Modified behavior in extra_tests/snippets/stdlib_zlib.py: Added drained_raw_deflate(), which creates a raw-deflate stream for 1168 repeated bytes, consumes output in 100-byte chunks, and asserts that 1100 bytes are returned while the decompressor is not yet at EOF.

Reliability and maintainability

• inferred — Risk-relevant change factors for crates: blast_radius_2; direct_dependents_2

Pre-merge checks: 5

Passed checks: 5 passed

Check name	Status	Explanation
Description Check		Check skipped - CodeRabbit’s high-level summary is enabled.
Title check		The title accurately identifies the zlib fix: return output that inflate still holds after the last input byte. It is specific to the main change.
Linked Issues check		Issue [#8907] requires bounded raw-deflate decompression to return output that remains in inflate after input is consumed. decompress_chunks now runs for empty input and continues when the output bu…
Out of Scope Changes check		The changes modify only the shared zlib decompression loop and add tests for the pending-output and empty-input behavior required by [#8907]. No unrelated production changes or unrelated files appear …
Docstring Coverage		Docstring coverage is 87.50% which is sufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 8 functions across 2 files.

🧪 Generate unit tests (beta)

• Create a new PR

• X
• Mastodon
• Reddit
• LinkedIn

🏁 Script executed:

#!/bin/bash
set -euo pipefail
ast-grep outline crates/common/src/compression/zlib.rs --items all
rg -n -C 7 'needs_input|fn decompress_chunks' crates/common/src/compression/zlib.rs
gh pr diff 8909 --repo RustPython/RustPython
python - <<'PY'
import urllib.request
for path in ("Modules/zlibmodule.c", "Modules/_bz2module.c", "Modules/_lzmamodule.c"):
    url = "https://raw.githubusercontent.com/python/cpython/main/" + path
    text = urllib.request.urlopen(url, timeout=30).read().decode()
    lines = text.splitlines()
    print("\nSOURCE:", url)
    for i, line in enumerate(lines):
        if "needs_input = " in line:
            lo, hi = max(0, i - 10), min(len(lines), i + 9)
            print("\n".join(f"{j+1}: {lines[j]}" for j in range(lo, hi)))
PY

Length of output: 15271

🏁 Script executed:

#!/bin/bash
set -euo pipefail
python - <<'PY'
import urllib.request
for path in ("Modules/zlibmodule.c", "Modules/_bz2module.c", "Modules/_lzmamodule.c"):
    url = "https://raw.githubusercontent.com/python/cpython/v3.14.0/" + path
    lines = urllib.request.urlopen(url, timeout=30).read().decode().splitlines()
    print("\nSOURCE:", url)
    spans = set()
    for i, line in enumerate(lines):
        if "needs_input" in line and "=" in line:
            spans.update(range(max(0, i-12), min(len(lines), i+8)))
    previous = -2
    for j in sorted(spans):
        if j != previous + 1:
            print("---")
        print(f"{j+1}: {lines[j]}")
        previous = j
PY

Length of output: 7042

Learnt from: fanninpm
URL: https://github.com/RustPython/RustPython/pull/8909

Timestamp: 2026-10-16T13:29:45.418Z
Learning: RustPython PR #8909 targets CPython parity for pending zlib output. In crates/common/src/compression/zlib.rs, ZlibDecompressor sets needs_input to true when input is exhausted; PR #8909 does not change that assignment. This matches CPython 3.14's _ZlibDecompressor and BZ2Decompressor, which do not check whether the output buffer is full in that branch. CPython's LZMADecompressor additionally checks avail_out == 0. Do not require LZMA-style needs_input behavior as part of PR #8909.

Note: Learnings are effective only in the context of similar code segments. To apply general coding guidelines, please configure review instructions. You can manage existing learnings in the app.