Skip to content

fallback: return available bytes on short feed-based reads - #752

Open
zhechen-cs wants to merge 1 commit into
msgpack:mainfrom
zhechen-cs:codex/return-available-raw-bytes
Open

zhechen-cs wants to merge 1 commit into
msgpack:mainfrom
zhechen-cs:codex/return-available-raw-bytes

Conversation

@zhechen-cs

Copy link
Copy Markdown

With a feed-based Unpacker, read_bytes(5) raises OutOfData in the pure-Python fallback when only b"ab" is buffered. The C extension returns b"ab", and both implementations already return short reads for file-backed input.

Honor raise_outofdata=False in the feed path so raw reads consume and return the available bytes. Normal unpacking still rolls back and raises OutOfData for incomplete objects. The regression covers empty and short buffers, reads after a decoded object, stream offsets, and feeding the next object afterward.

Validation on macOS arm64, CPython 3.12:

  • Both new cases fail on the unmodified fallback and pass on the unmodified C extension.
  • After the fix: fallback full suite 144 passed, 4 skipped; C extension full suite 147 passed, 1 skipped.
  • Ruff lint and format checks, and git diff --check, pass.

The existing thread_unsafe marker warning also occurs on the base. Other platforms and PyPy have not been tested locally.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant