Skip to content

Fix binary stdin and stdout handling in the code-generation judge - #161

Open
mantuoluozk wants to merge 1 commit into
LiveCodeBench:mainfrom
mantuoluozk:fix/binary-stdio-judge
Open

mantuoluozk wants to merge 1 commit into
LiveCodeBench:mainfrom
mantuoluozk:fix/binary-stdio-judge

Conversation

@mantuoluozk

@mantuoluozk mantuoluozk commented Sep 30, 2026 •

Copy link
Copy Markdown

Fixes #160.

Summary

Fix binary standard I/O handling in the code-generation judge.

  • Make sys.stdin.buffer use a cursor-backed byte stream, so consecutive readline() calls consume consecutive lines.
  • Make text and binary stdin reads share the same input position. Remove the separate stdin method patches that could return inconsistent data.
  • Capture sys.stdout.buffer.write(...) while preserving the order of text and binary writes.
  • Add regression tests for binary reads and writes, mixed text/binary I/O, and the standard-I/O judging path.

Validation

  • 8 new regression tests passed in the 0.5.18 Docker environments.
  • Rejudged the same extracted solutions and test cases from a full 175-problem LiveCodeBench v6 run generated with SGLang 0.5.18:
    • Unmodified upstream judge: 154/175
    • This PR: 159/175
    • 5 problems changed from fail to pass; none changed from pass to fail.

The recovered problems are abc391_f (stdin.buffer.readline()) and abc393_e, abc398_f, abc398_d, and arc191_a (stdout.buffer).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Code-generation judge rejects valid solutions using sys.stdin.buffer.readline() or sys.stdout.buffer

1 participant