Repository navigation
Conversation
Parser.setupCharStream() used ANTLR's CharStreams.fromReader(), which reads the document in fixed 4096-char chunks. CodePointBuffer.Builder keeps a pending high surrogate across chunks to support pairs split by a chunk boundary, but its end-of-chunk flush fails to clear that pending state, so the surrogate gets decoded twice: once as a lone code point at the chunk boundary, and again when paired with the next chunk's low surrogate. In practice this makes graphql-java reject an otherwise valid document whenever an astral character (e.g. an emoji) lands with its high surrogate at index 4095, 8191, etc., since the resulting lone surrogate fails GraphQL's StringCharacter grammar rule. This is a known upstream defect (antlr/antlr4#4943, open and unreviewed). Rather than wait on it, this replaces fromReader() with an equivalent local read loop that holds back a chunk-trailing unpaired high surrogate so it is carried into the next chunk instead of being flushed prematurely.
xuorig
commented
Sep 22, 2026
| * | ||
| * @see <a href="https://github.com/antlr/antlr4/pull/4943">antlr/antlr4#4943</a> | ||
| */ | ||
| private static CodePointCharStream readCodePointCharStream(Reader reader, int bufferSize) throws IOException { |
Author
There was a problem hiding this comment.
If you'd like to compare, here's the original: https://github.com/rootvector2/antlr4/blob/master/runtime/Java/src/org/antlr/v4/runtime/CharStreams.java#L189-L206
Contributor
Test ReportTest Results
Code Coverage (Java 25)
Changed Class Coverage (2 classes)
|
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Hey folks, I discovered (actually re-discovered it seems!) a rare but quite tricky issue with ANTLR while I was working on a PR to allow configuring the reader buffer size.
Somebody actually opened a PR for this recently on ANTLR: antlr/antlr4#4943 but I have no idea if it will get merged and/or when.
It's little work to take control of that read loop if we wanted to, and might actually help with allowing to configure that read buffer size (Will follow up with another PR)
What do you think? This actually explained a really weird issue we saw with a schema update months ago and could never reproduce. Basically a surrogate pair placed at exactly the wrong place (split by buffer size) would cause parsing errors due to a dangling lone surrogate.
cc @rootvector2 & @GeorgEchterling who seem to have faced this issue as well.