Repository navigation
Conversation
GeminiLlmConnection turned each live server message into responses without aggregating text, and marked a response partial only when its message carried turnComplete: false. Runner.runLive passed every event to appendEvent, while runAsync skips partial events. A live turn's text was therefore stored as one model event per chunk, or, with partial chunks, dropped by InMemorySessionService and stored chunk by chunk by FirestoreSessionService and VertexAiSessionService. Follow ADK Python's live connection and runner: - Only responses that stream text are partial. The connection also collects the text and emits it as one non-partial response before the end of the turn, an interruption or a tool call, and when thought and other text alternate or a part that is neither text nor inline data arrives. Text that arrives with the end of the turn or an interruption is emitted only in full. - runLive skips partial events when persisting, as runAsync does. Responses without text are no longer partial when their message carries turnComplete: false. Transcriptions are not aggregated yet.
Contributor
|
Hi @innoprej, Thank you for your contribution and for taking the time to submit this pull request. Our team is currently reviewing your changes, and we will reach out if we need any further information. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Please ensure you have read the contribution guide before creating a pull request.
Link to Issue or Description of Change
1. Link to an existing issue (if applicable):
2. Or, if no issue exists, describe the change:
Problem:
In live mode, what a turn leaves in the session depends on the session service.
GeminiLlmConnectionturns each server message into responses without aggregating text, and marks a response partial only when its message carriesturnComplete: false.Runner.runLivepasses every event toappendEvent, whilerunAsyncskips partial events. So a turn's text is stored as one model event per chunk, or, when the chunks are partial, dropped byInMemorySessionServiceand stored chunk by chunk byFirestoreSessionServiceandVertexAiSessionService. ADK Python stores it once. #1648 has the details and a reproduction.Solution:
Follow ADK Python's live connection (
gemini_llm_connection.py) and runner (runners.py):GeminiLlmConnection, only the responses that stream text are partial. The connection also collects that text in aPendingTextand emits it as one non-partial response before the response that completes the turn or reports an interruption (:550-559,:600-610), before a tool call (:619-627), and when thought and non-thought text alternate or a part that is neither text nor inline data arrives. Text that arrives in the same message as the end of the turn or an interruption is emitted only in that full response, as ADK Python does.turnComplete,interruptedand the transcriptions stay on the message's own response, where callers read them today. ADK Python marks the full-text responseinterruptedwhen an interruption ends the text, and then sends no separate interruption response unless the turn also completes. When a message streams text and also carries a transcription, the transcription gets a non-partial response of its own, so that it is still stored.Runner.runLiveskips partial events when persisting, asrunAsyncdoes.mainHelandlowithoutturnComplete, thenturnComplete: trueHel,lo, turn completeHel, partiallo,Hello, turn completeturnComplete: falseon the chunksHel, partiallo, turn completeHel, partiallo,Hello, turn completeHel, thenlowithturnComplete: trueHel,lowith turn completeHel,Hello, turn completeHel, theninterrupted: trueHel, interruptedHel,Hel, interruptedChecking, then a tool callChecking, tool callChecking,Checking, tool callrunLivenow hands only the non-partial events to the session service. Since every text chunk is partial, a turn that completes, is interrupted or calls a tool stores its text once, whether or not the Live API sendsturnComplete: falseon the intermediate messages, the question #1648 left open.Responses that do not stream text, such as audio, transcriptions and interruptions, are no longer partial; on
mainthey were partial whenever their message carriedturnComplete: false. If the Live API sends that flag,InMemorySessionServicenow stores them, as the other session services already did; otherwise nothing changes for them. ADK Python does not mark audio or interruptions partial either. Two differences from ADK Python remain:mainwhen the messages do not carryturnComplete: false. When they do,InMemorySessionServicenow stores it too, and the stored audio is sent back with the history of the next live connection. ADK Python keeps live model media with inline data out of the session in its runner (_should_append_event) and out of the history it sends (send_history). runLive passes partial events to appendEvent and never aggregates text chunks, unlike ADK Python #1648 lists this as a separate difference, and this PR adds neither check.Testing Plan
Unit Tests:
./mvnw -pl core clean test(Windows 11, Microsoft Build of OpenJDK 17.0.19): thedefault-testandbasicexecutions each run 2,214 tests with no failures (24 skipped), and the other four executions pass (apigee-llm-proxy-urlruns no tests, see #1557).mainGeminiLlmConnectionTest.receive_textChunks_streamedAsPartialAndEmittedInFullAtTurnComplete: an in-processGeminiLiveTransportdeliversHelwithoutturnComplete,lowithturnComplete: false, and an emptyturnComplete: truemain's connection gets[text:Hel, partial text:lo, turnComplete]and fails[partial text:Hel, partial text:lo, text:Hello, turnComplete]RunnerTest.runLive_partialEvent_streamedButNotPassedToSessionService, the live counterpart ofrunAsync_partialEvent_streamedButNotPassedToSessionServiceappendEventreceives[partial, final][final]GeminiLlmConnectionTest.convertToServerResponse_*tests for the rules above: text that arrives with the end of the turn, an interruption, a tool call, a thought switch, inline data (not partial, and does not end the text), another part (ends the text), text with a transcription, and an empty text partconvertToServerResponsehas noPendingTextparameter onmainFour existing
GeminiLlmConnectionTesttests now expect the full-text response before the message's own response:withInterruptedTrue_mapsInterruptedField,withoutInterruptedField_mapsEmptyOptional,withTurnCompleteTrue_mapsPartialFalseandwithContentAndUsageMetadata_emitsMultiple.withInterruptedTrue_mapsInterruptedFieldalso checks that the interruption response, whose message setsturnComplete: false, is not partial. The otherconvertToServerResponsetests only pass a newPendingText.Manual End-to-End (E2E) Tests:
Not run against the Gemini Live API. The connection tests drive the real
GeminiLlmConnectionand its conversion, and the Runner test usesTestLlm's live connection.Checklist
Additional context
runLivenow receive a turn's text twice, as partial chunks and then as one non-partial event, as ADK Python's callers do. A caller that shows the text of every model event needs to skip the partial events or the full one.main, its chunks were stored when they were not partial.FirestoreSessionService.appendEventstill has no partial check of its own, the smaller alignment mentioned in runLive passes partial events to appendEvent and never aggregates text chunks, unlike ADK Python #1648. With this change, neitherrunAsyncnorrunLivepasses it partial events.