feat: add persisted whole-chat summary with background generation#26657
Draft
jaaydenh wants to merge 2 commits into
Draft
feat: add persisted whole-chat summary with background generation#26657jaaydenh wants to merge 2 commits into
jaaydenh wants to merge 2 commits into
Conversation
Add a persisted chats.summary populated by a background generator after successful root-chat turns, delivered live via a new chat_summary_change watch event, with per-feature cost attribution and a deployment-wide summary-generation model override.
Docs preview📖 View docs preview for |
The UnknownContextReturns400 subtest hardcodes the valid override context list in its expected error. Adding the summary_generation context changed the message, so update both assertions.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds the persisted whole-chat summary that backs the chat summary popover. A new nullable
chats.summarycolumn is populated in the background after a successful root-chat turn and pushed to clients via a newchat_summary_changewatch event, so the popover readschat.summarystraight off the loadedChatwith no extra query.This is the data source for the popover and per-chat cost UI built in #26649; the popover can consume
chat.summaryonce this lands (the field is nullable, so merge order does not matter).How it works
chats.summary_generated_atfreshness marker. Generation reads compaction-aware history, renders it to a plain transcript, and asks for a 1-3 sentence summary via structured output.history_version(mirroringlast_turn_summary), so a background write racing a newer turn loses while worker lifecycle transitions cannot reject a fresh write.chat_messages.cost_sourcediscriminator. Summary spend is recorded as a hidden, soft-deleted accounting row tagged'summary'; the existing manual-title accounting row is tagged'title'. Ordinary turn spend staysNULL, so existing cost/usage queries are unchanged. The per-feature breakdown is left to the/costendpoint (out of scope, feat: add chat summary popover and per-chat cost endpoint #26649).summary_generationmodel override (mirroringtitle_generation); a configured-but-unusable override skips generation.Notes
000530addschats.summary,chats.summary_generated_at, andchat_messages.cost_source, and recreateschats_expandedto expose the new columns.Refs #26649