Skip to content

feat: render chat summaries as a headline plus bullets - #28013

Draft
jaaydenh wants to merge 7 commits into
mainfrom
chat-summary-7v0c
Draft

feat: render chat summaries as a headline plus bullets#28013
jaaydenh wants to merge 7 commits into
mainfrom
chat-summary-7v0c

Conversation

@jaaydenh

Copy link
Copy Markdown
Contributor

The whole-chat summary in the Summary tab rendered as a single prose blob. Three layers each independently prevented structure: the generation prompt ended with "No markdown, lists, headings, code fences, or surrounding quotes", the normalizer collapsed every newline into a space via strings.Join(strings.Fields(text), " "), and the panel rendered the result into one plain <p>. Validation was also looser than the prompt asked: the prompt requested 1-3 sentences while summaryMaxRunes = 1000 and summaryMaxSentences = 6 permitted roughly 15 unbroken lines in a 448px column.

Generation now returns a structured headline plus 2-4 bullets, which a shared serializer renders as markdown into the existing summary TEXT column. No migration is needed.

Investigated the flaky CI job in `coderd/x/chatd` and landed a fix.

- Traced the failure to a cache-layer race in `chatd.go`
- Added a regression test covering the race
- Opened PR #26649

Both producers go through formatChatSummaryMarkdown so the subagent path cannot bypass the format. Subagent summaries are extracted from an existing report rather than generated, so they pass nil bullets and the serializer returns their snippet unchanged, leaving subagentReportSummarySnippet untouched.

Validation moved onto the struct, before serialization. Applying the old sentence cap to serialized markdown would have silently stopped working, since bullets routinely omit trailing punctuation. Normalization is per-field rather than a change to the shared normalizeShortTextOutput, which has a second caller in title generation and strips trailing backticks; the new normalizeSummaryField preserves them so a headline ending in an inline code span keeps a balanced pair.

Const Old New
summaryMaxRunes 1000 600 (serialized total)
summaryMaxSentences 6 removed
summaryHeadlineMaxRunes - 200
summaryHeadlineMaxSentences - 2
summaryBulletMaxRunes - 160
summaryMinBullets / summaryMaxBullets - 2 / 4

On the frontend the panel renders through InlineMarkdown, whose allowlist plus unwrapDisallowed structurally bounds what a summary can render. Response was rejected because it permits full GFM including tables, mermaid diagrams, and a diff-viewer code block, none of which belong in a 360-480px column. The p override is required because InlineMarkdown renders p as a bare fragment, which would run the headline into the bullet list. ol is allowlisted alongside ul because a legacy prose summary starting with 1. otherwise emits li elements with no list parent.

Content is bounded with max-h-48 plus overflow-hidden and a Show more toggle, rather than line-clamp, which relies on display: -webkit-box and clamps unreliably across nested block children such as <ul><li>. This mainly protects the legacy prose summaries already stored at up to 1000 runes, and keeps the Created/Updated/Cost rows above the fold.

Summaries regenerate after three completed turns, so legacy prose and new bullets coexist for a while. Both render correctly and no backfill is planned.

Out of scope: last_turn_summary in the sidebar and search rows, the chat_summarized compaction card, chat title generation, and turn status labels.

The whole-chat summary shown in the Summary tab rendered as a single
prose blob. Three layers each independently prevented structure: the
generation prompt banned all markdown, the normalizer collapsed every
newline into a space, and the panel rendered the result into one plain
<p>. Validation was also looser than the prompt asked, permitting up
to 1000 runes and six sentences in a 448px column.

Generation now returns a structured headline plus 2-4 bullets, which a
shared serializer renders to markdown stored in the existing summary
column. No migration is needed.

Both summary producers go through formatChatSummaryMarkdown so the
subagent path cannot bypass the format. Subagent summaries are
extracted from an existing report rather than generated, so they pass
no bullets and the serializer returns their snippet unchanged.

Validation moved onto the struct, before serialization. Applying the
old sentence cap to serialized markdown would have silently stopped
working, since bullets routinely omit trailing punctuation.
Per-field normalization also preserves backticks, which the shared
normalizeShortTextOutput strips, so a headline ending in an inline
code span keeps a balanced pair.

The panel renders the markdown through InlineMarkdown with a narrow
allowlist, bounded to max-h-48 with a Show more toggle. Legacy prose
summaries continue to render; summaries regenerate after three
completed turns, so both shapes coexist without a backfill.
@jaaydenh

Copy link
Copy Markdown
Contributor Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: ede189b76a

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

const observer = new ResizeObserver(measure);
observer.observe(content);
return () => observer.disconnect();
}, [measure, isExpanded]);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Remeasure overflow when the summary changes

When a collapsed summary exactly fills the 192px bound, scrollHeight === clientHeight, so the toggle is hidden. If a chat-summary pubsub update then replaces it with longer text, the element remains clamped at 192px; ResizeObserver sees no box-size change, and this effect does not rerun because summary is absent from its dependencies. The updated summary is therefore clipped with no Show more control until another resize or remount, so remeasure whenever summary changes.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 634f890. Confirmed reachable: mergeWatchedChatIntoCachedChat writes summary into the cached chat (site/src/api/queries/chats.ts:570), so the panel updates in place without remounting.

Rather than adding summary to the dependency array, the observer now watches an inner unclamped element instead of the clamped box, and measurement still happens on the clamped box. The inner element height tracks content and its width tracks the resizable panel, so both content swaps and resizes re-evaluate the toggle. This also keeps Biome useExhaustiveDependencies happy, which rejects summary as a dependency because the effect body never reads it.

One caveat on test coverage: SummaryReplacedInPlace covers the in-place update path, but it does not fail against the old code. The only states that distinguish the two implementations require scrollHeight === clientHeight exactly, i.e. content exactly 192px tall, and pinning that deterministically needs a font-metric-dependent fixture that would be flaky. I opted for the non-brittle story plus the structural fix instead.

Comment on lines +150 to +152
className={`overflow-hidden font-sans text-sm font-normal leading-6 text-content-primary ${
isExpanded ? "" : "max-h-48"
}`}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Remove clipped links from the collapsed tab order

When a long summary contains a Markdown link below the eight-line clamp, overflow-hidden only clips it visually; InlineMarkdown still mounts the anchor, so keyboard and screen-reader users can reach invisible content before selecting Show more. Hide clipped interactive descendants from the accessibility tree and tab order while collapsed, or use a collapse primitive that handles focus.

AGENTS.md reference: site/AGENTS.md:L199-L204

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 634f890. Confirmed the guidance applies: site/AGENTS.md:199-204 requires visually hidden interactive elements to leave the tab order and accessibility tree, and overflow-hidden only clips visually.

Rather than managing focus for clipped descendants, I removed the interactive element class entirely: the a component now renders its children without an anchor, so no anchor is ever mounted and there is nothing to clip from the tab order. This also matches the rationale already in the PR for choosing InlineMarkdown over Response, which is to structurally bound what a summary can render. Summaries are generated descriptive text, not navigation, and link text is preserved.

Covered by the LinksRenderAsPlainText story, which asserts the link text renders while queryByRole("link") finds nothing.

const [isExpanded, setIsExpanded] = useState(false);
const [isOverflowing, setIsOverflowing] = useState(false);

const measure = useCallback(() => {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Let React Compiler memoize the measurement callback

This component is under src/pages/AgentsPage, where React Compiler already memoizes callbacks and repository guidance explicitly prohibits adding useCallback. Keeping this hook adds redundant dependency bookkeeping and works against the compiler-enabled convention; make measure a plain closure instead.

AGENTS.md reference: site/AGENTS.md:L243-L250

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 634f890. Verified the citation: site/AGENTS.md:243-250 explicitly says not to add useMemo, useCallback, or memo() under src/pages/AgentsPage/ because React Compiler handles memoization there. This was a straight violation on my part.

useCallback is gone and the measurement closure is now defined inside the effect, which also removes it from the dependency array rather than trading one dependency for another.

Observe overflow on an inner unclamped element rather than the clamped
box. The clamped box stops growing at its max height, so a summary that
arrives via an in-place cache update while the box is already pinned
there resized nothing and stayed clipped with no toggle.

Drop useCallback, which site/AGENTS.md prohibits under
src/pages/AgentsPage since React Compiler already memoizes callbacks
there. The measurement closure now lives inside the effect.

Render link text without an anchor. Clipping below the collapsed bound
is visual only, so a mounted anchor stayed reachable by keyboard and
screen readers while invisible.

Copy link
Copy Markdown
Contributor Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 634f89079a

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

return "", result.Usage, err
}
return summary, result.Usage, nil
return formatChatSummaryMarkdown(summary.Headline, summary.Bullets), result.Usage, nil

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Update the obsolete sentence-summary contract

This return now serializes a headline followed by Markdown bullets, but generateChatSummary is still documented immediately above as generating a "1-3 sentence whole-chat summary." That stale contract can mislead future callers and tests about the persisted format; update the function comment to describe the headline-plus-bullets output.

AGENTS.md reference: AGENTS.md:L180-L183

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in dff5f06. Correct, that comment was stale.

The same change had also invalidated a second comment you did not flag: coderd/x/chatd/chatd.go:4729 bounded the extracted subagent report snippet "near the 1-3 sentence generated summaries that root chats get". The snippet caps themselves are unchanged at 300 runes / 3 sentences, which now corresponds to the headline rather than a whole summary, so I updated that wording too.

Swept for other references to the old contract and found none; ARCHITECTURE.md documents no summary format contract, so it needed no change.

generateChatSummary now returns a headline plus bullets, but its doc
comment still described a 1-3 sentence summary. The subagent snippet
bound in chatd.go referenced the same obsolete contract.

Copy link
Copy Markdown
Contributor Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: dff5f061b7

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +153 to +155
className={`overflow-hidden font-sans text-sm font-normal leading-6 text-content-primary ${
isExpanded ? "" : "max-h-48"
}`}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Allow long identifiers to wrap inside the summary

When a generated summary contains an unbroken file path, function name, or error token wider than the resizable panel, this unconditional overflow-hidden clips it horizontally. The overflow check only compares scrollHeight and clientHeight, so horizontal clipping does not expose the Show more control, and expansion would retain overflow-hidden anyway. This is particularly reachable because the new generation prompt explicitly preserves identifiers and wraps them in inline code; add an appropriate word-breaking rule to the summary or code elements.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 9af0084. Reproduced it first, and the mechanism was worse than clipping: because the wrapper uses items-start, the summary box sized itself to the max-content width of the identifier and escaped its column entirely rather than being clipped inside it. Measured in Chrome, the box rendered 623px wide inside a 368px column.

Added break-words [overflow-wrap:anywhere] plus w-full, matching the existing pattern in ChatElements/tools/AdvisorTool.tsx. overflow-wrap: anywhere is the part that matters, since it also feeds intrinsic min-content sizing, so the box resolves to the column width instead of the token width.

Covered by the LongIdentifierWraps story, which pins a 300px column, below the panel's 360px minimum. This one is a genuine red-green regression test: it failed before the fix with expected 639.296875 to be less than or equal to 317, and passes after, with the identifier wrapping across 3 lines inside the column.

…ummaries

Summaries preserve identifiers verbatim and wrap them in backticks, so a
single token can be wider than the panel with no natural break
opportunity. The summary box sized itself to that token, escaping its
column, and the clipped remainder was unreachable because the toggle
only responds to vertical overflow.

Copy link
Copy Markdown
Contributor Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 9af0084547

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +153 to +155
await expect(identifier.getBoundingClientRect().right).toBeLessThanOrEqual(
column.getBoundingClientRect().right + 1,
);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Replace the DOM-geometry assertion

FE10 prohibits DOM-geometry assertions in Storybook tests because pixel coordinates vary with fonts and rendering environments, making this story fail after harmless layout changes or behave inconsistently across CI and local browsers. Keep the play assertions semantic and leave the exact wrapping layout to visual regression coverage instead.

AGENTS.md reference: site/AGENTS.md:L5-L7

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed and fixed in 90f83d6. FE10 is explicit that DOM-geometry assertions are prohibited, and the flakiness rationale applies directly here: my assertion depended on font metrics, which is the same reason I declined to write a font-metric-dependent fixture for the exact-fill case two rounds ago. It would have been inconsistent to defend it now.

The story keeps its pinned 300px column so it still serves as a visual regression fixture for the wrapped layout, and the play assertion is now semantic. The fix itself is unchanged.

Worth noting the tradeoff this leaves: the wrapping guarantee no longer has an executable regression test, so it rests on Pixel snapshots. That is what the rule prescribes, and I am not arguing against it, but a future refactor that drops [overflow-wrap:anywhere] will surface as a snapshot diff rather than a red test.

Also swept the rest of the stories in this PR for FE10 violations and found no other geometry, querySelector, class-name, or data-testid usage.

…rom summary story

FE10 prohibits DOM-geometry assertions because pixel values vary with
font rendering across environments. The long-identifier story keeps its
narrow column as a visual regression fixture and asserts semantically
instead.

Copy link
Copy Markdown
Contributor Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 90f83d6752

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +144 to +145
const observer = new ResizeObserver(measure);
observer.observe(content);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Throttle overflow measurement during panel resize

In the AgentsPage resizable right panel, every width update resizes content and invokes this observer, so measure repeatedly reads layout and calls setIsOverflowing throughout the drag even when the boolean remains unchanged. Throttle or animation-frame-schedule the measurement, with cancellation during cleanup, to avoid adding high-frequency state work to panel resizing.

AGENTS.md reference: site/AGENTS.md:L256-L259

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed in 32cd37b, though not with requestAnimationFrame, and I want to flag the reasoning rather than quietly deviate.

The underlying concern is right: a drag resizes the element continuously, and site/AGENTS.md:256-259 explicitly anticipates the "React bails out on identical state" counter-argument by pointing out the handler still runs.

But the specific requestAnimationFrame remedy would be a no-op here. ResizeObserver callbacks are delivered during the update-the-rendering steps, so they already fire at most once per animation frame; wrapping one in requestAnimationFrame defers the same work by a frame instead of doing less of it, and adds a frame of staleness. Note also that the layout reads happen inside the observer callback, after layout and before paint, so they do not force a synchronous reflow.

So I took the rule's other branch and used a trailing debounce (100ms). That is strictly better than a frame callback for this case: while a drag is in flight the handler only resets a timer and performs no layout reads at all, and the verdict is recomputed once the size settles. The frequently-changing state is also already isolated in the small ChatSummaryBody child rather than the parent, per the preceding bullet at L251-255.

Cleanup clears the pending timer alongside disconnecting the observer. All 23 stories still pass, including SummaryReplacedInPlace, which now has to survive the debounce.

If you would prefer the doc to be explicit that observer callbacks are already frame-aligned and debouncing is the appropriate choice there, I am happy to propose that edit to .claude/docs/FRONTEND_PATTERNS.md separately rather than expanding this PR.

…surement

Dragging the right panel resized the summary every frame, and each
observer callback read layout and set state even though the overflow
verdict is a boolean that flips at most a couple of times per drag.
Waiting for the resize to settle skips those reads entirely while a drag
is in flight.

Copy link
Copy Markdown
Contributor Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 32cd37b6e0

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

const ChatSummaryBody: FC<ChatSummaryBodyProps> = ({ summary }) => {
const clampRef = useRef<HTMLDivElement>(null);
const contentRef = useRef<HTMLDivElement>(null);
const [isExpanded, setIsExpanded] = useState(false);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Reset expansion when the displayed chat changes

When a user expands chat A and then navigates directly to chat B, React Router reuses AgentChatPageView, and ChatSummaryPanel receives the new chatId without remounting ChatSummaryBody. Consequently, this local isExpanded state carries over to B; because the measurement effect exits while expanded, B displays its entire summary with a misleading “Show less” button even when the new summary fits, until the user clicks it. Key the summary body by chat or reset its disclosure state when the displayed summary identity changes.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Valid, fixed in 1b6328d, though the mechanism is not quite the one described, and the difference is what makes it reproducible.

I wrote the regression test first and it passed against the unfixed code. The reason is that chat(chatId) sets no keepPreviousData/placeholderData (site/src/api/queries/chats.ts:1139-1142), so on a cache miss chatQuery.data is undefined for at least one render, the panel renders null, and that unmounts ChatSummary and discards the disclosure state as a side effect.

The leak needs a warm cache: a previously visited chat resolves synchronously on the first render with the new key, chatData is never undefined, the same summary instance is reused, and isExpanded carries over. So the bug is real, just gated on the chat already being cached, which is the common case when navigating back and forth.

Reproduced by extending the story to warm both chats before the switch that matters. It fails against the unfixed code with expected document not to contain element, found <button ... Show less, and passes with key={chatId} on ChatSummary.

I keyed on chatId rather than on the summary text deliberately: an in-place summary update for the same chat should preserve what the user opened, while a different chat should start collapsed.

…r chat

Switching chats swaps the panel's props instead of remounting it. An
uncached chat happened to reset the expanded state, because the panel
renders nothing while loading and that unmounts the summary, but a cached
chat resolves synchronously and kept the previous chat's expanded state,
offering to collapse a summary that may not overflow at all.

Copy link
Copy Markdown
Contributor Author

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. 🚀

Reviewed commit: 1b6328dccf

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant