fix(chatd): improve compaction prompt to preserve forward momentum (coder#22989)

kylecarbs · web-flow · commit 53bfbf7c03cc · 2026-03-12T13:03:06.000Z
## Problem

The summarization prompt explicitly tells the model to **"Omit
pleasantries and next-step suggestions"** and the summary prefix frames
the compacted context as passive history: `Summary of earlier chat
context:`. After compaction mid-task, the model reads a factual recap
with no forward momentum, loses its direction, and either stops or asks
the user what to do.

## Research

I compared our compaction prompt against several other agents:

| Agent | Key Pattern |
|---|---|
| **Codex** | Prompt says *"Include what remains to be done (clear next
steps)"*. Prefix: *"Another language model started to solve this
problem..."* |
| **Mux** | Includes *"Current state of the work (what's done, what's in
progress)"* + appends the user's follow-up intent |
| **Continue** | *"Make sure it is clear what the current stream of work
was at the very end prior to compaction so that you can continue exactly
where you left off"* |
| **Copilot Chat** | Dedicated sections for *Active Work State*, *Recent
Operations*, *Pre-Summary State*, and a *Continuation Plan* with
explicit next actions |

**Every other major agent explicitly preserves forward intent and
in-progress state.** Coder was the only one telling the model to omit
next steps.

## Changes

**Summary prompt:**
- Removes `Omit next-step suggestions`
- Adds structured `Include:` list with explicit items for in-progress
work, remaining work, and the specific action being performed when
compaction fired
- Frames the operation as `context compaction` (matching Codex's
framing)

**Summary prefix:**
- Old: `Summary of earlier chat context:`
- New: `The following is a summary of the earlier conversation. The
assistant was actively working when the context was compacted. Continue
the work described below:`

The prefix is the first thing the model reads post-compaction — framing
it as an active handoff with an explicit "Continue" directive primes the
model to resume work rather than wait.
diff --git a/coderd/chatd/chatloop/compaction.go b/coderd/chatd/chatloop/compaction.go
@@ -17,13 +17,26 @@ const (
 	minCompactionThresholdPercent     = int32(0)
 	maxCompactionThresholdPercent     = int32(100)
 
-	defaultCompactionSummaryPrompt = "Summarize the current chat so a " +
-		"new assistant can continue seamlessly. Include the user's goals, " +
-		"decisions made, concrete technical details (files, commands, APIs), " +
-		"errors encountered and fixes, and open questions. Be dense and factual. " +
-		"Omit pleasantries and next-step suggestions."
-	defaultCompactionSystemSummaryPrefix = "Summary of earlier chat context:"
-	defaultCompactionTimeout             = 90 * time.Second
+	defaultCompactionSummaryPrompt = "You are performing a context compaction. " +
+		"Summarize the conversation so a new assistant can seamlessly " +
+		"continue the work in progress.\n\n" +
+		"Include:\n" +
+		"- The user's overall goal and current task\n" +
+		"- Key decisions made and their rationale\n" +
+		"- Concrete technical details: file paths, function names, " +
+		"commands, APIs, and configurations\n" +
+		"- Errors encountered and how they were resolved\n" +
+		"- Current state of the work: what is DONE, what is IN PROGRESS, " +
+		"and what REMAINS to be done\n" +
+		"- The specific action the assistant was performing or about to " +
+		"perform when this summary was triggered\n\n" +
+		"Be dense and factual. Every sentence should convey essential " +
+		"context for continuation. Do not include pleasantries or " +
+		"conversational filler."
+	defaultCompactionSystemSummaryPrefix = "The following is a summary of " +
+		"the earlier conversation. The assistant was actively working when " +
+		"the context was compacted. Continue the work described below:"
+	defaultCompactionTimeout = 90 * time.Second
 )
 
 type CompactionOptions struct {
