feat: surface agent_description README frontmatter to agent template tools#26212
feat: surface agent_description README frontmatter to agent template tools#26212johnstcn wants to merge 21 commits into
Conversation
Docs preview📖 View docs preview for |
|
/coder-agents-review |
|
Chat: Spend limit reached | View chat deep-review v0.7.1 | Round 5 | Last posted: Round 5, 17 findings (1 P1, 2 P2, 9 P3, 5 Nit), REQUEST_CHANGES. Review Finding inventoryFindings
Contested and acknowledgedCRF-8 (P3, coderd/util/frontmatter/frontmatter.go:68) - Second fence-splitter implementation
Round logRound 1Netero-only. 1 P2. Reviewed against aba9407..c8a2b29. Round 2Panel (15 reviewers). CRF-1 addressed. 1 P2, 6 P3, 5 Nit new. Reviewed against aba9407..d6f3360. Round 3Panel (10 reviewers). CRF-2 through CRF-13 addressed. CRF-8 contested, panel closed (7/7). 2 P3 new. Reviewed against aba9407..adcf6f4. Round 4Churn guard: PROCEED. CRF-14 and CRF-15 addressed. Netero: clean. 0 open findings. All 15 findings resolved. APPROVE. Round 5Panel (9 reviewers). Post-approval re-review after scope expansion (list_templates surfacing). 1 P1, 1 P3 new. Reviewed against 78a6ec2..f5197a4. About deep-reviewCRF = Coder Review Finding (P0-P4, Nit, Note)
|
There was a problem hiding this comment.
First-pass review (Netero). This is a mechanical-checks-only round; the full review panel has not yet reviewed this PR. One P2 finding below. The panel will review after this is addressed.
The approach is clean: a lightweight frontmatter parser that replaces hugo, good test coverage (1.4:1 test-to-production ratio), correct reuse of existing coderstrings.Truncate, and the chatd surface handles the version fetch best-effort as it should.
"The chatd surface handles the same fetch best-effort, and its comment explicitly states 'a missing or unreadable version must not fail read_template.' The same reasoning applies here."
🤖 This review was automatically generated with Coder Agents.
|
/coder-agents-review |
There was a problem hiding this comment.
Panel review (15 reviewers). CRF-1 addressed cleanly in d6f3360; both surfaces now use the best-effort pattern. 1 P2, 6 P3, 5 Nit.
The overall design is solid: a lightweight frontmatter parser that replaces Hugo, shared typed schema across three consumers, correct reuse of coderstrings.Truncate, strong test coverage (1.4:1 test-to-production ratio), and no scope drift. The examplegen refactor produces byte-identical output. Authorization paths are correct in both surfaces. The approach is proportional to the problem.
"This is Texture Surprise: the parser presents a tolerant face (handles CRLF, leading blank lines, unknown keys) but hides a strict fence match that rejects a common real-world variation." (Hisoka)
Notes from the panel worth preserving: the extra TemplateVersion API round-trip per coder_get_template is acceptable for the detail path but worth watching if more version data is needed later. The detail-path-only placement means list_templates still has limited routing signal when many templates share similar short descriptions; this is an explicit, reasonable tradeoff.
🤖 This review was automatically generated with Coder Agents.
|
/coder-agents-review |
There was a problem hiding this comment.
Re-review (10 reviewers). All 12 prior findings addressed or resolved. CRF-8 (second fence-splitter) closed by panel vote (7/7 accept the author's defense). 2 new P3 findings.
All R2 fixes verified: fence matching handles trailing whitespace and BOM (CRF-2), error assertions now distinguish paths (CRF-3), missing-version regression test added for chattool (CRF-4), comments trimmed to invariants (CRF-5/6/7), subtests added (CRF-9), examples use >- format (CRF-10), naming and doc fixes (CRF-11/12/13). The fixup commits are clean and introduce no regressions.
"The maintenance cost of two simple, well-tested implementations is lower than the coupling cost of a shared abstraction." (Netero, on CRF-8)
Dropped from R3 panel: Gon P2 on Parse doc comment (frontmatter.go:34) dropped because Leorio praised this exact comment in R2 as exemplary; the caller-side error-handling guidance is actionable, not bloat. Bisky P3 on missing >- test dropped because it would test yaml.Unmarshal, not PR code. Bisky P3 on toolsdk CRF-4 gap dropped to Note; the author's explanation (concrete client, not an interface) is valid and the guard pattern is verified via the chattool test.
🤖 This review was automatically generated with Coder Agents.
|
/coder-agents-review |
There was a problem hiding this comment.
All 15 findings resolved across 4 rounds (2 P2, 8 P3, 5 Nit; 1 contested and closed by panel vote 7/7).
The PR delivers a clean, migration-free solution for agent template routing. A lightweight frontmatter parser replaces the Hugo dependency (117 fewer go.sum lines), a shared typed schema serves three consumers without drift, and both agent tool surfaces degrade gracefully on version-fetch failure. Test coverage is strong at 1.6:1 test-to-production ratio, including regression tests for the best-effort error path and hostile-input panic protection.
Dropped from R4: Bisky P3 re-raising CRF-4 on the toolsdk best-effort path. Verified against the code: TemplateVersionRichParameters (line 770) and TemplateVersionPresets (line 774) return hard errors before the best-effort TemplateVersion fetch is reached. Any test using a non-existent ActiveVersionID would fail at the parameters step, not test the best-effort path. The author's R2 explanation is correct; testing this path requires either an interface refactor or a partial-failure mock, both out of scope.
"I tried to build a case against this and could not." (Pariston, R2 and R4)
🤖 This review was automatically generated with Coder Agents.
There was a problem hiding this comment.
Pull request overview
This PR implements CODAGT-447 by surfacing a new agent_description README frontmatter field (stored in template_versions.readme) to agent-facing template detail tools, enabling longer agent-only template routing context without a DB migration.
Changes:
- Added a lightweight
coderd/util/frontmatterparser andAgentDescriptionhelper that trims and truncates to 2048 runes. - Surfaced
agent_description(best-effort) in agent detail tools:coder_get_template(toolsdk) andread_template(chattool), with tests. - Migrated
scripts/examplegenoff Hugo frontmatter parsing, removed Hugo from dependencies, and updated example template READMEs + docs to describeagent_description.
Reviewed changes
Copilot reviewed 14 out of 15 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| scripts/examplegen/main.go | Switches example parsing to the shared frontmatter parser and removes Hugo frontmatter usage. |
| go.mod | Drops github.com/gohugoio/hugo and related indirect deps; updates indirect dependency set after migration. |
| go.sum | Removes Hugo-related transitive sums consistent with the dependency removal. |
| examples/templates/kubernetes/README.md | Adds agent_description frontmatter for agent routing context. |
| examples/templates/gcp-linux/README.md | Adds agent_description frontmatter for agent routing context. |
| examples/templates/docker/README.md | Adds agent_description frontmatter for agent routing context. |
| examples/templates/aws-linux/README.md | Adds agent_description frontmatter for agent routing context. |
| docs/ai-coder/agents/platform-controls/template-optimization.md | Documents agent_description, its purpose, and an example snippet. |
| docs/ai-coder/agents/getting-started.md | Links to and recommends using agent_description when 128 chars isn’t sufficient. |
| codersdk/toolsdk/toolsdk.go | Extends template detail tool response with agent_description fetched from active version README (best-effort). |
| codersdk/toolsdk/toolsdk_test.go | Adds coverage for surfaced/absent/truncated agent_description behavior and JSON omission. |
| coderd/x/chatd/chattool/readtemplate.go | Adds best-effort agent_description inclusion in read_template output. |
| coderd/x/chatd/chattool/readtemplate_test.go | Adds coverage for surfaced/absent/truncated/missing-version behavior in read_template. |
| coderd/util/frontmatter/frontmatter.go | Introduces the shared frontmatter parser + AgentDescription truncation helper. |
| coderd/util/frontmatter/frontmatter_test.go | Adds comprehensive parsing and truncation tests (CRLF/BOM/whitespace/fence variations). |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
There was a problem hiding this comment.
This is the biggest question mark to me. I get the reason for removing the Hugo dependency, but it might be worth investigating a lighter-weight alternative?
DanielleMaywood
left a comment
There was a problem hiding this comment.
I'm definitely open for this approach. I'm not the biggest fan of the hand-rolled frontmatter extraction (from an edge case point of view) but maybe I'm too used to the JS ecosystem where you just pull a dependency in for everything
|
/coder-agents-review |
There was a problem hiding this comment.
Post-approval re-review (9 reviewers) after scope expansion: chatd list_templates now surfaces agent_description. 1 P1, 1 P3.
The new list_templates surfacing has a blocking RBAC issue (CRF-16): the batched GetTemplateVersionsByIDs call requires ResourceSystem, which only the site-level owner role grants. All other users (members, org admins, template admins) silently get no agent_description in list_templates. The feature works in read_template (which uses template-scoped GetTemplateVersionByID), so there is a behavioral split between the two tools.
The rest of the PR remains solid. All 15 prior findings stay resolved. The frontmatter parser, toolsdk surface, readtemplate surface, examplegen refactor, docs, and example templates are clean.
"It looks correct at a glance, and the test passes. But the test uses a raw database.Store from dbtestutil.NewDB, which has no dbauthz wrapper." (Hisoka)
🤖 This review was automatically generated with Coder Agents.
|
/coder-agents-review |
|
@johnstcn ⛔ This review has reached its per-chat spend limit ($103.53 / $100.00). Further review rounds are paused. To raise the limit and continue, comment: This is a per-chat budget, separate from any account-level usage limit.
|
Implements CODAGT-447 without a database migration, so it can be backported to a patch release.
Agents pick templates from
templates.description(VARCHAR(128)), which is too short. Rather than add a column (#25978, needs a migration), this surfaces a newagent_descriptionREADME frontmatter key from the existingtemplate_versions.readmecolumn to the agent template-detail tools.coderd/util/frontmatter: a lightweight, hugo-free parser with a typed, locked key set and anAgentDescriptionhelper (trims, truncates to 2048 runes).agent_descriptionto the agent template tools, best-effort so a missing or unreadable version never fails the tool:read_templateandlist_templateson the Coder Agents (chatd) surface, andcoder_get_templateon the public MCP surface. The chatdlist_templatesfetches each page template's active version with the template-scopedGetTemplateVersionByID(same access path asread_template, bounded by the page size of 10) and returns the full description so an agent can choose in a single call.scripts/examplegento the shared parser;examples.gen.jsonis byte-identical and the hugo dependency is dropped (go.sum-117 lines).agent_descriptionto the Docker, Kubernetes, AWS, and GCP example templates and document it underdocs/ai-coder/.The longer text (up to 2048 runes) is agent-only and never shown in the dashboard.
Decision log
abstractcolumn (feat: add template abstracts for agents #25978): the column approach needs a migration and cannot be backported to a patch; the customer issue warrants a backport.agent_descriptionkey, not reuseddescription: the dashboard starter-import flow prefillsCreateTemplateRequest.Descriptionfrom frontmatterdescription(bound bylt=128); overloading it would break that path.separateFrontmatterpattern incoder/registry.goldmark-frontmatterandadrg/frontmatter; both cost more dependency weight than the ~50 lines they would replace, and neither cleanly returns the verbatim body the byte-identicalexamples.gen.jsonguard needs.goldmark-frontmatteris a goldmark block parser that exposes only the parsed frontmatter (not the body), pulls inyuin/goldmarkplus a newBurntSushi/toml, and its fence handling would regress cases we hardened (trailing-whitespace fences, leading blank lines, BOM).adrg/frontmatteradds a newBurntSushi/tomldependency and parses withyaml.v2, a second YAML implementation alongside theyaml.v3used elsewhere. (goldmark-frontmatteris also not what Hugo uses for frontmatter; Hugo uses its ownparser/pageparser.)list_templatesandread_templateboth surface the fullagent_description;list_templatesreads each page template's active version via the template-scopedGetTemplateVersionByID(the same RBAC path asread_template, bounded by the page size of 10), enabling single-call template selection. A batchedGetTemplateVersionsByIDswas avoided because it requiresResourceSystemand would silently dropagent_descriptionfor every non-owner user. It is returned untruncated on purpose, truncating the rich description would defeat its purpose, and the maxed-out-descriptions-at-scale case is addressed by template scoping rather than truncation. The publiccoder_list_templatesis left unchanged to avoid adding a field to the codersdk API in this patch; it can be added later if requested.coder/registryandcoder/registry-serveruse non-strictyaml.Unmarshal, so the new key is ignored there.