feat: add agent runtime hour license claims and entitlement feature#27459
feat: add agent runtime hour license claims and entitlement feature#27459jaaydenh wants to merge 1 commit into
Conversation
Licenses can now carry three claims for agent runtime hours: agent_runtime_hours_allocation, agent_runtime_hours_limit_soft, and agent_runtime_hours_limit_hard (unit: hours). They surface as the new usage-period feature agent_runtime_hours in GET /api/v2/entitlements, where limit carries the allocation and the new optional soft_limit and hard_limit fields carry the thresholds. Invalid claim combinations (soft/hard without allocation, negative allocation, soft outside 0 <= soft < allocation, hard < allocation) reject the entire license, following the validateClaims precedent. Soft and hard limits are not comparison inputs in Feature.Compare; they ride along with whichever license wins (newest iat, existing behavior). Old servers ignore all three claims since none of them is a feature name, protecting rollout of licenses minted with the new claims. Part of CODAGT-837. The claim name constants defined here are the canonical contract for github.com/coder/license (X1).
Docs previewCheck off each page once it's been reviewed. If a page changes in a later push, its checkbox clears automatically so it gets a fresh look. Pages not yet wired into the docs navigation aren't listed here. |
|
/coder-agents-review |
|
Chat: Review posted | View chat Review history
deep-review v0.9.0 | Round 1 | Last posted: Round 1, 9 findings (4 P3, 1 P4, 4 Nit), COMMENT. Review Finding inventoryFinding inventory - PR #27459Findings
Round logRound 1Panel (18: Netero first-pass + Bisky, Hisoka, Mafu-san, Mafuuu, Pariston, Ging-go, Gon, Leorio, Kurapika, Knov, Ryosuke, Chopper, Melody, Komugi, Robin, Luffy, and wildcards Meruem, Razor). Netero clean at mechanical floor (P3 + Note only), so panel proceeded. 4 P3, 1 P4, 4 Nit. No P0/P1; event COMMENT. Reviewed against 671173b..3463d7e. Design-level Notes recorded but not posted as line findings (covered in review body): whole-license rejection blast radius on a bad claim combo (author's confirmed reject-license stance, fail-loud; Pariston/Hisoka/Mafuuu), claim-name string contract drift with the licensor X1 (Pariston/Hisoka), About deep-reviewCRF = Coder Review Finding (P0-P4, Nit, Note)
|
|
Codex Review: Didn't find any major issues. 👍 Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
There was a problem hiding this comment.
Clean, well-scoped groundwork. Three flat license claims (agent_runtime_hours_allocation, _limit_soft, _limit_hard) map onto one usage-period feature agent_runtime_hours, mirroring the managed-agent path rather than inventing a new mechanism. The panel verified the load-bearing invariants hold: validation runs on every parse path (ParseClaims and ParseClaimsIgnoreNbf both route through validateClaims), the raw feature-name claim is skipped so an unvalidated allocation cannot win Feature.Compare precedence, soft/hard ride along with the winning license via AddFeature whole-struct replacement (never merged), and the feature is correctly excluded from Premium/Enterprise defaults. Bisky on the tests: "this test suite is lovely, and this time it's real." Test density is 83.4% and the assertions check real numbers and wire keys, not mocks echoing themselves.
Severity count: 4 P3, 1 P4, 4 Nit. No P0/P1, so this is a COMMENT, not a request for changes. The strongest finding is a validation gap this PR itself introduces (allocation=0 accepting a positive hard ceiling); the rest are coverage, maintainability, naming, and doc-generation edges.
Design observations the human should weigh, none blocking and none needing a code change here:
- Whole-license rejection blast radius: a single malformed threshold rejects the entire license (400 on upload;
HasLicense=falseand every feature dropped for a stored license). This is the confirmed reject-license stance (decision 2) and fails loud rather than shipping a wrong limit; no field-minted license can carry these brand-new claim names, so the practical window is a future licensor bug. Worth confirming the blast radius is acceptable versus degrading to ignoring only the malformed feature. - Cross-repo name contract: the claim constants are declared canonical for
github.com/coder/license(X1) but matched by exact string with no mechanical link. A rename or typo on either side falls through the unknown-claim tolerance and the feature silently vanishes. Worth generating or checking the licensor's bounds against these same constants when X1 lands, rather than hand-duplicating. soft_limit/hard_limitare served onGET /api/v2/entitlements, which is registered outsideapiKeyMiddlewareand is unauthenticated. Same disclosure class as thelimit/enabled/entitlementalready exposed there, so nothing new, but the enforcement follow-ups should not treat these thresholds as secret.- soft/hard are inert data today (no reader in production; B4/C1/F1 are follow-ups). Correct scope, flagged so nobody reads three numbers off the entitlements response and assumes something enforces them.
FeatureAgentRuntimeHoursis added toUsesLimit()but that arm never fires (theUsesUsagePeriod()case precedes it). Harmless and consistent withFeatureManagedAgentLimit, which sets both; removing it would change nothing.
🤖 This review was automatically generated with Coder Agents.
| if hasSoft && (soft < 0 || soft >= allocation) { | ||
| return ErrInvalidAgentRuntimeSoftLimit | ||
| } | ||
| if hasHard && hard < allocation { |
There was a problem hiding this comment.
P3 [CRF-2] validateAgentRuntimeHours accepts allocation=0 with a positive hard_limit, minting a feature that reports Enabled: false yet advertises a non-zero enforcement ceiling. (Knov P3, Mafuuu P3)
With
allocation=0this reduces tohard < 0, so anyhard >= 0is accepted, includinghard=1000.
The soft check is symmetric (soft < 0 || soft >= allocation forbids any soft when allocation is 0), but hard is not: hard < allocation is vacuous at zero. The loop then builds and stores the feature with Enabled=false and HardLimit set. No reader consumes HardLimit today, so there is no runtime consequence now (hence P3), but this PR is where the contract is established, and the C1 admission gate is the consumer that would read a ceiling off a feature that is off. Either forbid soft and hard when allocation == 0 (symmetric with the missing-allocation case), or document that a hard ceiling on a zero allocation is intentional and define which of Enabled/Limit/HardLimit the future gate honors. ZeroAllocationWithZeroHard asserts allocation=0, hard=0 is valid, so if that stays intended, reject only hard > 0 at zero allocation. No test exercises the allocation=0, hard>0 combination.
🤖
| const ( | ||
| // ClaimAgentRuntimeHoursAllocation is the purchased runtime-hour | ||
| // allocation for the license term. It becomes the feature's Limit. | ||
| ClaimAgentRuntimeHoursAllocation = "agent_runtime_hours_allocation" |
There was a problem hiding this comment.
P3 [CRF-3] The rollout-safety invariant "none of the three claim names is a feature name" is load-bearing but unpinned by any test. (Chopper)
nothing fails if a future change registers one of the three claim strings as a feature name, or renames a constant to collide. The moment that happens, old deployments start interpreting a claim they were supposed to ignore, and the documented rollout guarantee breaks silently in the field
UnknownClaimsCompatibility uses invented names (future_feature_allocation) rather than the real claim constants, so it proves "unknown claims are ignored" but never asserts the real claim names are in fact unknown to FeatureNamesMap. One assertion closes it: require.NotContains(t, codersdk.FeatureNamesMap, codersdk.FeatureName(license.ClaimAgentRuntimeHoursAllocation)) for all three constants. This is the red test the next engineer touching feature registration would need.
🤖
| continue | ||
| } | ||
|
|
||
| // Agent runtime hours are encoded as up to three claims: the |
There was a problem hiding this comment.
P3 [CRF-4] The knowledge that agent_runtime_hours is composed of three claims is now encoded in three places instead of one decoder: validateAgentRuntimeHours, the remap/skip block here (300-317), and the inline soft/hard attachment in the UsesUsagePeriod case (345-354). (Ryosuke P3, Robin Note)
a claim added to the validator but not the decoder is silently ignored, the inverse rejects a license the decoder would have accepted
Consequence is maintenance drag and a silent-drift vector, not a runtime bug, and it does mirror the accepted managed-agent precedent, so this is a rough edge to weigh rather than a stop. Ryosuke's suggested collapse: lift claim-to-Feature decoding into a dedicated decodeAgentRuntimeHours(claims.Features) (codersdk.Feature, bool) that owns validation and soft/hard attachment together, called once from the loop. Robin notes a third such feature is the point at which this pattern is worth a table (claim-name to feature-name plus a soft/hard flag) instead of a fourth inline branch.
🤖
| // hard limit claims can be set with Feature using | ||
| // license.ClaimAgentRuntimeHoursLimitSoft and | ||
| // license.ClaimAgentRuntimeHoursLimitHard. | ||
| func (opts *LicenseOptions) AgentRuntimeHours(allocation int64) *LicenseOptions { |
There was a problem hiding this comment.
P3 [CRF-1] The new exported AgentRuntimeHours test builder has zero callers and is itself untested. (Netero)
Its sibling
ManagedAgentLimit(same file, line 237) has three production-test callers ... Every test added in this PR bypasses the builder and constructs the map directly
Because no test exercises it, a wrong mapping in the helper (for example pointing at the soft-limit claim) would ship undetected. Either delete it or use it in the new tests so it is covered. Assuming the usage-wiring follow-ups will call it is a deferral without a caller in this diff.
🤖
| } | ||
| // Agent runtime hours is a usage period feature and does not | ||
| // generate generic entitlement warnings. | ||
| if featureName == codersdk.FeatureAgentRuntimeHours { |
There was a problem hiding this comment.
P4 [CRF-5] The new warning-suppression branch runs but no test asserts its effect; the one test that reaches it skips the feature instead of checking the warning is gone. (Bisky)
Delete the production skip and every test still passes; the branch is executed for coverage but its behavior is unproven.
SingleLicenseNotEntitled enables every feature via all, so FeatureAgentRuntimeHours is Enabled and NotEntitled, exactly the state this skip suppresses, but the test adds the feature to its continue list rather than asserting the warning is absent. If the skip regresses, admins get a spurious "Agent Runtime Hours is enabled but your license is not entitled" warning. Assert require.NotContains on that warning string in a test where all enablements apply and no agent-runtime license is granted. Makes the PR bigger, not wrong, so P4.
🤖
| FeatureBoundary FeatureName = "boundary" | ||
| FeatureServiceAccounts FeatureName = "service_accounts" | ||
| FeatureAIGovernanceUserLimit FeatureName = "ai_governance_user_limit" | ||
| // AgentRuntimeHours is a usage period feature. It is never a license |
There was a problem hiding this comment.
Nit [CRF-6] Doc comment starts with AgentRuntimeHours, but the declared identifier is FeatureAgentRuntimeHours. (Gon, Leorio)
Go doc convention and revive's exported-comment rule want the comment to begin with the exact name. It reads // AgentRuntimeHours is a usage period feature; make it // FeatureAgentRuntimeHours is a usage period feature. This is the only const in the block that carries a doc comment, so the mismatch stands out and go doc renders it detached from the symbol.
🤖
| // Only certain features set these fields: | ||
| // - FeatureAgentRuntimeHours | ||
| SoftLimit *int64 `json:"soft_limit,omitempty"` | ||
| HardLimit *int64 `json:"hard_limit,omitempty"` |
There was a problem hiding this comment.
Nit [CRF-7] HardLimit has no doc comment of its own, so the generated API schema documents soft_limit but leaves hard_limit blank. (Meruem Nit, Leorio Note)
The shared doc block sits on SoftLimit (line 384), and swagger attaches a field description only to the field its comment immediately precedes, so docs/reference/api/schemas.md renders hard_limit with an empty description column. Add a one-line comment directly above HardLimit (for example, "HardLimit is the enforcement ceiling; see SoftLimit.") and regenerate with make gen.
🤖
| ErrMissingAccountType = xerrors.New("license must contain valid account type") | ||
| ErrMissingAccountID = xerrors.New("license must contain valid account ID") | ||
|
|
||
| ErrMissingAgentRuntimeAllocation = xerrors.Errorf("license has agent runtime limit claims but is missing the %s claim", ClaimAgentRuntimeHoursAllocation) |
There was a problem hiding this comment.
Nit [CRF-8] The four error vars drop Hours, breaking the naming pattern every other symbol for this feature follows. (Gon)
The feature is FeatureAgentRuntimeHours, the claims are ClaimAgentRuntimeHours*, the validator is validateAgentRuntimeHours, but the errors are ErrMissingAgentRuntimeAllocation, ErrInvalidAgentRuntimeAllocation, ErrInvalidAgentRuntimeSoftLimit, ErrInvalidAgentRuntimeHardLimit. Rename to ErrMissingAgentRuntimeHoursAllocation etc. so the set reads consistently and greps together.
🤖
| ErrMissingAccountType = xerrors.New("license must contain valid account type") | ||
| ErrMissingAccountID = xerrors.New("license must contain valid account ID") | ||
|
|
||
| ErrMissingAgentRuntimeAllocation = xerrors.Errorf("license has agent runtime limit claims but is missing the %s claim", ClaimAgentRuntimeHoursAllocation) |
There was a problem hiding this comment.
Nit [CRF-9] ErrMissingAgentRuntimeAllocation's message "license has agent runtime limit claims but is missing the ... claim" is vaguer than the code it guards. (Gon)
The guard fires when a soft or hard limit is present without an allocation. "agent runtime limit claims" is imprecise; say "license has agent runtime hours soft/hard limit claims but is missing the %s claim" so the operator knows which claims triggered it. (Leorio confirms the sibling messages are otherwise the right bar: they name the offending claim and the exact constraint.)
🤖
Licenses can now carry three agent runtime hour claims:
agent_runtime_hours_allocation,agent_runtime_hours_limit_soft, andagent_runtime_hours_limit_hard(unit: hours). They surface as the new usage-period featureagent_runtime_hoursinGET /api/v2/entitlements, wherelimitcarries the allocation and the new optionalsoft_limit/hard_limitfields oncodersdk.Featurecarry the thresholds.Invalid combinations reject the entire license via
validateClaims(both at upload and when computing entitlements for stored licenses): soft/hard without allocation, negative allocation, soft outside0 <= soft < allocation, orhard < allocation.Soft and hard limits are not comparison inputs in
Feature.Compare; they ride along with whichever license wins (newestiat, existing behavior). None of the three claim names is a feature name, so old servers ignore them via the existing unknown-claim tolerance, protecting rollout of licenses minted with the new claims.The claim name constants defined in
enterprise/coderd/licenseare the canonical contract forgithub.com/coder/license(X1).Part of CODAGT-837. Blocks B4 (usage wiring + warnings), C1 (hard-limit admission gate), F1 (licenses page), A4 (managed-agent coexistence), X1 (licensor).
Out of scope, handled by follow-up issues:
Actualusage wiring, threshold warnings, admission gating, premium defaults, and FE surfacing beyond regenerated types.Implementation plan and decision log
Decisions (confirmed by jaayden, 2026-07-23)
agent_runtime_hours_allocation- allocation (unit: hours, int64)agent_runtime_hours_limit_soft- soft limitagent_runtime_hours_limit_hard- hard limitFeatureName; all three map to the single new usage-period featureagent_runtime_hours(FeatureAgentRuntimeHours), mirroring howmanaged_agent_limit_softmapped ontomanaged_agent_limit. Old servers therefore ignore all three claims via theFeatureNamesMapcheck.validateClaims(upload returns 400 viaParseClaimsIgnoreNbf; already-stored licenses produce anInvalid license ... parsing claimsentitlements error and contribute nothing).Design notes
codersdk.Featurehad aSoftLimitfield until 051ed34 ("feat: convert soft_limit to limit", feat: convertsoft_limittolimit#22048) collapsed managed-agent soft/hard into a singlelimit. This reintroduces soft/hard as optional fields without changing managed-agent behavior.UsagePeriodfromnbf/exp(usagePeriodStart/usagePeriodEndinLicensesEntitlements); reused unchanged, consistent with managed agents.Entitlements.AddFeaturereplaces wholeFeaturestructs (no merging), so soft/hard automatically ride along with the winning license. NoFeature.Comparelogic change; doc updates plus tests pin that soft/hard are not comparison inputs.FeatureManagedAgentLimit; usage-based warnings arrive with B4.Changes
codersdk/deployment.go: newFeatureAgentRuntimeHours(inFeatureNames,UsesLimit(),UsesUsagePeriod(), keeping it out ofFeatureSetexpansion);Feature.SoftLimit/Feature.HardLimit(soft_limit/hard_limit, omitempty); doc updates forUsagePeriodandCompare.enterprise/coderd/license/license.go: canonical claim constants; validation helper called fromvalidateClaims; al-la-carte loop maps the allocation claim to the feature and attaches soft/hard from the companion claims; skips for the companion claims and the raw feature name; generic warning loop skip.enterprise/coderd/coderdenttest:AgentRuntimeHours(allocation)builder.TestAgentRuntimeHoursLicenses: entitled/grace round-trips (including JSON field assertions), allocation-only, explicit zero,IssuedAtRankingmirror, soft/hard ride-along with a newer allocation-only license, direct feature-name claim ignored, unknown-claims compatibility (old-server simulation).TestAgentRuntimeHoursClaimValidation: table of valid/invalid claim combinations againstParseClaims, plus stored-license entitlements error.TestPostLicense: API-level 400 rejection and a happy-path POST +GET /api/v2/entitlementsround-trip.TestFeatureComparison: soft/hard ignored in comparison; newestiatwins over larger soft/hard.make gen: regeneratedsite/src/api/typesGenerated.ts,coderd/apidoc/*,docs/reference/api/*.Verification
go test ./enterprise/coderd/license/ ./codersdk/andgo test ./enterprise/coderd/ -run 'TestPostLicense|TestEntitlements'pass.golangci-lintclean on changed packages;make lint/emdashclean; FEtsc --noEmitclean.Note
Generated by Coder Agents on behalf of @jaaydenh (Linear CODAGT-837 agent session).