From 0589be6989d8e3a5324eea9faa5efcaf669b1f66 Mon Sep 17 00:00:00 2001 From: Carl <74130028+Carl723000@users.noreply.github.com> Date: Mon, 24 Aug 2026 01:55:20 +0800 Subject: [PATCH 01/72] feat: add AI advice effectiveness foundation --- AI-ADVICE-EFFECTIVENESS-V2.3.1.md | 125 ++++++ src/adviceEffectiveness/comparison.ts | 239 +++++++++++ src/adviceEffectiveness/contract.ts | 363 ++++++++++++++++ src/adviceEffectiveness/feedback.ts | 166 ++++++++ src/adviceEffectiveness/payload.ts | 444 ++++++++++++++++++++ src/adviceEffectiveness/structuredOutput.ts | 346 +++++++++++++++ src/test/adviceComparison.test.ts | 106 +++++ src/test/adviceContract.test.ts | 84 ++++ src/test/adviceFeedback.test.ts | 72 ++++ src/test/adviceOutput.test.ts | 74 ++++ src/test/advicePayload.test.ts | 84 ++++ src/test/advicePrivacy.test.ts | 64 +++ src/test/adviceTestFixtures.ts | 130 ++++++ 13 files changed, 2297 insertions(+) create mode 100644 AI-ADVICE-EFFECTIVENESS-V2.3.1.md create mode 100644 src/adviceEffectiveness/comparison.ts create mode 100644 src/adviceEffectiveness/contract.ts create mode 100644 src/adviceEffectiveness/feedback.ts create mode 100644 src/adviceEffectiveness/payload.ts create mode 100644 src/adviceEffectiveness/structuredOutput.ts create mode 100644 src/test/adviceComparison.test.ts create mode 100644 src/test/adviceContract.test.ts create mode 100644 src/test/adviceFeedback.test.ts create mode 100644 src/test/adviceOutput.test.ts create mode 100644 src/test/advicePayload.test.ts create mode 100644 src/test/advicePrivacy.test.ts create mode 100644 src/test/adviceTestFixtures.ts diff --git a/AI-ADVICE-EFFECTIVENESS-V2.3.1.md b/AI-ADVICE-EFFECTIVENESS-V2.3.1.md new file mode 100644 index 00000000..8c1a3ce0 --- /dev/null +++ b/AI-ADVICE-EFFECTIVENESS-V2.3.1.md @@ -0,0 +1,125 @@ +# v2.3.1 AI 建议有效性基础:接入与迁移说明 + +## 状态与边界 + +本基础开发从独立 worktree 的 `0b4b750`(tag `v2.2.0`)开始。R8 worktree +`codex/v2.3.0-energy-efficiency`(`48e5e77`)只用于核对未来接口,没有复制或修改。 + +本提交不会改变当前 UI、设置、版本号、发布 workflow 或网络入口。新增代码没有被 +`extension.ts`、`advisor.ts`、`webview.ts` 或 settings 引用,因此默认完全 inert。 +当前旧 AI Advice/Optimizer 行为也保持不变;迁移必须在 v2.3.1 的独立 feature flag +下完成,并经过单独的 UI、隐私和多语言评审。 + +## 新增核心 + +### `src/adviceEffectiveness/contract.ts` + +`AdviceContract` 是 provider-neutral 的 v1 契约,统一保存: + +- host 生成的本地 observation 与 evidence; +- direct / correlational / proxy 证据强度; +- AI 只能引用已有证据的 explanation; +- 带 `when` 与 stop condition 的 conditional action; +- 带最小可比任务数与质量 guardrail 的 success criterion; +- host 盖章的 privacy、source、confidence、quality flags 与 provenance。 + +模型不能提供 observation、evidence、privacy 或 provenance。`createAdviceContract()` +只在整个引用图和隐私声明一致时返回 `{ ok: true }`。空 recommendations 是合法的 +“没有足够证据下结论”,不会自动补通用建议。 + +### `src/adviceEffectiveness/payload.ts` + +`buildAdviceAggregateSnapshot()` 只消费现有 `UsageData` aggregate,并通过 allow-list +重建 DTO。完整/自定义 model ID 被压成 `opus / sonnet / haiku / fable / other`;raw +record、project label/path、cwd、session、prompt、response、credential 没有目标字段。 +可远程发送的 metric 还有单独 allow-list;新增 metric 必须经过显式隐私评审。low/unknown +confidence 或仍带 quality flag 的 source 会在 payload 生成前失败,不会要求模型填补缺口。 + +`prepareAdvicePayload()` 默认生成 `aggregates-only`,且序列化结果中完全没有 +`promptSamples` key。只有调用者额外提供: + +```ts +{ consent: 'explicit', samples: [{ text: '...' }] } +``` + +才会加入受数量、单条长度和总长度限制的 prompt 文本;sample 上的 `cwd` 等多余字段 +仍会被丢弃。`adviceUserContext` 不属于 aggregate,未来不能悄悄接入该默认路径;若要 +发送,必须另设一项明确 consent 与 preview 设计。 + +payload 只序列化一次。`previewAdvicePayload()` 返回该 `serializedBody`, +`sendPreparedAdvicePayload()` 把同一个字符串原样交给 transport。未来 transport 不得 +重新从 object 构建或“清理” body;API key、endpoint 和 headers 应在此字符串之外处理。 + +### `src/adviceEffectiveness/feedback.ts` + +feedback ledger 只接受 `helpful / not-helpful / applied`、advice/recommendation/event ID +和本地时间。它不接受自由文本、prompt、payload、解释或 endpoint。helpful 与 +not-helpful 对同一建议互斥,applied 可独立共存。存储通过最小 `get/update` port 注入, +未来可直接适配 VS Code `ExtensionContext.globalState`;模块本身没有任何网络依赖。 +损坏、未知版本或带额外字段的本地数据 fail closed,记录操作不会覆盖它。 + +### `src/adviceEffectiveness/comparison.ts` + +`compareAdviceEffectiveness()` 是无时间/IO 依赖的 paired comparison。只有以下条件全部 +满足,才可能返回 `improved`: + +- task kind、complexity、provider、model family、effort、metric definition、单位、方向、 + quality rubric 在 pair 内和 cohort 间一致; +- coverage complete,confidence 不是 low/unknown,无未允许的 quality flag; +- 达到预先声明的最小 pair 数,数值均为有限值且 baseline 非零; +- before/after 质量证据已知,after 全部通过 rubric; +- 主指标达到阈值,同时 minimum quality 与最大允许退化两个 guardrail 都守住。 + +任何可比性、覆盖、置信度、数值或质量证据问题都返回 `insufficient-evidence`。主指标 +改善但质量退化返回 `quality-guardrail-failed`;没有达到预设幅度只返回 +`no-demonstrated-improvement`,不声称建议造成伤害或收益。 + +### `src/adviceEffectiveness/structuredOutput.ts` + +`parseStructuredAdviceOutput()` 只接受一个严格 JSON 值,并逐层拒绝未知/缺失字段、错误 +schema version、非法枚举、超长/超量内容、重复 ID、空 action/criterion 和未知证据引用。 +任一 recommendation 非法会拒绝整批。它不会剥 Markdown fence、修 JSON、接受尾随 prose、 +返回 partial batch,或把 raw 输出 fallback 成看似有效的建议;错误结果也不回显 raw 模型 +正文。`recommendations: []` 是明确的 no-conclusion 成功结果。 + +## v2.3.1 迁移顺序 + +1. **薄 adapter,不碰 raw records**:Claude 从 `ClaudeDataLoader.getAllTimeData()` 或等价 + scoped aggregate 映射;本地 insight 先转成 numeric observation/evidence。不要复用当前 + `buildAdviceSummary()`,因为它默认混入 prompt、cwd-derived scope 和自由文本信号。 +2. **R8 Codex adapter**:rebase/cherry-pick 到 v2.3.0 后,从 + `CodexScopedInsights` / `CodexUsageScopeView` 映射 `kind、scope、numeric evidence、proxy`, + 同时保留 `periodCoverage.complete`、confidence 和 `qualityFlags`。coverage 不完整或没有 + triggered evidence 时不发 remote request、不生成通用 recommendation。 +3. **独立 preview/consent**:先生成 prepared payload 并展示其 exact body/manifest;prompt + sample 和任何 user context 分开询问、分开授权。取消授权时保留 aggregates-only body。 +4. **transport**:让 provider adapter 接收 `serializedBody`,把它作为实际 user payload + 原样发送;重试继续复用同一字符串。不要调用旧 `buildAdviceSummary()` 后再次拼字符串。 +5. **strict response**:只在 parser `ok` 后用 host observations/evidence/privacy/provenance + 调用 `createAdviceContract()`;失败只展示错误/重试,不展示 raw fallback 建议。 +6. **本地 feedback**:以 `globalState` adapter 接入三个 UI action;不得连接 analytics、 + contribution endpoint 或 advice transport。 +7. **有效性评估**:先固定 task cohort、metric definition、quality rubric 与阈值,再收集 + comparable pairs。比较结果是证据边界,不是因果证明。 + +## 当前已知限制与未来接入点 + +- 当前 UI 仍走 legacy Markdown advice;本基础未接线,因此现有版本行为完全不变。 +- 尚无 Claude local-insight adapter、R8 Codex adapter、provider-specific HTTP adapter 或 UI。 +- prompt consent 由未来 host/UI 负责取得;核心只接受显式 capability 并精确预览结果。 +- comparator 提供预先声明阈值的描述性 paired result,不做统计显著性或因果识别。 +- feedback 只存最小本地状态,没有 free-text 原因、同步或 telemetry。 + +未来 Claude 接入点是 `extension.ts` 当前 `buildAdviceSummary()` / `runAdviceRequest()` 边界; +未来 Codex 接入点是 R8 的 `buildScopedCodexInsights()` 输出。两者都应通过薄 adapter 接入本 +目录,而不是让核心模块 import provider、VS Code 或 transport 类型。 + +## 验证 + +测试平铺在 `src/test/`,覆盖契约引用、canonical preview/send、默认 aggregates-only、 +prompt 独立 opt-in、敌意隐私字段注入、本地 feedback 损坏数据、可比性/质量 guardrail、 +以及严格 JSON fail-closed。运行: + +```bash +npm test +``` diff --git a/src/adviceEffectiveness/comparison.ts b/src/adviceEffectiveness/comparison.ts new file mode 100644 index 00000000..027a1235 --- /dev/null +++ b/src/adviceEffectiveness/comparison.ts @@ -0,0 +1,239 @@ +/** Pure, paired before/after comparison for genuinely comparable tasks. */ + +export interface ComparableTaskContext { + taskKind: string; + complexityBand: 'low' | 'medium' | 'high'; + provider: string; + modelFamily: string; + effort: string; + metricDefinitionVersion: string; + qualityRubricId: string; +} + +export interface ComparableTaskOutcome { + context: ComparableTaskContext; + primaryMetric: { + name: string; + unit: string; + direction: 'lower-is-better' | 'higher-is-better'; + value: number; + }; + quality: { + score: number | null; + passed: boolean | null; + evidenceCount: number; + }; + evidence: { + coverage: 'complete' | 'partial' | 'unknown'; + confidence: 'high' | 'medium' | 'low' | 'unknown'; + qualityFlags: string[]; + }; +} + +export interface ComparableTaskPair { + pairId: string; + before: ComparableTaskOutcome; + after: ComparableTaskOutcome; +} + +export interface AdviceComparisonPolicy { + minComparablePairs: number; + minRelativeImprovement: number; + minAfterQualityScore: number; + maxMeanQualityRegression: number; + allowedQualityFlags: string[]; +} + +export const DEFAULT_ADVICE_COMPARISON_POLICY: AdviceComparisonPolicy = { + minComparablePairs: 5, + minRelativeImprovement: 0.1, + minAfterQualityScore: 0.8, + maxMeanQualityRegression: 0.02, + allowedQualityFlags: [], +}; + +export interface AdviceComparisonStats { + comparablePairs: number; + meanBeforeValue: number; + meanAfterValue: number; + meanRelativeImprovement: number; + meanBeforeQuality: number; + meanAfterQuality: number; + meanQualityChange: number; +} + +export type AdviceComparisonResult = + | { + status: 'insufficient-evidence'; + reasons: string[]; + comparablePairs: number; + } + | { + status: 'quality-guardrail-failed'; + reasons: string[]; + stats: AdviceComparisonStats; + } + | { + status: 'improved' | 'no-demonstrated-improvement'; + reasons: string[]; + stats: AdviceComparisonStats; + }; + +function mean(values: readonly number[]): number { + return values.reduce((sum, value) => sum + value, 0) / values.length; +} + +function sameContext(a: ComparableTaskOutcome, b: ComparableTaskOutcome): boolean { + return ( + a.context.taskKind === b.context.taskKind && + a.context.complexityBand === b.context.complexityBand && + a.context.provider === b.context.provider && + a.context.modelFamily === b.context.modelFamily && + a.context.effort === b.context.effort && + a.context.metricDefinitionVersion === b.context.metricDefinitionVersion && + a.context.qualityRubricId === b.context.qualityRubricId && + a.primaryMetric.name === b.primaryMetric.name && + a.primaryMetric.unit === b.primaryMetric.unit && + a.primaryMetric.direction === b.primaryMetric.direction + ); +} + +function validScore(value: number | null): value is number { + return value !== null && Number.isFinite(value) && value >= 0 && value <= 1; +} + +function policyIssues(policy: AdviceComparisonPolicy): string[] { + const issues: string[] = []; + if (!Number.isInteger(policy.minComparablePairs) || policy.minComparablePairs < 2) { + issues.push('comparison policy requires at least two pairs'); + } + if (!Number.isFinite(policy.minRelativeImprovement) || policy.minRelativeImprovement < 0) { + issues.push('comparison policy has an invalid improvement threshold'); + } + if ( + !Number.isFinite(policy.minAfterQualityScore) || + policy.minAfterQualityScore < 0 || + policy.minAfterQualityScore > 1 || + !Number.isFinite(policy.maxMeanQualityRegression) || + policy.maxMeanQualityRegression < 0 || + policy.maxMeanQualityRegression > 1 + ) { + issues.push('comparison policy has an invalid quality guardrail'); + } + return issues; +} + +/** + * Compare paired tasks without inferring causation. Any ambiguity in task + * comparability, coverage, confidence, metric values, or quality evidence + * returns insufficient-evidence rather than silently dropping the bad pair. + */ +export function compareAdviceEffectiveness( + pairs: readonly ComparableTaskPair[], + policy: AdviceComparisonPolicy = DEFAULT_ADVICE_COMPARISON_POLICY +): AdviceComparisonResult { + const reasons = policyIssues(policy); + const pairIds = new Set(); + const allowedFlags = new Set(policy.allowedQualityFlags); + const cohort = pairs.length > 0 ? pairs[0].before : undefined; + + for (const pair of pairs) { + if (pairIds.has(pair.pairId)) reasons.push(`duplicate pair id: ${pair.pairId}`); + pairIds.add(pair.pairId); + if (!sameContext(pair.before, pair.after)) { + reasons.push(`pair ${pair.pairId} is not comparable`); + continue; + } + if (cohort && !sameContext(cohort, pair.before)) { + reasons.push(`pair ${pair.pairId} is outside the comparison cohort`); + continue; + } + for (const [period, outcome] of [ + ['before', pair.before], + ['after', pair.after], + ] as const) { + if (outcome.evidence.coverage !== 'complete') { + reasons.push(`pair ${pair.pairId} ${period} coverage is incomplete`); + } + if (outcome.evidence.confidence === 'unknown' || outcome.evidence.confidence === 'low') { + reasons.push(`pair ${pair.pairId} ${period} confidence is insufficient`); + } + const disallowedFlags = outcome.evidence.qualityFlags.filter((flag) => !allowedFlags.has(flag)); + if (disallowedFlags.length > 0) { + reasons.push(`pair ${pair.pairId} ${period} has unresolved quality flags`); + } + if (!Number.isFinite(outcome.primaryMetric.value) || outcome.primaryMetric.value < 0) { + reasons.push(`pair ${pair.pairId} ${period} metric is invalid`); + } + if (period === 'before' && outcome.primaryMetric.value === 0) { + reasons.push(`pair ${pair.pairId} has a zero baseline`); + } + if (!validScore(outcome.quality.score) || outcome.quality.passed === null) { + reasons.push(`pair ${pair.pairId} ${period} quality evidence is unknown`); + } + if (!Number.isInteger(outcome.quality.evidenceCount) || outcome.quality.evidenceCount < 1) { + reasons.push(`pair ${pair.pairId} ${period} has no quality evidence`); + } + if (period === 'before' && outcome.quality.passed === false) { + reasons.push(`pair ${pair.pairId} baseline did not pass its quality rubric`); + } + } + } + + if (pairs.length < policy.minComparablePairs) { + reasons.push(`need at least ${policy.minComparablePairs} comparable pairs`); + } + if (reasons.length > 0) { + return { status: 'insufficient-evidence', reasons, comparablePairs: pairs.length }; + } + + const beforeValues = pairs.map((pair) => pair.before.primaryMetric.value); + const afterValues = pairs.map((pair) => pair.after.primaryMetric.value); + const beforeQuality = pairs.map((pair) => pair.before.quality.score as number); + const afterQuality = pairs.map((pair) => pair.after.quality.score as number); + const relativeImprovements = pairs.map((pair) => { + const before = pair.before.primaryMetric.value; + const after = pair.after.primaryMetric.value; + return pair.before.primaryMetric.direction === 'lower-is-better' + ? (before - after) / before + : (after - before) / before; + }); + const meanBeforeQuality = mean(beforeQuality); + const meanAfterQuality = mean(afterQuality); + const stats: AdviceComparisonStats = { + comparablePairs: pairs.length, + meanBeforeValue: mean(beforeValues), + meanAfterValue: mean(afterValues), + meanRelativeImprovement: mean(relativeImprovements), + meanBeforeQuality, + meanAfterQuality, + meanQualityChange: meanAfterQuality - meanBeforeQuality, + }; + + const guardrailReasons: string[] = []; + if (pairs.some((pair) => pair.after.quality.passed !== true)) { + guardrailReasons.push('at least one after task failed its quality rubric'); + } + if (meanAfterQuality < policy.minAfterQualityScore) { + guardrailReasons.push('mean after quality is below the required score'); + } + if (stats.meanQualityChange < -policy.maxMeanQualityRegression) { + guardrailReasons.push('mean quality regressed beyond the allowed guardrail'); + } + if (guardrailReasons.length > 0) { + return { status: 'quality-guardrail-failed', reasons: guardrailReasons, stats }; + } + + if (stats.meanRelativeImprovement >= policy.minRelativeImprovement) { + return { + status: 'improved', + reasons: ['primary metric met the threshold while the quality guardrail held'], + stats, + }; + } + return { + status: 'no-demonstrated-improvement', + reasons: ['primary metric did not meet the pre-declared improvement threshold'], + stats, + }; +} diff --git a/src/adviceEffectiveness/contract.ts b/src/adviceEffectiveness/contract.ts new file mode 100644 index 00000000..58962c10 --- /dev/null +++ b/src/adviceEffectiveness/contract.ts @@ -0,0 +1,363 @@ +/** + * Versioned, provider-neutral contract for evidence-backed advice. + * + * The host owns observations, evidence, provenance, and privacy metadata. A + * remote model may only propose recommendations that reference those host-owned + * IDs; it never gets to manufacture measurements or claim its own provenance. + */ + +export const ADVICE_CONTRACT_VERSION = 1 as const; + +export type AdviceScope = 'overall' | 'project' | 'task-cohort'; + +export type AdviceWindow = + | { kind: 'rolling-days'; days: number } + | { kind: 'all-time' } + | { kind: 'task-cohort' }; + +export type AdviceSourceKind = + | 'claude-usage-aggregate' + | 'claude-local-insight' + | 'codex-local-insight' + | 'task-comparison'; + +export interface AdviceSourceInfo { + id: string; + kind: AdviceSourceKind; + scope: AdviceScope; + window: AdviceWindow; + confidence: 'high' | 'medium' | 'low' | 'unknown'; + /** Machine-readable quality flags only; never raw paths, IDs, or content. */ + qualityFlags: string[]; +} + +export type AdviceMetricUnit = + | 'count' + | 'tokens' + | 'ratio' + | 'percent' + | 'milliseconds' + | 'usd-estimate' + | 'boolean'; + +export interface AdviceObservation { + id: string; + /** Stable machine-readable metric name, e.g. cache-read-share. */ + metric: string; + value: number | boolean; + unit: AdviceMetricUnit; + method: 'measured' | 'estimated' | 'structural-proxy'; + sourceId: string; + /** Host-authored readable observation. It is not included in remote payloads. */ + summary: string; +} + +export interface AdviceEvidence { + id: string; + observationIds: string[]; + strength: 'direct' | 'correlational' | 'proxy'; + /** Host-authored readable evidence. It is not included in remote payloads. */ + summary: string; + limitations: string[]; +} + +export interface AdviceExplanation { + summary: string; + /** Observation IDs used only as proxies, never as proof of causation. */ + proxyMetricObservationIds: string[]; + limitations: string[]; +} + +export interface AdviceConditionalAction { + when: string; + action: string; + evidenceIds: string[]; + stopCondition?: string; +} + +export interface AdviceQualityGuardrail { + /** Stable rubric ID shared by every task in the comparison cohort. */ + rubricId: string; + /** Normalized 0..1 quality score required after applying the advice. */ + minimumScore: number; + /** Maximum allowed mean quality-score drop, normalized to 0..1. */ + maximumRegression: number; +} + +export interface AdviceSuccessCriterion { + metricObservationId: string; + direction: 'increase' | 'decrease' | 'maintain'; + target: { + kind: 'relative-change' | 'absolute'; + value: number; + }; + minimumComparableTasks: number; + qualityGuardrail: AdviceQualityGuardrail; +} + +export interface AdviceRecommendation { + id: string; + title: string; + evidenceIds: string[]; + explanation: AdviceExplanation; + conditionalActions: AdviceConditionalAction[]; + successCriteria: AdviceSuccessCriterion[]; +} + +export interface AdvicePrivacyInfo { + dataMode: 'local-only' | 'aggregates-only' | 'aggregates-with-prompt-samples'; + promptSampleConsent: 'not-applicable' | 'not-granted' | 'explicit'; + promptSampleCount: number; + feedbackStorage: 'local-only'; +} + +export interface AdviceProvenance { + generatedBy: { + kind: 'local-rules' | 'remote-model'; + /** Coarse family only; never a full custom model or endpoint identifier. */ + modelFamily?: string; + }; + generatedAt: string; + locale: string; + sources: AdviceSourceInfo[]; +} + +export interface AdviceContract { + schemaVersion: typeof ADVICE_CONTRACT_VERSION; + adviceId: string; + observations: AdviceObservation[]; + evidence: AdviceEvidence[]; + recommendations: AdviceRecommendation[]; + privacy: AdvicePrivacyInfo; + provenance: AdviceProvenance; +} + +export type AdviceContractBuildResult = + | { ok: true; value: AdviceContract } + | { ok: false; issues: string[] }; + +export interface AdviceContractInput { + adviceId: string; + observations: AdviceObservation[]; + evidence: AdviceEvidence[]; + recommendations: AdviceRecommendation[]; + privacy: AdvicePrivacyInfo; + provenance: AdviceProvenance; +} + +const ID_PATTERN = /^[a-z][a-z0-9._-]{0,95}$/; + +export function isAdviceIdentifier(value: string): boolean { + return ID_PATTERN.test(value); +} + +function uniqueIds(items: T[], label: string, issues: string[]): Set { + const ids = new Set(); + for (const item of items) { + if (!isAdviceIdentifier(item.id)) { + issues.push(`${label} has invalid id: ${item.id}`); + } else if (ids.has(item.id)) { + issues.push(`${label} has duplicate id: ${item.id}`); + } + ids.add(item.id); + } + return ids; +} + +function nonEmpty(value: string): boolean { + return value.trim().length > 0; +} + +function validFiniteNumber(value: number): boolean { + return Number.isFinite(value); +} + +/** Validate the complete host-owned advice artifact. No partial artifact is returned. */ +export function validateAdviceContract(contract: AdviceContract): string[] { + const issues: string[] = []; + if (contract.schemaVersion !== ADVICE_CONTRACT_VERSION) { + issues.push(`unsupported schemaVersion: ${String(contract.schemaVersion)}`); + } + if (!isAdviceIdentifier(contract.adviceId)) { + issues.push('adviceId must be a stable machine-readable identifier'); + } + if (contract.recommendations.length > 0 && contract.observations.length === 0) { + issues.push('recommendations require at least one observation'); + } + if (contract.recommendations.length > 0 && contract.evidence.length === 0) { + issues.push('recommendations require at least one evidence item'); + } + + const sourceIds = uniqueIds(contract.provenance.sources, 'source', issues); + const observationIds = uniqueIds(contract.observations, 'observation', issues); + const evidenceIds = uniqueIds(contract.evidence, 'evidence', issues); + uniqueIds(contract.recommendations, 'recommendation', issues); + + for (const source of contract.provenance.sources) { + if (source.window.kind === 'rolling-days') { + if (!Number.isInteger(source.window.days) || source.window.days < 1) { + issues.push(`source ${source.id} has an invalid rolling window`); + } + } + for (const flag of source.qualityFlags) { + if (!isAdviceIdentifier(flag)) { + issues.push(`source ${source.id} has an invalid quality flag`); + } + } + } + + for (const observation of contract.observations) { + if (!isAdviceIdentifier(observation.metric)) { + issues.push(`observation ${observation.id} has an invalid metric`); + } + if (typeof observation.value === 'number' && !validFiniteNumber(observation.value)) { + issues.push(`observation ${observation.id} has a non-finite value`); + } + if (!sourceIds.has(observation.sourceId)) { + issues.push(`observation ${observation.id} references unknown source ${observation.sourceId}`); + } + if (!nonEmpty(observation.summary)) { + issues.push(`observation ${observation.id} has an empty summary`); + } + } + + for (const item of contract.evidence) { + if (item.observationIds.length === 0) { + issues.push(`evidence ${item.id} must reference an observation`); + } + for (const observationId of item.observationIds) { + if (!observationIds.has(observationId)) { + issues.push(`evidence ${item.id} references unknown observation ${observationId}`); + } + } + if (!nonEmpty(item.summary)) issues.push(`evidence ${item.id} has an empty summary`); + if (item.strength === 'proxy' && item.limitations.length === 0) { + issues.push(`proxy evidence ${item.id} must state a limitation`); + } + } + + for (const recommendation of contract.recommendations) { + if (!nonEmpty(recommendation.title)) { + issues.push(`recommendation ${recommendation.id} has an empty title`); + } + if (recommendation.evidenceIds.length === 0) { + issues.push(`recommendation ${recommendation.id} must reference evidence`); + } + for (const evidenceId of recommendation.evidenceIds) { + if (!evidenceIds.has(evidenceId)) { + issues.push(`recommendation ${recommendation.id} references unknown evidence ${evidenceId}`); + } + } + if (!nonEmpty(recommendation.explanation.summary)) { + issues.push(`recommendation ${recommendation.id} has an empty explanation`); + } + for (const observationId of recommendation.explanation.proxyMetricObservationIds) { + if (!observationIds.has(observationId)) { + issues.push(`recommendation ${recommendation.id} references unknown proxy ${observationId}`); + } + } + if ( + recommendation.explanation.proxyMetricObservationIds.length > 0 && + recommendation.explanation.limitations.length === 0 + ) { + issues.push(`recommendation ${recommendation.id} must limit its proxy interpretation`); + } + if (recommendation.conditionalActions.length === 0) { + issues.push(`recommendation ${recommendation.id} must include a conditional action`); + } + for (const action of recommendation.conditionalActions) { + if (!nonEmpty(action.when) || !nonEmpty(action.action)) { + issues.push(`recommendation ${recommendation.id} has an empty condition or action`); + } + if (action.evidenceIds.length === 0) { + issues.push(`recommendation ${recommendation.id} action must reference evidence`); + } + for (const evidenceId of action.evidenceIds) { + if (!evidenceIds.has(evidenceId)) { + issues.push(`recommendation ${recommendation.id} action references unknown evidence ${evidenceId}`); + } + } + } + if (recommendation.successCriteria.length === 0) { + issues.push(`recommendation ${recommendation.id} must include a success criterion`); + } + for (const criterion of recommendation.successCriteria) { + if (!observationIds.has(criterion.metricObservationId)) { + issues.push( + `recommendation ${recommendation.id} criterion references unknown observation ${criterion.metricObservationId}` + ); + } + if (!validFiniteNumber(criterion.target.value) || criterion.target.value < 0) { + issues.push(`recommendation ${recommendation.id} has an invalid success target`); + } + if (!Number.isInteger(criterion.minimumComparableTasks) || criterion.minimumComparableTasks < 2) { + issues.push(`recommendation ${recommendation.id} needs at least two comparable tasks`); + } + const guardrail = criterion.qualityGuardrail; + if (!isAdviceIdentifier(guardrail.rubricId)) { + issues.push(`recommendation ${recommendation.id} has an invalid quality rubric`); + } + if ( + !validFiniteNumber(guardrail.minimumScore) || + guardrail.minimumScore < 0 || + guardrail.minimumScore > 1 || + !validFiniteNumber(guardrail.maximumRegression) || + guardrail.maximumRegression < 0 || + guardrail.maximumRegression > 1 + ) { + issues.push(`recommendation ${recommendation.id} has an invalid quality guardrail`); + } + } + } + + const privacy = contract.privacy; + if (!Number.isInteger(privacy.promptSampleCount) || privacy.promptSampleCount < 0) { + issues.push('promptSampleCount must be a non-negative integer'); + } + if ( + privacy.dataMode === 'local-only' && + (privacy.promptSampleConsent !== 'not-applicable' || privacy.promptSampleCount !== 0) + ) { + issues.push('local-only advice cannot carry prompt-sample consent or samples'); + } + if ( + privacy.dataMode === 'aggregates-only' && + (privacy.promptSampleConsent !== 'not-granted' || privacy.promptSampleCount !== 0) + ) { + issues.push('aggregates-only advice must contain zero prompt samples'); + } + if ( + privacy.dataMode === 'aggregates-with-prompt-samples' && + (privacy.promptSampleConsent !== 'explicit' || privacy.promptSampleCount < 1) + ) { + issues.push('prompt samples require explicit consent and a non-zero sample count'); + } + + if (!nonEmpty(contract.provenance.locale)) issues.push('provenance locale is required'); + if (!Number.isFinite(Date.parse(contract.provenance.generatedAt))) { + issues.push('provenance generatedAt must be an ISO-compatible timestamp'); + } + if ( + contract.provenance.generatedBy.modelFamily !== undefined && + !isAdviceIdentifier(contract.provenance.generatedBy.modelFamily) + ) { + issues.push('provenance modelFamily must be coarse and machine-readable'); + } + + return issues; +} + +/** Assemble host metadata with recommendations only after the whole contract validates. */ +export function createAdviceContract(input: AdviceContractInput): AdviceContractBuildResult { + const value: AdviceContract = { + schemaVersion: ADVICE_CONTRACT_VERSION, + adviceId: input.adviceId, + observations: input.observations, + evidence: input.evidence, + recommendations: input.recommendations, + privacy: input.privacy, + provenance: input.provenance, + }; + const issues = validateAdviceContract(value); + return issues.length === 0 ? { ok: true, value } : { ok: false, issues }; +} diff --git a/src/adviceEffectiveness/feedback.ts b/src/adviceEffectiveness/feedback.ts new file mode 100644 index 00000000..143b15b1 --- /dev/null +++ b/src/adviceEffectiveness/feedback.ts @@ -0,0 +1,166 @@ +import { isAdviceIdentifier } from './contract'; + +export const ADVICE_FEEDBACK_STORAGE_KEY = 'claudeCodeUsage.adviceEffectiveness.feedback.v1'; +export const ADVICE_FEEDBACK_VERSION = 1 as const; +export const MAX_ADVICE_FEEDBACK_EVENTS = 500; + +export type AdviceFeedbackKind = 'helpful' | 'not-helpful' | 'applied'; + +export interface AdviceFeedbackEvent { + schemaVersion: typeof ADVICE_FEEDBACK_VERSION; + eventId: string; + adviceId: string; + recommendationId: string; + kind: AdviceFeedbackKind; + recordedAt: string; + localOnly: true; +} + +export interface AdviceFeedbackStorage { + get(key: string): T | undefined; + update(key: string, value: unknown): PromiseLike; +} + +interface StoredAdviceFeedback { + schemaVersion: typeof ADVICE_FEEDBACK_VERSION; + events: AdviceFeedbackEvent[]; +} + +export type AdviceFeedbackLoadResult = + | { ok: true; events: AdviceFeedbackEvent[] } + | { ok: false; reason: 'invalid-local-data' }; + +export type AdviceFeedbackWriteResult = + | { ok: true; events: AdviceFeedbackEvent[] } + | { ok: false; reason: 'invalid-event' | 'invalid-local-data' }; + +const EVENT_KEYS = [ + 'adviceId', + 'eventId', + 'kind', + 'localOnly', + 'recordedAt', + 'recommendationId', + 'schemaVersion', +] as const; + +function isObject(value: unknown): value is Record { + return typeof value === 'object' && value !== null && !Array.isArray(value); +} + +function hasExactKeys(value: Record, expected: readonly string[]): boolean { + const actual = Object.keys(value).sort(); + const sortedExpected = [...expected].sort(); + return actual.length === sortedExpected.length && actual.every((key, index) => key === sortedExpected[index]); +} + +function isFeedbackKind(value: unknown): value is AdviceFeedbackKind { + return value === 'helpful' || value === 'not-helpful' || value === 'applied'; +} + +function parseEvent(value: unknown): AdviceFeedbackEvent | undefined { + if (!isObject(value) || !hasExactKeys(value, EVENT_KEYS)) return undefined; + if ( + value.schemaVersion !== ADVICE_FEEDBACK_VERSION || + value.localOnly !== true || + typeof value.eventId !== 'string' || + typeof value.adviceId !== 'string' || + typeof value.recommendationId !== 'string' || + typeof value.recordedAt !== 'string' || + !isFeedbackKind(value.kind) || + !isAdviceIdentifier(value.eventId) || + !isAdviceIdentifier(value.adviceId) || + !isAdviceIdentifier(value.recommendationId) || + !Number.isFinite(Date.parse(value.recordedAt)) + ) { + return undefined; + } + return { + schemaVersion: ADVICE_FEEDBACK_VERSION, + eventId: value.eventId, + adviceId: value.adviceId, + recommendationId: value.recommendationId, + kind: value.kind, + recordedAt: value.recordedAt, + localOnly: true, + }; +} + +/** Read only the versioned local ledger; corrupt or unknown shapes fail closed. */ +export function loadAdviceFeedback(storage: AdviceFeedbackStorage): AdviceFeedbackLoadResult { + const raw = storage.get(ADVICE_FEEDBACK_STORAGE_KEY); + if (raw === undefined) return { ok: true, events: [] }; + if (!isObject(raw) || !hasExactKeys(raw, ['events', 'schemaVersion'])) { + return { ok: false, reason: 'invalid-local-data' }; + } + if (raw.schemaVersion !== ADVICE_FEEDBACK_VERSION || !Array.isArray(raw.events)) { + return { ok: false, reason: 'invalid-local-data' }; + } + const events: AdviceFeedbackEvent[] = []; + const eventIds = new Set(); + for (const rawEvent of raw.events) { + const event = parseEvent(rawEvent); + if (!event || eventIds.has(event.eventId)) return { ok: false, reason: 'invalid-local-data' }; + eventIds.add(event.eventId); + events.push(event); + } + return { ok: true, events }; +} + +/** + * Store a local feedback state transition. Helpful/not-helpful are mutually + * exclusive for one recommendation; applied is independent and may coexist. + * No prompt, payload, explanation, endpoint, or free-text field is accepted. + */ +export async function recordAdviceFeedback( + storage: AdviceFeedbackStorage, + eventInput: Omit +): Promise { + const event = parseEvent({ + schemaVersion: ADVICE_FEEDBACK_VERSION, + eventId: eventInput.eventId, + adviceId: eventInput.adviceId, + recommendationId: eventInput.recommendationId, + kind: eventInput.kind, + recordedAt: eventInput.recordedAt, + localOnly: true, + }); + if (!event) return { ok: false, reason: 'invalid-event' }; + + const loaded = loadAdviceFeedback(storage); + if (!loaded.ok) return loaded; + + const sameTarget = (candidate: AdviceFeedbackEvent): boolean => + candidate.adviceId === event.adviceId && candidate.recommendationId === event.recommendationId; + const next = loaded.events.filter((candidate) => { + if (candidate.eventId === event.eventId) return false; + if (!sameTarget(candidate)) return true; + if (event.kind === 'applied') return candidate.kind !== 'applied'; + return candidate.kind === 'applied'; + }); + next.push(event); + const bounded = next.slice(-MAX_ADVICE_FEEDBACK_EVENTS); + const stored: StoredAdviceFeedback = { schemaVersion: ADVICE_FEEDBACK_VERSION, events: bounded }; + await storage.update(ADVICE_FEEDBACK_STORAGE_KEY, stored); + return { ok: true, events: bounded }; +} + +export interface AdviceFeedbackSummary { + rating: 'helpful' | 'not-helpful' | 'unrated'; + applied: boolean; +} + +export function summarizeAdviceFeedback( + events: readonly AdviceFeedbackEvent[], + adviceId: string, + recommendationId: string +): AdviceFeedbackSummary { + let rating: AdviceFeedbackSummary['rating'] = 'unrated'; + let applied = false; + for (const event of events) { + if (event.adviceId !== adviceId || event.recommendationId !== recommendationId) continue; + if (event.kind === 'applied') applied = true; + else rating = event.kind; + } + return { rating, applied }; +} diff --git a/src/adviceEffectiveness/payload.ts b/src/adviceEffectiveness/payload.ts new file mode 100644 index 00000000..00ac0d92 --- /dev/null +++ b/src/adviceEffectiveness/payload.ts @@ -0,0 +1,444 @@ +import { UsageData } from '../types'; +import { + ADVICE_CONTRACT_VERSION, + AdviceEvidence, + AdviceObservation, + AdviceSourceInfo, + isAdviceIdentifier, +} from './contract'; + +export const ADVICE_REMOTE_PAYLOAD_VERSION = 1 as const; +export const MAX_PROMPT_SAMPLES = 20; +export const MAX_PROMPT_SAMPLE_CHARS = 1_000; +export const MAX_PROMPT_SAMPLE_TOTAL_CHARS = 12_000; + +/** Adding a metric here is an explicit remote-data/privacy review point. */ +export const REMOTE_ADVICE_METRIC_ALLOWLIST = [ + 'approval-reviewer-share', + 'cache-creation-tokens', + 'cache-hit-rate', + 'cache-read-share', + 'cache-read-tokens', + 'estimated-cost-usd', + 'high-effort-share', + 'input-tokens', + 'large-context-share', + 'long-session-share', + 'message-count', + 'multi-agent-share', + 'output-tokens', + 'post-patch-tool-intensity', + 'processed-to-fresh-ratio', + 'quality-score', + 'subagent-share', + 'task-primary-metric', + 'thinking-share', + 'workflow-share', +] as const; + +export type AdviceModelFamily = 'opus' | 'sonnet' | 'haiku' | 'fable' | 'other'; + +export interface AdviceAggregateSnapshot { + scope: 'overall' | 'project'; + windowDays: number; + totals: { + inputTokens: number; + outputTokens: number; + cacheCreationTokens: number; + cacheReadTokens: number; + messageCount: number; + estimatedCostUsd: number; + }; + modelFamilies: { + family: AdviceModelFamily; + inputTokens: number; + outputTokens: number; + cacheCreationTokens: number; + cacheReadTokens: number; + count: number; + }[]; +} + +export interface PromptSampleOptIn { + consent: 'explicit'; + /** Only text is accepted. Extra runtime fields such as cwd are discarded. */ + samples: readonly { text: string }[]; +} + +export interface PrepareAdvicePayloadInput { + locale: string; + aggregate: AdviceAggregateSnapshot; + sources: readonly AdviceSourceInfo[]; + observations: readonly AdviceObservation[]; + evidence: readonly AdviceEvidence[]; + promptSamples?: PromptSampleOptIn; +} + +export interface PreparedAdvicePayload { + contentType: 'application/json'; + dataMode: 'aggregates-only' | 'aggregates-with-prompt-samples'; + promptSampleCount: number; + /** The one canonical serialization used by both preview and transport. */ + serializedBody: string; +} + +export interface AdvicePayloadPreview { + contentType: 'application/json'; + dataMode: PreparedAdvicePayload['dataMode']; + promptSampleCount: number; + body: string; +} + +interface RemoteAdviceRequest { + schemaVersion: typeof ADVICE_REMOTE_PAYLOAD_VERSION; + responseContractVersion: typeof ADVICE_CONTRACT_VERSION; + locale: string; + aggregate: AdviceAggregateSnapshot; + signals: { + sources: { + id: string; + kind: AdviceSourceInfo['kind']; + scope: AdviceSourceInfo['scope']; + window: AdviceSourceInfo['window']; + confidence: 'high' | 'medium'; + }[]; + observations: { + id: string; + metric: string; + value: number | boolean; + unit: AdviceObservation['unit']; + method: AdviceObservation['method']; + sourceId: string; + }[]; + evidence: { + id: string; + observationIds: string[]; + strength: AdviceEvidence['strength']; + }[]; + }; + responseRules: { + format: 'strict-json'; + unknownFields: 'reject'; + evidenceReferences: 'required'; + qualityGuardrail: 'required'; + hostOwnedFields: ['observations', 'evidence', 'privacy', 'provenance']; + }; + privacy: { + dataMode: PreparedAdvicePayload['dataMode']; + promptSampleConsent: 'not-granted' | 'explicit'; + promptSampleCount: number; + }; + promptSamples?: { id: string; text: string }[]; +} + +function requireFiniteNonNegative(value: number, label: string): number { + if (!Number.isFinite(value) || value < 0) { + throw new Error(`${label} must be a finite non-negative number`); + } + return value; +} + +function requireCount(value: number, label: string): number { + requireFiniteNonNegative(value, label); + if (!Number.isInteger(value)) throw new Error(`${label} must be an integer`); + return value; +} + +function modelFamily(model: string): AdviceModelFamily { + const normalized = model.toLowerCase(); + if (normalized.includes('opus')) return 'opus'; + if (normalized.includes('sonnet')) return 'sonnet'; + if (normalized.includes('haiku')) return 'haiku'; + if (normalized.includes('fable')) return 'fable'; + return 'other'; +} + +/** + * Convert the existing UsageData aggregate through an explicit allow-list. + * Full/custom model IDs, project labels, paths, sessions, and prompt content + * have no destination field and therefore cannot leak through object spreads. + */ +export function buildAdviceAggregateSnapshot( + usage: UsageData, + scope: AdviceAggregateSnapshot['scope'], + windowDays: number +): AdviceAggregateSnapshot { + if (!Number.isInteger(windowDays) || windowDays < 1 || windowDays > 3_650) { + throw new Error('windowDays must be an integer from 1 to 3650'); + } + const families = new Map(); + for (const model of Object.keys(usage.modelBreakdown)) { + const row = usage.modelBreakdown[model]; + const family = modelFamily(model); + const current = families.get(family) || { + family, + inputTokens: 0, + outputTokens: 0, + cacheCreationTokens: 0, + cacheReadTokens: 0, + count: 0, + }; + current.inputTokens += requireFiniteNonNegative(row.inputTokens, `${family}.inputTokens`); + current.outputTokens += requireFiniteNonNegative(row.outputTokens, `${family}.outputTokens`); + current.cacheCreationTokens += requireFiniteNonNegative( + row.cacheCreationTokens, + `${family}.cacheCreationTokens` + ); + current.cacheReadTokens += requireFiniteNonNegative(row.cacheReadTokens, `${family}.cacheReadTokens`); + current.count += requireCount(row.count, `${family}.count`); + families.set(family, current); + } + + return { + scope, + windowDays, + totals: { + inputTokens: requireFiniteNonNegative(usage.totalInputTokens, 'totalInputTokens'), + outputTokens: requireFiniteNonNegative(usage.totalOutputTokens, 'totalOutputTokens'), + cacheCreationTokens: requireFiniteNonNegative( + usage.totalCacheCreationTokens, + 'totalCacheCreationTokens' + ), + cacheReadTokens: requireFiniteNonNegative(usage.totalCacheReadTokens, 'totalCacheReadTokens'), + messageCount: requireCount(usage.messageCount, 'messageCount'), + estimatedCostUsd: requireFiniteNonNegative(usage.totalCost, 'totalCost'), + }, + modelFamilies: Array.from(families.values()).sort((a, b) => a.family.localeCompare(b.family)), + }; +} + +function validateAggregate(aggregate: AdviceAggregateSnapshot): void { + if (aggregate.scope !== 'overall' && aggregate.scope !== 'project') { + throw new Error('aggregate scope is invalid'); + } + if (!Number.isInteger(aggregate.windowDays) || aggregate.windowDays < 1 || aggregate.windowDays > 3_650) { + throw new Error('aggregate windowDays is invalid'); + } + requireFiniteNonNegative(aggregate.totals.inputTokens, 'aggregate.inputTokens'); + requireFiniteNonNegative(aggregate.totals.outputTokens, 'aggregate.outputTokens'); + requireFiniteNonNegative(aggregate.totals.cacheCreationTokens, 'aggregate.cacheCreationTokens'); + requireFiniteNonNegative(aggregate.totals.cacheReadTokens, 'aggregate.cacheReadTokens'); + requireCount(aggregate.totals.messageCount, 'aggregate.messageCount'); + requireFiniteNonNegative(aggregate.totals.estimatedCostUsd, 'aggregate.estimatedCostUsd'); + const allowedFamilies = new Set(['opus', 'sonnet', 'haiku', 'fable', 'other']); + const seen = new Set(); + for (const row of aggregate.modelFamilies) { + if (!allowedFamilies.has(row.family) || seen.has(row.family)) { + throw new Error('aggregate model family is invalid or duplicated'); + } + seen.add(row.family); + requireFiniteNonNegative(row.inputTokens, `${row.family}.inputTokens`); + requireFiniteNonNegative(row.outputTokens, `${row.family}.outputTokens`); + requireFiniteNonNegative(row.cacheCreationTokens, `${row.family}.cacheCreationTokens`); + requireFiniteNonNegative(row.cacheReadTokens, `${row.family}.cacheReadTokens`); + requireCount(row.count, `${row.family}.count`); + } +} + +function validateSignalGraph(input: PrepareAdvicePayloadInput): void { + if (!/^[A-Za-z]{2,3}(?:-[A-Za-z0-9]{2,8})*$/.test(input.locale)) { + throw new Error('locale must be a BCP-47-style locale code'); + } + if (input.sources.length === 0 || input.observations.length === 0 || input.evidence.length === 0) { + throw new Error('evidence-backed advice requires sources, observations, and evidence'); + } + const sourceIds = new Set(); + for (const source of input.sources) { + if (!isAdviceIdentifier(source.id) || sourceIds.has(source.id)) throw new Error('invalid source id'); + sourceIds.add(source.id); + if (source.confidence !== 'high' && source.confidence !== 'medium') { + throw new Error('remote advice requires high or medium source confidence'); + } + if (source.qualityFlags.length > 0) { + throw new Error('remote advice does not accept unresolved source quality flags'); + } + if (source.window.kind === 'rolling-days') { + if (!Number.isInteger(source.window.days) || source.window.days < 1) { + throw new Error('invalid source window'); + } + } + for (const flag of source.qualityFlags) { + if (!isAdviceIdentifier(flag)) throw new Error('invalid source quality flag'); + } + } + const observationIds = new Set(); + const allowedMetrics = new Set(REMOTE_ADVICE_METRIC_ALLOWLIST); + for (const observation of input.observations) { + if (!isAdviceIdentifier(observation.id) || observationIds.has(observation.id)) { + throw new Error('invalid observation id'); + } + observationIds.add(observation.id); + if (!allowedMetrics.has(observation.metric) || !sourceIds.has(observation.sourceId)) { + throw new Error('invalid observation metric or source reference'); + } + if (typeof observation.value === 'number') { + requireFiniteNonNegative(observation.value, observation.metric); + } + if (observation.unit === 'ratio' && typeof observation.value === 'number' && observation.value > 1) { + throw new Error('ratio observations must be between 0 and 1'); + } + if (observation.unit === 'percent' && typeof observation.value === 'number' && observation.value > 100) { + throw new Error('percent observations must be between 0 and 100'); + } + if (observation.unit === 'boolean' && typeof observation.value !== 'boolean') { + throw new Error('boolean observations require a boolean value'); + } + } + const evidenceIds = new Set(); + for (const item of input.evidence) { + if (!isAdviceIdentifier(item.id) || evidenceIds.has(item.id) || item.observationIds.length === 0) { + throw new Error('invalid evidence id or empty evidence references'); + } + evidenceIds.add(item.id); + for (const observationId of item.observationIds) { + if (!observationIds.has(observationId)) throw new Error('evidence references an unknown observation'); + } + } +} + +function promptSamples(optIn: PromptSampleOptIn | undefined): { id: string; text: string }[] { + if (optIn === undefined) return []; + if (optIn.consent !== 'explicit' || !Array.isArray(optIn.samples)) { + throw new Error('prompt samples require a separate explicit opt-in'); + } + const out: { id: string; text: string }[] = []; + let totalChars = 0; + for (const sample of optIn.samples.slice(0, MAX_PROMPT_SAMPLES)) { + if (!sample || typeof sample.text !== 'string') throw new Error('prompt sample text must be a string'); + const text = sample.text.trim().slice(0, MAX_PROMPT_SAMPLE_CHARS); + if (text.length === 0) continue; + const remaining = MAX_PROMPT_SAMPLE_TOTAL_CHARS - totalChars; + if (remaining <= 0) break; + const bounded = text.slice(0, remaining); + out.push({ id: `prompt-${out.length + 1}`, text: bounded }); + totalChars += bounded.length; + } + if (out.length === 0) throw new Error('explicit prompt opt-in must contain at least one non-empty sample'); + return out; +} + +function canonicalize(value: unknown): unknown { + if (value === null || typeof value === 'string' || typeof value === 'boolean') return value; + if (typeof value === 'number') { + if (!Number.isFinite(value)) throw new Error('payload contains a non-finite number'); + return value; + } + if (Array.isArray(value)) return value.map((item) => canonicalize(item)); + if (typeof value === 'object') { + const source = value as Record; + const out: Record = {}; + for (const key of Object.keys(source).sort()) { + const item = source[key]; + if (item !== undefined) out[key] = canonicalize(item); + } + return out; + } + throw new Error(`payload contains unsupported ${typeof value} value`); +} + +export function stableAdvicePayloadStringify(value: unknown): string { + return JSON.stringify(canonicalize(value)); +} + +/** Build and serialize once. Aggregates-only is the default and has no promptSamples key. */ +export function prepareAdvicePayload(input: PrepareAdvicePayloadInput): PreparedAdvicePayload { + validateAggregate(input.aggregate); + validateSignalGraph(input); + const samples = promptSamples(input.promptSamples); + const dataMode: PreparedAdvicePayload['dataMode'] = + samples.length > 0 ? 'aggregates-with-prompt-samples' : 'aggregates-only'; + + const request: RemoteAdviceRequest = { + schemaVersion: ADVICE_REMOTE_PAYLOAD_VERSION, + responseContractVersion: ADVICE_CONTRACT_VERSION, + locale: input.locale, + aggregate: { + scope: input.aggregate.scope, + windowDays: input.aggregate.windowDays, + totals: { + inputTokens: input.aggregate.totals.inputTokens, + outputTokens: input.aggregate.totals.outputTokens, + cacheCreationTokens: input.aggregate.totals.cacheCreationTokens, + cacheReadTokens: input.aggregate.totals.cacheReadTokens, + messageCount: input.aggregate.totals.messageCount, + estimatedCostUsd: input.aggregate.totals.estimatedCostUsd, + }, + modelFamilies: input.aggregate.modelFamilies.map((row) => ({ + family: row.family, + inputTokens: row.inputTokens, + outputTokens: row.outputTokens, + cacheCreationTokens: row.cacheCreationTokens, + cacheReadTokens: row.cacheReadTokens, + count: row.count, + })), + }, + signals: { + sources: input.sources.map((source) => ({ + id: source.id, + kind: source.kind, + scope: source.scope, + window: + source.window.kind === 'rolling-days' + ? { kind: 'rolling-days', days: source.window.days } + : { kind: source.window.kind }, + confidence: source.confidence as 'high' | 'medium', + })), + observations: input.observations.map((observation) => ({ + id: observation.id, + metric: observation.metric, + value: observation.value, + unit: observation.unit, + method: observation.method, + sourceId: observation.sourceId, + })), + evidence: input.evidence.map((item) => ({ + id: item.id, + observationIds: [...item.observationIds], + strength: item.strength, + })), + }, + responseRules: { + format: 'strict-json', + unknownFields: 'reject', + evidenceReferences: 'required', + qualityGuardrail: 'required', + hostOwnedFields: ['observations', 'evidence', 'privacy', 'provenance'], + }, + privacy: { + dataMode, + promptSampleConsent: samples.length > 0 ? 'explicit' : 'not-granted', + promptSampleCount: samples.length, + }, + }; + if (samples.length > 0) request.promptSamples = samples; + + return { + contentType: 'application/json', + dataMode, + promptSampleCount: samples.length, + serializedBody: stableAdvicePayloadStringify(request), + }; +} + +/** Preview exposes the exact immutable string that the sender receives. */ +export function previewAdvicePayload(prepared: PreparedAdvicePayload): AdvicePayloadPreview { + return { + contentType: prepared.contentType, + dataMode: prepared.dataMode, + promptSampleCount: prepared.promptSampleCount, + body: prepared.serializedBody, + }; +} + +export type AdvicePayloadSender = ( + serializedBody: string, + contentType: PreparedAdvicePayload['contentType'] +) => Promise; + +/** This boundary deliberately cannot reserialize: it only receives the prepared bytes. */ +export function sendPreparedAdvicePayload( + prepared: PreparedAdvicePayload, + sender: AdvicePayloadSender +): Promise { + return sender(prepared.serializedBody, prepared.contentType); +} diff --git a/src/adviceEffectiveness/structuredOutput.ts b/src/adviceEffectiveness/structuredOutput.ts new file mode 100644 index 00000000..017e7b76 --- /dev/null +++ b/src/adviceEffectiveness/structuredOutput.ts @@ -0,0 +1,346 @@ +import { + ADVICE_CONTRACT_VERSION, + AdviceConditionalAction, + AdviceExplanation, + AdviceQualityGuardrail, + AdviceRecommendation, + AdviceSuccessCriterion, + isAdviceIdentifier, +} from './contract'; + +export interface StructuredAdviceOutput { + schemaVersion: typeof ADVICE_CONTRACT_VERSION; + /** Empty is a valid "no supported recommendation" result. */ + recommendations: AdviceRecommendation[]; +} + +export interface StructuredAdviceReferences { + observationIds: readonly string[]; + evidenceIds: readonly string[]; +} + +export type StructuredAdviceParseErrorCode = + | 'empty-output' + | 'output-too-large' + | 'invalid-json' + | 'invalid-schema' + | 'unknown-reference'; + +export type StructuredAdviceParseResult = + | { ok: true; value: StructuredAdviceOutput } + | { ok: false; code: StructuredAdviceParseErrorCode; issues: string[] }; + +const MAX_RAW_CHARS = 100_000; +const MAX_RECOMMENDATIONS = 8; +const MAX_ACTIONS = 4; +const MAX_CRITERIA = 4; +const MAX_REFERENCES = 12; +const MAX_LIMITATIONS = 8; + +function isObject(value: unknown): value is Record { + return typeof value === 'object' && value !== null && !Array.isArray(value); +} + +function exactKeys(value: Record, required: string[], optional: string[] = []): boolean { + const actual = Object.keys(value); + const allowed = new Set([...required, ...optional]); + return required.every((key) => actual.includes(key)) && actual.every((key) => allowed.has(key)); +} + +function boundedText(value: unknown, max: number): string | undefined { + if (typeof value !== 'string') return undefined; + const text = value.trim(); + return text.length > 0 && text.length <= max ? text : undefined; +} + +function idArray( + value: unknown, + max: number, + known: Set, + path: string, + issues: string[], + allowEmpty: boolean = false +): string[] | undefined { + if (!Array.isArray(value) || value.length > max || (!allowEmpty && value.length === 0)) { + issues.push(`${path} must be a bounded reference array`); + return undefined; + } + const out: string[] = []; + const seen = new Set(); + for (const item of value) { + if (typeof item !== 'string' || !isAdviceIdentifier(item) || seen.has(item)) { + issues.push(`${path} contains an invalid or duplicate id`); + return undefined; + } + if (!known.has(item)) { + issues.push(`${path} contains an unknown reference`); + return undefined; + } + seen.add(item); + out.push(item); + } + return out; +} + +function textArray(value: unknown, path: string, issues: string[]): string[] | undefined { + if (!Array.isArray(value) || value.length > MAX_LIMITATIONS) { + issues.push(`${path} must be a bounded string array`); + return undefined; + } + const out: string[] = []; + for (const item of value) { + const text = boundedText(item, 400); + if (!text) { + issues.push(`${path} contains invalid text`); + return undefined; + } + out.push(text); + } + return out; +} + +function parseExplanation( + value: unknown, + observationIds: Set, + path: string, + issues: string[] +): AdviceExplanation | undefined { + if (!isObject(value) || !exactKeys(value, ['limitations', 'proxyMetricObservationIds', 'summary'])) { + issues.push(`${path} has an invalid shape`); + return undefined; + } + const summary = boundedText(value.summary, 1_500); + const proxyMetricObservationIds = idArray( + value.proxyMetricObservationIds, + MAX_REFERENCES, + observationIds, + `${path}.proxyMetricObservationIds`, + issues, + true + ); + const limitations = textArray(value.limitations, `${path}.limitations`, issues); + if (!summary || !proxyMetricObservationIds || !limitations) { + if (!summary) issues.push(`${path}.summary is empty or too long`); + return undefined; + } + if (proxyMetricObservationIds.length > 0 && limitations.length === 0) { + issues.push(`${path} must state limitations for proxy metrics`); + return undefined; + } + return { summary, proxyMetricObservationIds, limitations }; +} + +function parseAction( + value: unknown, + evidenceIds: Set, + path: string, + issues: string[] +): AdviceConditionalAction | undefined { + if (!isObject(value) || !exactKeys(value, ['action', 'evidenceIds', 'when'], ['stopCondition'])) { + issues.push(`${path} has an invalid shape`); + return undefined; + } + const when = boundedText(value.when, 600); + const action = boundedText(value.action, 800); + const references = idArray(value.evidenceIds, MAX_REFERENCES, evidenceIds, `${path}.evidenceIds`, issues); + const stopCondition = value.stopCondition === undefined ? undefined : boundedText(value.stopCondition, 600); + if (!when || !action || !references || (value.stopCondition !== undefined && !stopCondition)) { + issues.push(`${path} has empty or oversized text`); + return undefined; + } + return stopCondition ? { when, action, evidenceIds: references, stopCondition } : { when, action, evidenceIds: references }; +} + +function finiteRange(value: unknown, min: number, max: number): value is number { + return typeof value === 'number' && Number.isFinite(value) && value >= min && value <= max; +} + +function parseGuardrail(value: unknown, path: string, issues: string[]): AdviceQualityGuardrail | undefined { + if (!isObject(value) || !exactKeys(value, ['maximumRegression', 'minimumScore', 'rubricId'])) { + issues.push(`${path} has an invalid shape`); + return undefined; + } + if ( + typeof value.rubricId !== 'string' || + !isAdviceIdentifier(value.rubricId) || + !finiteRange(value.minimumScore, 0, 1) || + !finiteRange(value.maximumRegression, 0, 1) + ) { + issues.push(`${path} has invalid bounds or rubric id`); + return undefined; + } + return { + rubricId: value.rubricId, + minimumScore: value.minimumScore, + maximumRegression: value.maximumRegression, + }; +} + +function parseCriterion( + value: unknown, + observationIds: Set, + path: string, + issues: string[] +): AdviceSuccessCriterion | undefined { + if ( + !isObject(value) || + !exactKeys(value, [ + 'direction', + 'metricObservationId', + 'minimumComparableTasks', + 'qualityGuardrail', + 'target', + ]) + ) { + issues.push(`${path} has an invalid shape`); + return undefined; + } + if ( + typeof value.metricObservationId !== 'string' || + !observationIds.has(value.metricObservationId) || + (value.direction !== 'increase' && value.direction !== 'decrease' && value.direction !== 'maintain') || + !Number.isInteger(value.minimumComparableTasks) || + (value.minimumComparableTasks as number) < 2 || + (value.minimumComparableTasks as number) > 10_000 + ) { + issues.push(`${path} has an invalid metric reference, direction, or sample size`); + return undefined; + } + if (!isObject(value.target) || !exactKeys(value.target, ['kind', 'value'])) { + issues.push(`${path}.target has an invalid shape`); + return undefined; + } + if ( + (value.target.kind !== 'relative-change' && value.target.kind !== 'absolute') || + !finiteRange(value.target.value, 0, Number.MAX_SAFE_INTEGER) + ) { + issues.push(`${path}.target has an invalid kind or value`); + return undefined; + } + const guardrail = parseGuardrail(value.qualityGuardrail, `${path}.qualityGuardrail`, issues); + if (!guardrail) return undefined; + return { + metricObservationId: value.metricObservationId, + direction: value.direction, + target: { kind: value.target.kind, value: value.target.value }, + minimumComparableTasks: value.minimumComparableTasks as number, + qualityGuardrail: guardrail, + }; +} + +function parseRecommendation( + value: unknown, + references: { observationIds: Set; evidenceIds: Set }, + path: string, + issues: string[] +): AdviceRecommendation | undefined { + if ( + !isObject(value) || + !exactKeys(value, [ + 'conditionalActions', + 'evidenceIds', + 'explanation', + 'id', + 'successCriteria', + 'title', + ]) + ) { + issues.push(`${path} has an invalid shape`); + return undefined; + } + if (typeof value.id !== 'string' || !isAdviceIdentifier(value.id)) { + issues.push(`${path}.id is invalid`); + return undefined; + } + const title = boundedText(value.title, 160); + const evidenceIds = idArray( + value.evidenceIds, + MAX_REFERENCES, + references.evidenceIds, + `${path}.evidenceIds`, + issues + ); + const explanation = parseExplanation(value.explanation, references.observationIds, `${path}.explanation`, issues); + if (!Array.isArray(value.conditionalActions) || value.conditionalActions.length < 1 || value.conditionalActions.length > MAX_ACTIONS) { + issues.push(`${path}.conditionalActions must be a bounded non-empty array`); + return undefined; + } + if (!Array.isArray(value.successCriteria) || value.successCriteria.length < 1 || value.successCriteria.length > MAX_CRITERIA) { + issues.push(`${path}.successCriteria must be a bounded non-empty array`); + return undefined; + } + const conditionalActions = value.conditionalActions.map((item, index) => + parseAction(item, references.evidenceIds, `${path}.conditionalActions[${index}]`, issues) + ); + const successCriteria = value.successCriteria.map((item, index) => + parseCriterion(item, references.observationIds, `${path}.successCriteria[${index}]`, issues) + ); + if (!title || !evidenceIds || !explanation || conditionalActions.some((item) => !item) || successCriteria.some((item) => !item)) { + if (!title) issues.push(`${path}.title is empty or too long`); + return undefined; + } + return { + id: value.id, + title, + evidenceIds, + explanation, + conditionalActions: conditionalActions as AdviceConditionalAction[], + successCriteria: successCriteria as AdviceSuccessCriterion[], + }; +} + +/** + * Strict JSON-only parser. It never strips fences, repairs output, returns a + * partial batch, or turns malformed prose into a plausible recommendation. + */ +export function parseStructuredAdviceOutput( + raw: string, + knownReferences: StructuredAdviceReferences +): StructuredAdviceParseResult { + const text = raw.trim(); + if (text.length === 0) return { ok: false, code: 'empty-output', issues: ['model output was empty'] }; + if (text.length > MAX_RAW_CHARS) { + return { ok: false, code: 'output-too-large', issues: ['model output exceeded the size limit'] }; + } + let parsed: unknown; + try { + parsed = JSON.parse(text) as unknown; + } catch { + return { ok: false, code: 'invalid-json', issues: ['model output was not one strict JSON value'] }; + } + if (!isObject(parsed) || !exactKeys(parsed, ['recommendations', 'schemaVersion'])) { + return { ok: false, code: 'invalid-schema', issues: ['top-level response shape is invalid'] }; + } + if (parsed.schemaVersion !== ADVICE_CONTRACT_VERSION || !Array.isArray(parsed.recommendations)) { + return { ok: false, code: 'invalid-schema', issues: ['response schema version or recommendations is invalid'] }; + } + if (parsed.recommendations.length > MAX_RECOMMENDATIONS) { + return { ok: false, code: 'invalid-schema', issues: ['too many recommendations'] }; + } + + const observationIds = new Set(knownReferences.observationIds); + const evidenceIds = new Set(knownReferences.evidenceIds); + const issues: string[] = []; + const recommendations = parsed.recommendations.map((item, index) => + parseRecommendation(item, { observationIds, evidenceIds }, `recommendations[${index}]`, issues) + ); + const recommendationIds = new Set(); + for (const recommendation of recommendations) { + if (!recommendation) continue; + if (recommendationIds.has(recommendation.id)) issues.push('recommendation ids must be unique'); + recommendationIds.add(recommendation.id); + } + if (issues.length > 0 || recommendations.some((item) => !item)) { + const code: StructuredAdviceParseErrorCode = issues.some((issue) => issue.includes('unknown reference')) + ? 'unknown-reference' + : 'invalid-schema'; + return { ok: false, code, issues }; + } + return { + ok: true, + value: { + schemaVersion: ADVICE_CONTRACT_VERSION, + recommendations: recommendations as AdviceRecommendation[], + }, + }; +} diff --git a/src/test/adviceComparison.test.ts b/src/test/adviceComparison.test.ts new file mode 100644 index 00000000..6b7f0b3b --- /dev/null +++ b/src/test/adviceComparison.test.ts @@ -0,0 +1,106 @@ +import { test } from 'node:test'; +import * as assert from 'node:assert/strict'; + +import { + AdviceComparisonPolicy, + ComparableTaskPair, + compareAdviceEffectiveness, +} from '../adviceEffectiveness/comparison'; + +const policy: AdviceComparisonPolicy = { + minComparablePairs: 5, + minRelativeImprovement: 0.1, + minAfterQualityScore: 0.8, + maxMeanQualityRegression: 0.02, + allowedQualityFlags: [], +}; + +function pairs(afterValue: number = 80, afterQuality: number = 0.9): ComparableTaskPair[] { + return Array.from({ length: 5 }, (_, index) => ({ + pairId: `pair-${index + 1}`, + before: { + context: { + taskKind: 'focused-code-change', + complexityBand: 'medium' as const, + provider: 'claude', + modelFamily: 'sonnet', + effort: 'high', + metricDefinitionVersion: 'tokens-v1', + qualityRubricId: 'acceptance-v1', + }, + primaryMetric: { + name: 'total-tokens', + unit: 'tokens', + direction: 'lower-is-better' as const, + value: 100, + }, + quality: { score: 0.9, passed: true, evidenceCount: 2 }, + evidence: { coverage: 'complete' as const, confidence: 'high' as const, qualityFlags: [] }, + }, + after: { + context: { + taskKind: 'focused-code-change', + complexityBand: 'medium' as const, + provider: 'claude', + modelFamily: 'sonnet', + effort: 'high', + metricDefinitionVersion: 'tokens-v1', + qualityRubricId: 'acceptance-v1', + }, + primaryMetric: { + name: 'total-tokens', + unit: 'tokens', + direction: 'lower-is-better' as const, + value: afterValue, + }, + quality: { score: afterQuality, passed: true, evidenceCount: 2 }, + evidence: { coverage: 'complete' as const, confidence: 'high' as const, qualityFlags: [] }, + }, + })); +} + +test('concludes improved only after the metric threshold and quality guardrail both hold', () => { + const input = pairs(); + const before = JSON.stringify(input); + const result = compareAdviceEffectiveness(input, policy); + assert.equal(result.status, 'improved'); + if (result.status === 'improved') assert.equal(result.stats.meanRelativeImprovement, 0.2); + assert.equal(JSON.stringify(input), before, 'comparison must not mutate measurements'); +}); + +test('returns insufficient evidence for small, incomparable, partial, or unknown-quality cohorts', () => { + assert.equal(compareAdviceEffectiveness(pairs().slice(0, 4), policy).status, 'insufficient-evidence'); + + const incomparable = pairs(); + incomparable[1].after.context.modelFamily = 'opus'; + assert.equal(compareAdviceEffectiveness(incomparable, policy).status, 'insufficient-evidence'); + + const partial = pairs(); + partial[2].before.evidence.coverage = 'partial'; + assert.equal(compareAdviceEffectiveness(partial, policy).status, 'insufficient-evidence'); + + const unknownQuality = pairs(); + unknownQuality[3].after.quality.score = null; + assert.equal(compareAdviceEffectiveness(unknownQuality, policy).status, 'insufficient-evidence'); +}); + +test('does not claim effectiveness when the primary metric improves but quality regresses', () => { + const result = compareAdviceEffectiveness(pairs(70, 0.75), policy); + assert.equal(result.status, 'quality-guardrail-failed'); +}); + +test('reports no demonstrated improvement without turning it into a causal harm claim', () => { + const result = compareAdviceEffectiveness(pairs(95, 0.9), policy); + assert.equal(result.status, 'no-demonstrated-improvement'); + assert.ok(result.reasons[0].includes('pre-declared')); +}); + +test('zero baselines and non-finite measurements are inconclusive', () => { + const zero = pairs(); + zero[0].before.primaryMetric.value = 0; + assert.equal(compareAdviceEffectiveness(zero, policy).status, 'insufficient-evidence'); + + const invalid = pairs(); + invalid[0].after.primaryMetric.value = Number.NaN; + assert.equal(compareAdviceEffectiveness(invalid, policy).status, 'insufficient-evidence'); +}); diff --git a/src/test/adviceContract.test.ts b/src/test/adviceContract.test.ts new file mode 100644 index 00000000..9f627641 --- /dev/null +++ b/src/test/adviceContract.test.ts @@ -0,0 +1,84 @@ +import { test } from 'node:test'; +import * as assert from 'node:assert/strict'; + +import { AdviceContractInput, createAdviceContract } from '../adviceEffectiveness/contract'; +import { + evidenceFixture, + observationFixture, + recommendationFixture, + sourceFixture, +} from './adviceTestFixtures'; + +function validInput(): AdviceContractInput { + return { + adviceId: 'advice-run-1', + observations: [observationFixture], + evidence: [evidenceFixture], + recommendations: [recommendationFixture], + privacy: { + dataMode: 'aggregates-only', + promptSampleConsent: 'not-granted', + promptSampleCount: 0, + feedbackStorage: 'local-only', + }, + provenance: { + generatedBy: { kind: 'remote-model', modelFamily: 'sonnet' }, + generatedAt: '2026-08-24T00:00:00.000Z', + locale: 'en', + sources: [sourceFixture], + }, + }; +} + +test('creates one versioned contract with observation, evidence, action, criterion, source, and privacy', () => { + const result = createAdviceContract(validInput()); + assert.equal(result.ok, true); + if (!result.ok) return; + assert.equal(result.value.schemaVersion, 1); + assert.equal(result.value.recommendations[0].conditionalActions.length, 1); + assert.equal(result.value.recommendations[0].successCriteria[0].qualityGuardrail.minimumScore, 0.8); + assert.equal(result.value.privacy.dataMode, 'aggregates-only'); + assert.equal(result.value.provenance.sources[0].kind, 'claude-usage-aggregate'); +}); + +test('rejects invented evidence references and proxy explanations without limitations', () => { + const input = validInput(); + input.recommendations = [ + { + ...recommendationFixture, + evidenceIds: ['evidence-invented'], + explanation: { + summary: 'Looks causal.', + proxyMetricObservationIds: [observationFixture.id], + limitations: [], + }, + }, + ]; + const result = createAdviceContract(input); + assert.equal(result.ok, false); + if (result.ok) return; + assert.ok(result.issues.some((issue) => issue.includes('unknown evidence'))); + assert.ok(result.issues.some((issue) => issue.includes('proxy interpretation'))); +}); + +test('rejects privacy metadata that claims aggregates-only while carrying prompt samples', () => { + const input = validInput(); + input.privacy = { + dataMode: 'aggregates-only', + promptSampleConsent: 'explicit', + promptSampleCount: 1, + feedbackStorage: 'local-only', + }; + const result = createAdviceContract(input); + assert.equal(result.ok, false); + if (result.ok) return; + assert.ok(result.issues.some((issue) => issue.includes('zero prompt samples'))); +}); + +test('allows an empty recommendation list to represent no supported conclusion', () => { + const input = validInput(); + input.recommendations = []; + const result = createAdviceContract(input); + assert.equal(result.ok, true); + if (result.ok) assert.deepEqual(result.value.recommendations, []); +}); diff --git a/src/test/adviceFeedback.test.ts b/src/test/adviceFeedback.test.ts new file mode 100644 index 00000000..910b49f1 --- /dev/null +++ b/src/test/adviceFeedback.test.ts @@ -0,0 +1,72 @@ +import { test } from 'node:test'; +import * as assert from 'node:assert/strict'; + +import { + ADVICE_FEEDBACK_STORAGE_KEY, + AdviceFeedbackStorage, + loadAdviceFeedback, + recordAdviceFeedback, + summarizeAdviceFeedback, +} from '../adviceEffectiveness/feedback'; + +class MemoryStorage implements AdviceFeedbackStorage { + value: unknown; + updates = 0; + + get(_key: string): T | undefined { + return this.value as T | undefined; + } + + async update(key: string, value: unknown): Promise { + assert.equal(key, ADVICE_FEEDBACK_STORAGE_KEY); + this.value = value; + this.updates += 1; + } +} + +function event(eventId: string, kind: 'helpful' | 'not-helpful' | 'applied') { + return { + eventId, + adviceId: 'advice-run-1', + recommendationId: 'recommendation-1', + kind, + recordedAt: `2026-08-24T00:00:0${eventId.slice(-1)}.000Z`, + }; +} + +test('records rating and applied state locally while keeping them semantically independent', async () => { + const storage = new MemoryStorage(); + await recordAdviceFeedback(storage, event('event-1', 'helpful')); + await recordAdviceFeedback(storage, event('event-2', 'applied')); + const result = await recordAdviceFeedback(storage, event('event-3', 'not-helpful')); + assert.equal(result.ok, true); + if (!result.ok) return; + assert.equal(storage.updates, 3); + assert.equal(result.events.length, 2, 'the new rating replaces the previous rating, not applied'); + assert.deepEqual( + summarizeAdviceFeedback(result.events, 'advice-run-1', 'recommendation-1'), + { rating: 'not-helpful', applied: true } + ); + const stored = JSON.stringify(storage.value); + assert.doesNotMatch(stored, /prompt|payload|explanation|endpoint|PRIVATE/); + assert.match(stored, /"localOnly":true/); +}); + +test('invalid events are rejected without writing', async () => { + const storage = new MemoryStorage(); + const result = await recordAdviceFeedback(storage, { + ...event('event-1', 'helpful'), + recordedAt: 'not-a-date', + }); + assert.deepEqual(result, { ok: false, reason: 'invalid-event' }); + assert.equal(storage.updates, 0); +}); + +test('corrupt or unknown-version local data fails closed and is not overwritten', async () => { + const storage = new MemoryStorage(); + storage.value = { schemaVersion: 99, events: [{ prompt: 'private' }] }; + assert.deepEqual(loadAdviceFeedback(storage), { ok: false, reason: 'invalid-local-data' }); + const result = await recordAdviceFeedback(storage, event('event-1', 'helpful')); + assert.deepEqual(result, { ok: false, reason: 'invalid-local-data' }); + assert.equal(storage.updates, 0); +}); diff --git a/src/test/adviceOutput.test.ts b/src/test/adviceOutput.test.ts new file mode 100644 index 00000000..5492fac5 --- /dev/null +++ b/src/test/adviceOutput.test.ts @@ -0,0 +1,74 @@ +import { test } from 'node:test'; +import * as assert from 'node:assert/strict'; + +import { parseStructuredAdviceOutput } from '../adviceEffectiveness/structuredOutput'; +import { + evidenceFixture, + observationFixture, + recommendationFixture, + structuredOutputFixture, +} from './adviceTestFixtures'; + +const references = { + observationIds: [observationFixture.id], + evidenceIds: [evidenceFixture.id], +}; + +test('parses one exact JSON response whose recommendations reference host evidence', () => { + const result = parseStructuredAdviceOutput(JSON.stringify(structuredOutputFixture()), references); + assert.equal(result.ok, true); + if (result.ok) assert.deepEqual(result.value.recommendations, [recommendationFixture]); +}); + +test('accepts an empty recommendation list as an explicit no-conclusion result', () => { + const result = parseStructuredAdviceOutput(JSON.stringify({ schemaVersion: 1, recommendations: [] }), references); + assert.deepEqual(result, { ok: true, value: { schemaVersion: 1, recommendations: [] } }); +}); + +test('fails closed for prose, markdown fences, trailing text, and extra host-owned fields', () => { + const valid = JSON.stringify(structuredOutputFixture()); + for (const raw of [ + 'This looks useful.', + `\`\`\`json\n${valid}\n\`\`\``, + `${valid}\nextra prose`, + JSON.stringify({ ...structuredOutputFixture() as object, privacy: { dataMode: 'local-only' } }), + ]) { + const result = parseStructuredAdviceOutput(raw, references); + assert.equal(result.ok, false); + } +}); + +test('rejects unknown evidence references and never echoes raw model output in the error', () => { + const secret = 'PRIVATE_MODEL_OUTPUT_SENTINEL'; + const output = structuredOutputFixture() as { schemaVersion: number; recommendations: typeof recommendationFixture[] }; + output.recommendations = [ + { + ...recommendationFixture, + evidenceIds: ['evidence-invented'], + title: secret, + }, + ]; + const result = parseStructuredAdviceOutput(JSON.stringify(output), references); + assert.equal(result.ok, false); + if (result.ok) return; + assert.equal(result.code, 'unknown-reference'); + assert.equal(JSON.stringify(result).includes(secret), false); +}); + +test('rejects the entire batch when one recommendation is malformed or duplicated', () => { + const malformed = { + ...recommendationFixture, + successCriteria: [{ ...recommendationFixture.successCriteria[0], qualityGuardrail: undefined }], + }; + const result = parseStructuredAdviceOutput( + JSON.stringify({ schemaVersion: 1, recommendations: [recommendationFixture, malformed] }), + references + ); + assert.equal(result.ok, false); + + const duplicate = parseStructuredAdviceOutput( + JSON.stringify({ schemaVersion: 1, recommendations: [recommendationFixture, recommendationFixture] }), + references + ); + assert.equal(duplicate.ok, false); +}); diff --git a/src/test/advicePayload.test.ts b/src/test/advicePayload.test.ts new file mode 100644 index 00000000..80c1f4f0 --- /dev/null +++ b/src/test/advicePayload.test.ts @@ -0,0 +1,84 @@ +import { test } from 'node:test'; +import * as assert from 'node:assert/strict'; + +import { + buildAdviceAggregateSnapshot, + prepareAdvicePayload, + previewAdvicePayload, + sendPreparedAdvicePayload, +} from '../adviceEffectiveness/payload'; +import { payloadInputFixture, usageFixture } from './adviceTestFixtures'; + +test('builds a coarse aggregate without retaining full or custom model names', () => { + const aggregate = buildAdviceAggregateSnapshot(usageFixture, 'overall', 30); + assert.deepEqual(aggregate.modelFamilies.map((row) => row.family), ['other', 'sonnet']); + assert.doesNotMatch(JSON.stringify(aggregate), /private-router|20250514/); +}); + +test('defaults to aggregates-only and omits the promptSamples field entirely', () => { + const prepared = prepareAdvicePayload(payloadInputFixture()); + const parsed = JSON.parse(prepared.serializedBody) as Record; + assert.equal(prepared.dataMode, 'aggregates-only'); + assert.equal(prepared.promptSampleCount, 0); + assert.equal(Object.prototype.hasOwnProperty.call(parsed, 'promptSamples'), false); +}); + +test('preview and sender consume the exact same canonical serialization', async () => { + const input = payloadInputFixture(); + const before = JSON.stringify(input); + const first = prepareAdvicePayload(input); + const second = prepareAdvicePayload(input); + const preview = previewAdvicePayload(first); + let captured = ''; + const result = await sendPreparedAdvicePayload(first, async (body, contentType) => { + captured = body; + assert.equal(contentType, 'application/json'); + return 'sent'; + }); + assert.equal(result, 'sent'); + assert.equal(preview.body, first.serializedBody); + assert.equal(captured, preview.body); + assert.equal(first.serializedBody, second.serializedBody); + assert.equal(JSON.stringify(input), before, 'payload preparation must not mutate input'); +}); + +test('prompt text appears only behind the separate explicit opt-in and is bounded', () => { + const input = payloadInputFixture(); + input.promptSamples = { + consent: 'explicit', + samples: [{ text: `private prompt ${'x'.repeat(2_000)}` }], + }; + const prepared = prepareAdvicePayload(input); + const parsed = JSON.parse(prepared.serializedBody) as { + promptSamples: { text: string }[]; + privacy: { promptSampleConsent: string }; + }; + assert.equal(prepared.dataMode, 'aggregates-with-prompt-samples'); + assert.equal(parsed.privacy.promptSampleConsent, 'explicit'); + assert.equal(parsed.promptSamples.length, 1); + assert.equal(parsed.promptSamples[0].text.length, 1_000); +}); + +test('invalid aggregates and empty explicit prompt consent fail before a payload exists', () => { + const invalid = payloadInputFixture(); + invalid.aggregate.totals.inputTokens = Number.NaN; + assert.throws(() => prepareAdvicePayload(invalid), /finite non-negative/); + + const empty = payloadInputFixture(); + empty.promptSamples = { consent: 'explicit', samples: [{ text: ' ' }] }; + assert.throws(() => prepareAdvicePayload(empty), /at least one non-empty/); +}); + +test('remote observations require reviewed metrics and sufficiently clean source evidence', () => { + const unknownMetric = payloadInputFixture(); + unknownMetric.observations = [{ ...unknownMetric.observations[0], metric: 'private-user-name' }]; + assert.throws(() => prepareAdvicePayload(unknownMetric), /invalid observation metric/); + + const degraded = payloadInputFixture(); + degraded.sources = [{ ...degraded.sources[0], qualityFlags: ['partial-coverage'] }]; + assert.throws(() => prepareAdvicePayload(degraded), /unresolved source quality flags/); + + const unknownConfidence = payloadInputFixture(); + unknownConfidence.sources = [{ ...unknownConfidence.sources[0], confidence: 'unknown' }]; + assert.throws(() => prepareAdvicePayload(unknownConfidence), /source confidence/); +}); diff --git a/src/test/advicePrivacy.test.ts b/src/test/advicePrivacy.test.ts new file mode 100644 index 00000000..425a5994 --- /dev/null +++ b/src/test/advicePrivacy.test.ts @@ -0,0 +1,64 @@ +import { test } from 'node:test'; +import * as assert from 'node:assert/strict'; + +import { prepareAdvicePayload, previewAdvicePayload } from '../adviceEffectiveness/payload'; +import { payloadInputFixture } from './adviceTestFixtures'; + +const SENTINELS = { + prompt: 'PRIVATE_PROMPT_SENTINEL', + response: 'PRIVATE_RESPONSE_SENTINEL', + path: '/Users/private/repository', + session: 'PRIVATE_SESSION_ID', + credential: 'sk-ant-private-credential', + username: 'private-user-name', +}; + +function hostileInput(includePrompt: boolean) { + const input = payloadInputFixture() as ReturnType & Record; + input.cwd = SENTINELS.path; + input.sessionId = SENTINELS.session; + input.apiKey = SENTINELS.credential; + (input.aggregate as unknown as Record).response = SENTINELS.response; + (input.aggregate.totals as unknown as Record).username = SENTINELS.username; + (input.aggregate.modelFamilies[0] as unknown as Record).model = 'private-custom-model'; + (input.sources[0] as unknown as Record).repoPath = SENTINELS.path; + (input.sources[0].window as unknown as Record).rawSessionId = SENTINELS.session; + (input.observations[0] as unknown as Record).prompt = SENTINELS.prompt; + (input.evidence[0] as unknown as Record).response = SENTINELS.response; + if (includePrompt) { + input.promptSamples = { + consent: 'explicit', + samples: [ + { + text: SENTINELS.prompt, + cwd: SENTINELS.path, + response: SENTINELS.response, + apiKey: SENTINELS.credential, + } as { text: string }, + ], + }; + } + return input; +} + +test('aggregates-only serialization strips hostile raw content and identifying fields at every depth', () => { + const prepared = prepareAdvicePayload(hostileInput(false)); + const preview = previewAdvicePayload(prepared); + assert.equal(preview.body, prepared.serializedBody); + for (const sentinel of Object.values(SENTINELS)) { + assert.equal(prepared.serializedBody.includes(sentinel), false, `leaked sentinel: ${sentinel}`); + } + assert.doesNotMatch( + prepared.serializedBody, + /"(?:cwd|path|repoPath|sessionId|rawSessionId|username|apiKey|response|prompt)"/ + ); +}); + +test('prompt opt-in permits only bounded prompt text and still strips cwd, response, and credentials', () => { + const prepared = prepareAdvicePayload(hostileInput(true)); + assert.equal(prepared.serializedBody.includes(SENTINELS.prompt), true); + for (const sentinel of [SENTINELS.path, SENTINELS.response, SENTINELS.credential, SENTINELS.session]) { + assert.equal(prepared.serializedBody.includes(sentinel), false, `leaked non-consented sentinel: ${sentinel}`); + } + assert.doesNotMatch(prepared.serializedBody, /"(?:cwd|repoPath|sessionId|apiKey|response)"/); +}); diff --git a/src/test/adviceTestFixtures.ts b/src/test/adviceTestFixtures.ts new file mode 100644 index 00000000..888641c6 --- /dev/null +++ b/src/test/adviceTestFixtures.ts @@ -0,0 +1,130 @@ +import { + AdviceEvidence, + AdviceObservation, + AdviceRecommendation, + AdviceSourceInfo, +} from '../adviceEffectiveness/contract'; +import { PrepareAdvicePayloadInput, buildAdviceAggregateSnapshot } from '../adviceEffectiveness/payload'; +import { UsageData } from '../types'; + +export const usageFixture: UsageData = { + totalInputTokens: 1_000, + totalOutputTokens: 400, + totalCacheCreationTokens: 200, + totalCacheReadTokens: 2_000, + totalCost: 1.25, + costBreakdown: { input: 0.1, output: 0.4, cacheWrite: 0.25, cacheRead: 0.5 }, + messageCount: 10, + modelBreakdown: { + 'claude-sonnet-4-20250514': { + inputTokens: 700, + outputTokens: 300, + cacheCreationTokens: 100, + cacheReadTokens: 1_500, + cost: 1, + count: 7, + }, + 'private-router-model-name': { + inputTokens: 300, + outputTokens: 100, + cacheCreationTokens: 100, + cacheReadTokens: 500, + cost: 0.25, + count: 3, + }, + }, +}; + +export const sourceFixture: AdviceSourceInfo = { + id: 'source-usage-30d', + kind: 'claude-usage-aggregate', + scope: 'overall', + window: { kind: 'rolling-days', days: 30 }, + confidence: 'high', + qualityFlags: [], +}; + +export const observationFixture: AdviceObservation = { + id: 'observation-cache-share', + metric: 'cache-read-share', + value: 0.625, + unit: 'ratio', + method: 'measured', + sourceId: sourceFixture.id, + summary: 'Cache reads are 62.5% of observed input-side tokens.', +}; + +export const proxyObservationFixture: AdviceObservation = { + id: 'observation-tool-intensity', + metric: 'post-patch-tool-intensity', + value: 5, + unit: 'count', + method: 'structural-proxy', + sourceId: sourceFixture.id, + summary: 'Five tool calls were observed after a patch.', +}; + +export const evidenceFixture: AdviceEvidence = { + id: 'evidence-cache-share', + observationIds: [observationFixture.id], + strength: 'correlational', + summary: 'The aggregate has substantial cache-read reuse.', + limitations: ['Cache share does not measure task quality.'], +}; + +export const proxyEvidenceFixture: AdviceEvidence = { + id: 'evidence-tool-intensity', + observationIds: [proxyObservationFixture.id], + strength: 'proxy', + summary: 'Post-patch tool-call count is elevated.', + limitations: ['Tool names and command bodies are not inspected.'], +}; + +export const recommendationFixture: AdviceRecommendation = { + id: 'recommendation-compare-boundaries', + title: 'Compare task boundaries on representative work', + evidenceIds: [evidenceFixture.id], + explanation: { + summary: 'A controlled comparison can test whether a smaller task boundary helps.', + proxyMetricObservationIds: [], + limitations: ['This aggregate alone does not establish causation.'], + }, + conditionalActions: [ + { + when: 'When the next task matches the same complexity and rubric', + action: 'Compare the current boundary with one smaller boundary.', + evidenceIds: [evidenceFixture.id], + stopCondition: 'Stop if the quality rubric fails.', + }, + ], + successCriteria: [ + { + metricObservationId: observationFixture.id, + direction: 'increase', + target: { kind: 'relative-change', value: 0.1 }, + minimumComparableTasks: 5, + qualityGuardrail: { + rubricId: 'acceptance-rubric-v1', + minimumScore: 0.8, + maximumRegression: 0.02, + }, + }, + ], +}; + +export function payloadInputFixture(): PrepareAdvicePayloadInput { + return { + locale: 'en', + aggregate: buildAdviceAggregateSnapshot(usageFixture, 'overall', 30), + sources: [sourceFixture], + observations: [observationFixture], + evidence: [evidenceFixture], + }; +} + +export function structuredOutputFixture(): unknown { + return { + schemaVersion: 1, + recommendations: [recommendationFixture], + }; +} From 158c4d94e7e55239f09d631ef0b015cb9ca020c5 Mon Sep 17 00:00:00 2001 From: Carl <74130028+Carl723000@users.noreply.github.com> Date: Sat, 29 Aug 2026 23:21:56 +0800 Subject: [PATCH 02/72] docs: define v2.3.1 feature and resource contract Co-authored-by: OpenAI Codex <215057067+openai-codex[bot]@users.noreply.github.com> --- V2.3.1-CANDIDATE-CONTRACT.md | 50 ++++++++++++++++++++++++++++++ V2.3.1-CANDIDATE-CONTRACT.zh-CN.md | 50 ++++++++++++++++++++++++++++++ 2 files changed, 100 insertions(+) create mode 100644 V2.3.1-CANDIDATE-CONTRACT.md create mode 100644 V2.3.1-CANDIDATE-CONTRACT.zh-CN.md diff --git a/V2.3.1-CANDIDATE-CONTRACT.md b/V2.3.1-CANDIDATE-CONTRACT.md new file mode 100644 index 00000000..bf22a5ac --- /dev/null +++ b/V2.3.1-CANDIDATE-CONTRACT.md @@ -0,0 +1,50 @@ +# v2.3.1 Local Candidate: Feature and Resource Contract + +Companion documents: [中文契约](V2.3.1-CANDIDATE-CONTRACT.zh-CN.md) · [English contract](V2.3.1-CANDIDATE-CONTRACT.md) + +## User benefit + +v2.3.1 converges existing advice into one verifiable loop. A user first sees the local aggregate evidence and success measure behind a recommendation, then chooses whether to use optional AI personalization. Afterwards they can record helpful, not helpful, and applied feedback, and compare tasks only when provider, task kind, complexity, measurement semantics, and quality guardrails match. When reliable pairs do not exist, the product says that evidence is insufficient instead of turning a fluctuation into a causal claim. + +The same release adds Codex hour detail for the last 30 days. Every populated date reads the sparse hourly aggregate already maintained by the provider index, with zero raw JSONL reads on click. Claude and Codex share the configured timezone and `HH:00` label format. + +## Furthest boundary for this candidate + +- AI sending reuses only the existing legal BYOK or user-configured endpoint and occurs only after an explicit click. Refreshes, background work, and hidden pages never generate AI requests. +- The host prepares one `PreparedAdvicePayload`. Preview, byte count, SHA-256, retries, and the actual request body consume the same canonical UTF-8 bytes from that object. Reconstruction or disagreement fails closed. +- The default payload contains allow-listed aggregates only. Prompt personalization is a separate, default-off, revocable consent; only text supplied or explicitly selected by the user can appear verbatim in preview and send. +- Feedback, comparable pairs, and the versioned comparison-result envelope remain in bounded local state. They never store prompts, responses, sessions, paths, project names, raw records, or free-text feedback. +- Legacy Get AI Advice and Prompt Optimizer share the preparation, preview, send, parse, and feedback boundary. Optimizer still processes only the user's pasted draft and never treats a permissive text fallback as structured advice. +- Account center, authentication, multi-account switching, achievements, GitHub, default/background AI, version changes, default-setting changes, release-workflow changes, and publishing remain out of scope. + +## Single data producer + +- `ClaudeUsageIndex` remains the sole reader of Claude logs. The existing persistent provider index and its worker remain the sole readers of Codex logs. +- Advice, feedback comparison, hourly expansion, and the Webview consume only main-index aggregates or bounded projections carrying `provider + measurementProfileVersion + revision + coverage`. They do not create a second scanner, scheduler, or competing cache of truth. +- The Codex index maintains the sparse last-30-day `date -> hour` buckets during its normal incremental pass and evicts day 31 in the configured timezone. Its one-time fill migration runs in the existing Codex worker and shares the same durable progress state. + +## Resource ownership and stop conditions + +| Resource | Creator | Creation condition | Stop condition and disposer | +| --- | --- | --- | --- | +| Claude / Codex recurring timers and watchers | Existing window lifecycle coordinator in `extension.ts` | Provider enabled and window lifecycle permits work | Window loses focus, provider is disabled, or extension disposes; lifecycle coordinator disposes them | +| Codex index / backfill worker | Existing `CodexIndexClient` | Main index needs a cold build, compatible migration, or incremental work and E-03 allows execution | Completion, checkpoint at budget exhaustion, user pause, provider disable, or extension dispose; client terminates the worker. A bounded first-availability backfill may continue after focus loss but never after explicit pause, disable, or dispose | +| AI network request | Single host sender | User clicks Send after preview, the Prepared object is valid, and BYOK configuration is legal | Response, error, user cancellation, consent withdrawal, feature disable, or extension dispose; the sender cancellation owner aborts it. No retry timer or background poll exists | +| Advice projections and comparison | No long-lived resource | Pure projection on a changed main-index revision, local feedback, or on-demand visible render | Released synchronously; no recomputation or render while hidden or disabled | +| E-03 progress record | Main-index durable state | First migration or backfill need | Not a runtime resource; completion retains only version, verified position, `failStreak`, `nextEligibleAt`, `pausedReason`, and required counters | + +The E-03 success path continues immediately without an artificial delay. Only failure, lack of progress, or explicit pause creates a durable cooldown. Ordinary refresh, watch, focus, reset, menu open, or extension upgrade cannot bypass `nextEligibleAt`. Restart resumes from the verified checkpoint. + +## Zero additional long-lived resources when off + +Relative to v2.3.0, a disabled, unconsented, unsent, or hidden advice loop owns zero new timers, watchers, workers, network requests, log enumeration/body reads, and hidden Webview renders. After the Codex base-index migration finishes or stops, the resource inventory returns to baseline; no empty loop remains for hypothetical future work. + +## Acceptance boundary + +- Red-green focused tests first cover canonical bytes, explicit sending, strict parsing, real pair ingestion, the versioned comparison envelope, resource registration/disposal, durable backoff/resume, and 30-day hourly expansion. +- Warm unchanged work produces zero JSONL body reads, derived mutations, and durable writes. Date expansion performs zero JSONL reads, and the hourly sum matches the daily total. +- All user-facing copy covers eight locales. UI state has operation -> full reload -> state remains or snapshot invalidates coverage. +- The candidate runs compile, focused and full Node/UI tests, disabled/unconsented zero-resource checks, warm-unchanged checks, VSIX inventory, and version/settings/workflow boundary checks, plus F5 and installed-VSIX smoke where feasible. + +--- +🤖 Generated with [OpenAI Codex](https://developers.openai.com/codex/) diff --git a/V2.3.1-CANDIDATE-CONTRACT.zh-CN.md b/V2.3.1-CANDIDATE-CONTRACT.zh-CN.md new file mode 100644 index 00000000..8237ddbe --- /dev/null +++ b/V2.3.1-CANDIDATE-CONTRACT.zh-CN.md @@ -0,0 +1,50 @@ +# v2.3.1 本地候选:功能与资源契约 + +配套文档:[中文契约](V2.3.1-CANDIDATE-CONTRACT.zh-CN.md) · [English contract](V2.3.1-CANDIDATE-CONTRACT.md) + +## 用户增益 + +v2.3.1 把现有建议收敛成一条可验证的闭环:用户先看到建议所依据的本地聚合证据和成功标准,再决定是否使用可选的 AI 个性化;之后可以记录“有用 / 无用 / 已应用”,并用满足相同 provider、任务类型、复杂度、测量口径和质量护栏的任务对判断是否观察到改善。证据不足时只显示“证据不足”,不把波动写成因果结论。 + +同版补齐 Codex 最近 30 天的日期到小时展开。任一有数据日期都从主索引已经保存的稀疏小时聚合读取,点击时不读原始 JSONL;Claude 与 Codex 使用同一配置时区和 `HH:00` 标签。 + +## 本轮最远边界 + +- AI 发送只复用现有合法 BYOK / 用户配置 endpoint,并且只由用户点击触发;没有默认、后台、刷新触发或隐藏页面触发的 AI 请求。 +- host 只准备一次 `PreparedAdvicePayload`。预览、字节数、SHA-256、重试与实际请求体都消费同一对象的同一组 canonical UTF-8 bytes,任何重建或不一致都失败关闭。 +- 默认只包含 allowlist 聚合。prompt 个性化是独立、默认关闭、可撤回的同意;只有用户提供或明确选择的文本才能逐字进入预览和发送。 +- 反馈、可比任务对和版本化 comparison result envelope 只保存在本地有界状态中;不保存 prompt、response、session、路径、项目名、原始记录或自由文本反馈。 +- 旧 Get AI Advice 与 Prompt Optimizer 复用同一准备、预览、发送、解析和反馈边界;Optimizer 仍只处理用户粘贴的草稿,不把宽松文本 fallback 当作结构化建议。 +- 不做账户中心、认证、多账户切换、成就页、GitHub、默认或后台 AI 请求;不改版本号、默认设置或 release workflow,也不发布。 + +## 唯一数据生产者 + +- Claude 原始日志的唯一读取者是 `ClaudeUsageIndex`;Codex 原始日志的唯一读取者是现有 persistent provider index 与其 worker。 +- advice、feedback comparison、小时展开和 Webview 只消费带 `provider + measurementProfileVersion + revision + coverage` 的主索引聚合或其有界投影;它们不建立第二套 scanner、缓存真相或 scheduler。 +- Codex 最近 30 天稀疏 `date -> hour` 桶由同一次主索引增量处理维护,并在配置时区下淘汰第 31 天。一次补齐迁移由现有 Codex worker 执行并复用同一持久进度状态。 + +## 资源所有权与停止条件 + +| 资源 | 创建者 | 创建条件 | 停止条件与实际销毁者 | +| --- | --- | --- | --- | +| Claude / Codex 周期 timer 与 watcher | `extension.ts` 的既有窗口生命周期协调器 | provider 启用且窗口处于允许状态 | 失焦、provider 关闭或 extension dispose;由生命周期协调器清理 | +| Codex index / backfill worker | 现有 `CodexIndexClient` | 主索引需要 cold build、兼容迁移或增量工作,且 E-03 允许执行 | 完成、预算耗尽并 checkpoint、用户暂停、provider 关闭或 extension dispose;由 client 终止 worker。首次可用性 backfill 可在失焦后有界继续,但不能越过明确暂停、关闭或 dispose | +| AI 网络请求 | 唯一 host sender | 用户预览后主动点击发送,Prepared 对象仍有效,且 BYOK 配置合法 | 响应、错误、用户取消、同意撤回、功能关闭或 extension dispose;由 sender 的 cancellation owner 中止。不存在 retry timer 或后台轮询 | +| 建议投影与 comparison | 不创建长期资源 | 仅在主索引 revision 变化、用户反馈或相关页面按需展示时执行纯投影 | 同步完成即释放;页面隐藏或功能关闭后不重算、不重绘 | +| E-03 进度记录 | 主索引 durable state | migration / backfill 第一次需要时 | 不是运行资源;完成后只保留版本、完成位置、`failStreak`、`nextEligibleAt`、`pausedReason` 与必要统计 | + +E-03 的成功路径立即继续,不人为等待。只有失败、无进展或明确暂停才设置持久 cooldown;普通 refresh、watch、focus、reset、菜单打开或插件升级不能绕过 `nextEligibleAt`。重启从已验证 checkpoint 继续。 + +## 关闭后的零新增长期资源 + +相对 v2.3.0:建议闭环关闭、未同意、未发送或页面隐藏时,新增资源必须为 0 timer、0 watcher、0 worker、0 network、0 额外日志枚举 / 正文读取、0 隐藏 Webview 重绘。Codex 基础索引迁移结束或停止后资源清单必须回到基线;不存在为“以后可能有用”保留的空循环。 + +## 验收边界 + +- red-green 聚焦测试先覆盖 canonical bytes、显式发送、严格解析、真实 pair ingestion、版本化 comparison envelope、资源登记 / 销毁、持久 backoff / resume 和 30 天小时展开。 +- warm unchanged 为 0 JSONL body read、0 派生 mutation、0 durable write;展开日期为 0 JSONL read,小时合计与当日汇总一致。 +- 所有用户文案覆盖八个 locale;UI 状态具备“操作 -> full reload -> 状态保留 / 快照失效”测试。 +- 最终运行 compile、聚焦与全量 Node/UI 测试、关闭 / 未授权零资源、warm unchanged、VSIX 清单及版本 / 设置 / workflow 边界检查;可行时完成 F5 与 installed-VSIX smoke。 + +--- +🤖 Generated with [OpenAI Codex](https://developers.openai.com/codex/) From 237d7e655882bec2400c334b39216df9be018c04 Mon Sep 17 00:00:00 2001 From: Carl <74130028+Carl723000@users.noreply.github.com> Date: Mon, 24 Aug 2026 03:21:14 +0800 Subject: [PATCH 03/72] feat: prepare v2.3.1 AI advice integration Add the default-off evidence spine, canonical consent-sealed payload previews, local feedback and comparison persistence, strict BYOK bridge, provider adapters, i18n, and regression coverage. Co-authored-by: OpenAI Codex <215057067+openai-codex[bot]@users.noreply.github.com> --- AI-ADVICE-INTEGRATION-DESIGN-V2.3.1.md | 99 ++ AI-ADVICE-INTEGRATION-DESIGN-V2.3.1.zh-CN.md | 76 ++ AI-ADVICE-INTEGRATION-V2.3.1.md | 243 +++++ AI-ADVICE-INTEGRATION-V2.3.1.zh-CN.md | 243 +++++ package.json | 2 +- src/adviceEffectiveness/adapters.ts | 597 +++++++++++ src/adviceEffectiveness/contract.ts | 1 + src/adviceEffectiveness/integration.ts | 111 ++ src/adviceEffectiveness/legacyBridge.ts | 129 +++ src/adviceEffectiveness/modelExperiment.ts | 125 +++ src/adviceEffectiveness/payload.ts | 47 +- .../versionedPersistence.ts | 612 ++++++++++++ src/advisor.ts | 14 +- src/claudeApiClient.ts | 6 +- src/claudeIncrementalIndex.ts | 6 + src/dataLoader.ts | 133 ++- src/extension.ts | 133 ++- src/i18n.ts | 459 +++++++++ src/promptOrigin.ts | 40 + src/settings.ts | 10 + src/test/adviceAdapters.test.ts | 312 ++++++ src/test/adviceEffectivenessI18n.test.ts | 106 ++ src/test/adviceIntegration.test.ts | 124 +++ src/test/adviceLegacyBridge.test.ts | 128 +++ src/test/adviceModelExperiment.test.ts | 117 +++ src/test/advicePayload.test.ts | 31 +- src/test/advicePersistence.test.ts | 293 ++++++ src/test/adviceRuntimePolicy.test.ts | 24 + src/test/dataLoader.test.ts | 118 +++ src/test/promptOrigin.test.ts | 36 + src/types.ts | 41 +- src/webview.ts | 945 +++++++++++++++++- tests/ui/advice-effectiveness.spec.mjs | 426 ++++++++ .../support/advice-effectiveness-fixture.cjs | 179 ++++ tests/ui/support/render-harness.cjs | 69 +- tests/ui/support/server.mjs | 24 +- 36 files changed, 5974 insertions(+), 85 deletions(-) create mode 100644 AI-ADVICE-INTEGRATION-DESIGN-V2.3.1.md create mode 100644 AI-ADVICE-INTEGRATION-DESIGN-V2.3.1.zh-CN.md create mode 100644 AI-ADVICE-INTEGRATION-V2.3.1.md create mode 100644 AI-ADVICE-INTEGRATION-V2.3.1.zh-CN.md create mode 100644 src/adviceEffectiveness/adapters.ts create mode 100644 src/adviceEffectiveness/integration.ts create mode 100644 src/adviceEffectiveness/legacyBridge.ts create mode 100644 src/adviceEffectiveness/modelExperiment.ts create mode 100644 src/adviceEffectiveness/versionedPersistence.ts create mode 100644 src/promptOrigin.ts create mode 100644 src/test/adviceAdapters.test.ts create mode 100644 src/test/adviceEffectivenessI18n.test.ts create mode 100644 src/test/adviceIntegration.test.ts create mode 100644 src/test/adviceLegacyBridge.test.ts create mode 100644 src/test/adviceModelExperiment.test.ts create mode 100644 src/test/advicePersistence.test.ts create mode 100644 src/test/adviceRuntimePolicy.test.ts create mode 100644 src/test/promptOrigin.test.ts create mode 100644 tests/ui/advice-effectiveness.spec.mjs create mode 100644 tests/ui/support/advice-effectiveness-fixture.cjs diff --git a/AI-ADVICE-INTEGRATION-DESIGN-V2.3.1.md b/AI-ADVICE-INTEGRATION-DESIGN-V2.3.1.md new file mode 100644 index 00000000..215a554f --- /dev/null +++ b/AI-ADVICE-INTEGRATION-DESIGN-V2.3.1.md @@ -0,0 +1,99 @@ +# v2.3.1 AI Advice Effectiveness: Experimental UI Design Plan + +> Status: v2.3.1 integration candidate only. The feature flag is off by default; +> this document is not a current-release commitment. + +## Goal and boundaries + +The UI answers one question: can a user see which local evidence supports an +advice candidate, exactly what would leave the device, and how effectiveness +would be judged before opting in? It joins the existing AI Advice / Codex local +optimization areas on the Content tab. It does not add a third top-level entry, +replace Prompt Optimizer, or remove the existing BYOK path. + +Nothing is rendered while the experiment is off. Enabling it still performs no +automatic network call. This iteration provides an exact sealed payload preview, +local feedback, and evidence-bound result states, with no send button or production +endpoint. + +## Visual system + +- **Color:** use existing VS Code theme tokens only. Background and copy use + `--vscode-editor-background` / `--vscode-foreground`; borders and secondary + copy use `--vscode-panel-border` / `--vscode-descriptionForeground`; actions + and focus use `--vscode-button-background` / `--vscode-focusBorder`; the + snapshot uses `--vscode-editorWidget-background`. Light+ and Dark+ remain + host-owned, with no new gradient or hard-coded brand accent. +- **Type:** explanatory copy uses `--vscode-font-family` at the existing 13px + density. Metrics, byte counts, and JSON use `--vscode-editor-font-family`. + Small 10–11px status stamps never carry meaning by color alone. +- **Spacing:** reuse `.action-card`, `.model-details-stacked`, and the existing + button/input rhythm. Avoid a new oversized KPI-card language. +- **Signature:** one continuous evidence spine joins five ordered stages. The + sealed snapshot header reads `SEALED · N BYTES · SHA-256`, and its body is + decoded verbatim from the same canonical UTF-8 bytes. + +## Information architecture and wireframe + +Wide layout: + +```text +┌ AI advice effectiveness (experimental) ──────────────────────────┐ +│ [Observed] ─── [Evidence] ─── [Advice] ─── [Action] ─── [Result] │ +│ values strength conditional local feedback │ +│ sample/guardrail│ +│ │ +│ □ Allow aggregates only □ Allow prompt samples (separate, off) │ +│ [Create sealed snapshot] │ +│ ▾ SEALED · 1248 BYTES · SHA-256 · AGGREGATES ONLY │ +│ {"schemaVersion":1,...} │ +└──────────────────────────────────────────────────────────────────┘ +``` + +At narrow widths the five stages stack vertically and the spine becomes a left +rail. Consent controls and actions wrap naturally. JSON scrolls horizontally; +actual transmitted bytes are never compressed or elided. + +## Interaction and accessibility + +- Aggregate consent and prompt-personalization consent are separate values in + version-migrated VS Code `globalState`. Both start off; prompt consent is + disabled until aggregate consent is selected. Every included prompt is + visible in the sealed snapshot. +- “Create sealed snapshot” asks the extension host to prepare the object. The + host rechecks the flag and consent, serializes once, and retains canonical + UTF-8 bytes. Preview text is decoded from those bytes; byte count and SHA-256 + are derived from the same bytes; a future transport may consume only that + Prepared object and cannot reserialize it. +- Helpful / not helpful / applied are native buttons with `aria-pressed`. + Helpful and not helpful are mutually exclusive; applied is independent. The + events write only to VS Code `globalState`. +- Every action is keyboard reachable. Headings, lists, `fieldset` / `legend`, + `aria-live`, and VS Code focus tokens give screen-reader and focus semantics. + Consent, disclosure, and provider UI state survive a webview reload. +- The result stage says “insufficient evidence” until comparable sample and + quality guardrails are met; a one-off movement is never called effective. + +## Content and privacy rules + +- The default route carries same-window aggregate numbers, coarse model families, + allow-listed metrics, confidence, and evidence references only. It never accepts + or serializes raw sessions, per-record data, prompts, responses, project paths, + cwd values, or stable session IDs. +- Long sessions use elapsed-duration as a proxy; large context uses numeric token + thresholds. A conditional “start a new session / `/clear` at a stable task + boundary” action appears only when deterministic thresholds and sample rules pass. +- Topic drift is semantic. It may enter a future model path only after separate + prompt-personalization consent, with every prompt visible in the exact preview. + This iteration never infers drift from body-free signals. +- Strict parse failure, unknown persistence versions, damaged state, or incomplete + evidence closes the result and never produces a permissive fallback suggestion. + +## Design critique and revision + +An early direction risked becoming another generic dashboard and a third isolated +experience. The revision embeds the experiment in the existing Advice / Codex card, +reuses the action-card language, and makes one continuous evidence spine express the +evidence boundary. The only new visual signature is the sealed byte snapshot because +it directly supports privacy review. Narrow layouts, keyboard use, screen readers, +and theme changes are launch checks; with the flag off, the current UI is unchanged. diff --git a/AI-ADVICE-INTEGRATION-DESIGN-V2.3.1.zh-CN.md b/AI-ADVICE-INTEGRATION-DESIGN-V2.3.1.zh-CN.md new file mode 100644 index 00000000..73c25c6e --- /dev/null +++ b/AI-ADVICE-INTEGRATION-DESIGN-V2.3.1.zh-CN.md @@ -0,0 +1,76 @@ +# v2.3.1 AI 建议有效性:实验界面设计计划 + +> 状态:仅用于 v2.3.1 候选集成。功能开关默认关闭;本文不代表当前发布承诺。 + +## 设计目标与边界 + +界面服务于一个明确问题:用户能否先看清“哪些本地证据支持哪条建议、若继续会发送什么、 +应用后怎样判断是否有效”,再决定是否参与实验。它接入现有 Content 页的 AI Advice / Codex +本地优化建议区域,不增加第三个顶层入口,不替换 Prompt Optimizer,也不改变现有 BYOK 能力。 + +实验关闭时不渲染此界面。实验打开后也不会自动联网;本轮只提供精确 payload 密封预览、 +本地反馈和证据不足状态,不提供发送按钮或生产端点。 + +## 视觉系统 + +- **颜色**:只使用现有 VS Code 主题令牌。背景与正文使用 + `--vscode-editor-background` / `--vscode-foreground`;边界和次级文本使用 + `--vscode-panel-border` / `--vscode-descriptionForeground`;交互和焦点使用 + `--vscode-button-background` / `--vscode-focusBorder`;快照使用 + `--vscode-editorWidget-background` 与编辑器等宽字体。Light+ 与 Dark+ 均由宿主主题决定, + 不引入品牌外的渐变或硬编码强调色。 +- **字体**:说明文字沿用 `--vscode-font-family` 与现有 13px 密度;指标、字节数和 JSON + 使用 `--vscode-editor-font-family`。小型状态章保持 10–11px,但不承载唯一语义。 +- **空间**:沿用 `.action-card`、`.model-details-stacked`、现有按钮与输入组件的间距, + 保持 Content 页的信息密度,不新增“大号 KPI 卡片”风格。 +- **标志性元素**:一条连续的“证据脊柱”连接五个有序阶段;密封快照标题显示 + `SEALED · N BYTES · SHA-256`,正文逐字呈现同一组 canonical UTF-8 bytes。 + +## 信息架构与草图 + +宽屏: + +```text +┌ AI 建议有效性(实验)──────────────────────────────────────────────┐ +│ [观察] ─── [证据] ─── [建议] ─── [行动] ─── [结果] │ +│ 数值 强度 有条件 本地反馈 样本/质量护栏 │ +│ │ +│ □ 仅允许聚合数据 □ 允许附加用户 prompt(默认关,独立同意) │ +│ [生成密封快照] │ +│ ▾ SEALED · 1248 BYTES · SHA-256 · AGGREGATES ONLY │ +│ {"schemaVersion":1,...} │ +└──────────────────────────────────────────────────────────────────┘ +``` + +窄屏时五个阶段改为纵向排列,脊柱成为左侧竖线;同意项和按钮自然换行,JSON 区域横向滚动, +不压缩或省略发送字节。 + +## 交互与可访问性 + +- 两项同意分开保存于带版本迁移的 VS Code `globalState`:聚合数据默认未同意;prompt 个性化默认关闭, + 且只有聚合同意后才可选择。prompt 的正文必然出现在密封快照中。 +- “生成密封快照”只向 extension host 请求准备对象。host 再次验证实验开关与同意状态, + 只序列化一次并保留 canonical UTF-8 bytes;预览正文从同一 bytes 解码,byte count 与 SHA-256 + 也从同一 bytes 派生,未来 transport 只能接收这个 Prepared 对象,不能重新序列化。 +- 有用 / 无用 / 已应用使用原生 `'; + feedbackHtml = + '
' + + '
' + html(t.feedbackTitle) + '
' + + '
' + + button('helpful', t.helpful, feedback?.rating === 'helpful') + + button('not-helpful', t.notHelpful, feedback?.rating === 'not-helpful') + + button('applied', t.applied, feedback?.applied === 'applied') + + '
' + + '

' + html(t.feedbackLocalOnly) + '

' + + '

' + + '
'; + } + + let payloadHtml = '

' + html(t.codexPreviewUnavailable) + '

'; + if (state?.provider === 'claude' && state.remotePreviewEligible && state.aggregate) { + const ready = this.adviceLocalStateStatus === 'ready'; + const aggregateChecked = ready && this.adviceLocalState.aggregateConsent === 'explicit'; + const promptAvailable = state.promptSamples.length > 0; + const promptChecked = + aggregateChecked && promptAvailable && this.adviceLocalState.promptSampleConsent === 'explicit'; + const aggregateDisabled = ready ? '' : ' disabled'; + const promptDisabled = ready && aggregateChecked && promptAvailable ? '' : ' disabled'; + const previewDisabled = ready && aggregateChecked ? '' : ' disabled'; + payloadHtml = + '
' + + '
' + html(t.payloadTitle) + '
' + + '

' + html(t.payloadDescription) + '

' + + '' + + '
' + html(t.noNetworkTransport) + '
' + + '

' + + '
'; + } + + const step = (index: number, label: string, content: string): string => + '
  • ' + + '
    ' + html(label) + '

    ' + content + '

  • '; + return ( + '
    ' + + '

    ' + html(t.title) + + ' v2.3.1

    ' + html(t.description) + '

    ' + + '

    ' + html(t.candidateNotice) + '

    ' + + '
      ' + + step(1, t.observation, observations.map((item) => html(item)).join('
      ')) + + step(2, t.evidence, html(evidenceText) + '
      ' + html(limitationText) + '') + + step(3, t.recommendation, html(recommendationText)) + + step(4, t.action, html(actionText)) + + step(5, t.result, html(resultText) + '
      ' + html(pairText + ' · ' + minimumText) + '') + + '
    ' + + feedbackHtml + payloadHtml + + '
    ' + ); + } + /** * Prominent "AI advice" card at the top of the Content tab (Phase 9a). The * advice button used to live in the content-analysis header; this gives it a @@ -3567,6 +4026,7 @@ export class UsageWebviewProvider { '' + '' + '' + + this.renderAdviceEffectivenessBody('claude') + '' ); } @@ -4010,7 +4470,8 @@ export class UsageWebviewProvider { : '

    ' + this.escapeHtml(copy.recommendationPartial) + '

    '; return '
    ✨' + '

    ' + this.escapeHtml(copy.optimization) + '

    ' + - '

    ' + this.escapeHtml(copy.structuralProxy) + '

    ' + + '

    ' + this.escapeHtml(copy.structuralProxy) + '

    ' + + this.renderAdviceEffectivenessBody('codex') + '' + '

    ' + this.escapeHtml(copy.behavior + ' · ' + copy.last30Days) + '

    ' + partial + behaviorSummary + insightHtml + '
    '; @@ -6419,6 +6880,262 @@ export class UsageWebviewProvider { color: var(--vscode-button-foreground); border-color: var(--vscode-button-background); } + + .advice-effectiveness-body { + margin-top: 13px; + padding-top: 14px; + border-top: 1px solid var(--vscode-panel-border); + background: var(--vscode-editor-background); + min-width: 0; + } + .advice-effectiveness-heading h4, + .advice-feedback-section h5, + .advice-payload-section h5 { + margin: 0; + font-size: 12px; + font-weight: 600; + } + .advice-effectiveness-heading h4 { + display: flex; + align-items: center; + gap: 7px; + } + .advice-effectiveness-heading p, + .advice-candidate-note, + .advice-local-note, + .advice-inline-status { + margin: 4px 0 0; + color: var(--vscode-descriptionForeground); + font-size: 11px; + line-height: 1.45; + } + .advice-candidate-note { + margin-top: 7px; + } + .advice-spine { + position: relative; + display: grid; + grid-template-columns: repeat(5, minmax(0, 1fr)); + gap: 10px; + list-style: none; + margin: 14px 0 12px; + padding: 0; + } + .advice-spine::before { + content: ""; + position: absolute; + z-index: 0; + left: 10%; + right: 10%; + top: 13px; + height: 1px; + background: var(--vscode-textLink-foreground); + } + .advice-spine li { + position: relative; + z-index: 1; + min-width: 0; + } + .advice-step-marker { + display: flex; + width: 26px; + height: 26px; + margin: 0 auto 7px; + align-items: center; + justify-content: center; + border: 1px solid var(--vscode-textLink-foreground); + border-radius: 999px; + background: var(--vscode-editor-background); + color: var(--vscode-textLink-foreground); + font-size: 10px; + font-weight: 700; + } + .advice-spine li > div { + text-align: center; + } + .advice-spine strong { + font-size: 11px; + } + .advice-spine p { + margin: 3px 0 0; + color: var(--vscode-descriptionForeground); + font-size: 10.5px; + line-height: 1.4; + overflow-wrap: anywhere; + } + .advice-step-caveat { + font-size: 10px; + } + .advice-feedback-section, + .advice-payload-section { + margin-top: 12px; + padding-top: 11px; + border-top: 1px solid var(--vscode-panel-border); + } + .advice-feedback-group { + display: flex; + flex-wrap: wrap; + gap: 7px; + margin-top: 7px; + } + .advice-feedback-button { + padding: 4px 9px; + border: 1px solid var(--vscode-button-secondaryBackground, var(--vscode-panel-border)); + border-radius: 999px; + background: transparent; + color: var(--vscode-foreground); + cursor: pointer; + font-size: 11px; + } + .advice-feedback-button.is-selected { + border-width: 2px; + border-color: var(--vscode-focusBorder); + font-weight: 600; + } + .advice-feedback-button:disabled, + .advice-consent input:disabled { + opacity: 0.55; + cursor: default; + } + .advice-feedback-button:focus-visible, + .advice-consent input:focus-visible, + .advice-payload-actions button:focus-visible, + .advice-payload-preview summary:focus-visible, + .advice-payload-preview pre:focus-visible { + outline: 2px solid var(--vscode-focusBorder); + outline-offset: 2px; + } + .advice-consent { + display: grid; + gap: 8px; + margin: 9px 0 0; + padding: 10px; + border: 1px solid var(--vscode-panel-border); + border-radius: 6px; + } + .advice-consent legend { + padding: 0 5px; + color: var(--vscode-descriptionForeground); + font-size: 10px; + } + .advice-consent label { + display: flex; + align-items: flex-start; + gap: 8px; + cursor: pointer; + } + .advice-consent input { + flex: 0 0 auto; + margin-top: 2px; + } + .advice-consent label span { + display: grid; + gap: 2px; + min-width: 0; + } + .advice-consent label strong, + .advice-consent label small { + font-size: 11px; + line-height: 1.4; + } + .advice-consent label small { + color: var(--vscode-descriptionForeground); + } + .advice-payload-actions { + display: flex; + flex-wrap: wrap; + align-items: center; + gap: 8px; + margin-top: 9px; + } + .advice-payload-actions span { + color: var(--vscode-descriptionForeground); + font-size: 10.5px; + } + .advice-payload-preview { + margin-top: 9px; + border: 1px solid var(--vscode-panel-border); + border-radius: 6px; + overflow: hidden; + } + .advice-payload-preview summary { + display: flex; + align-items: center; + justify-content: space-between; + gap: 8px; + padding: 7px 9px; + cursor: pointer; + background: var(--vscode-sideBar-background, var(--vscode-editor-background)); + font-size: 11px; + font-weight: 600; + } + .advice-seal-stamp { + flex: 0 0 auto; + padding: 1px 5px; + border: 1px solid var(--vscode-textLink-foreground); + border-radius: 3px; + color: var(--vscode-textLink-foreground); + font-family: var(--vscode-editor-font-family, monospace); + font-size: 9px; + letter-spacing: 0.4px; + } + .advice-payload-meta { + display: flex; + flex-wrap: wrap; + gap: 5px 10px; + padding: 8px 9px 0; + color: var(--vscode-descriptionForeground); + font-size: 10px; + } + .advice-payload-meta code { + flex-basis: 100%; + color: var(--vscode-foreground); + overflow-wrap: anywhere; + } + .advice-payload-preview pre { + max-height: 280px; + margin: 8px 9px 9px; + padding: 9px; + overflow: auto; + border: 1px solid var(--vscode-panel-border); + border-radius: 4px; + background: var(--vscode-textCodeBlock-background, var(--vscode-editor-background)); + color: var(--vscode-editor-foreground, var(--vscode-foreground)); + font-family: var(--vscode-editor-font-family, monospace); + font-size: 10px; + line-height: 1.45; + white-space: pre; + } + @media (max-width: 800px) { + .action-card-head { + align-items: flex-start; + flex-wrap: wrap; + } + .advice-spine { + grid-template-columns: minmax(0, 1fr); + gap: 9px; + padding-left: 0; + } + .advice-spine::before { + left: 13px; + right: auto; + top: 13px; + bottom: 13px; + width: 1px; + height: auto; + } + .advice-spine li { + display: grid; + grid-template-columns: 26px minmax(0, 1fr); + gap: 9px; + } + .advice-step-marker { + margin: 0; + } + .advice-spine li > div { + text-align: left; + } + } .sr-only { position: absolute; width: 1px; @@ -6437,6 +7154,7 @@ export class UsageWebviewProvider { return ` // Get VSCode API const vscode = acquireVsCodeApi(); +const __adviceCopy = ${JSON.stringify(I18n.t.popup.adviceEffectiveness)}; function ccuReadUiState() { try { return vscode.getState() || {}; } catch (e) { return {}; } @@ -6459,6 +7177,63 @@ function ccuElementStateKey(element, kind) { return ccuProviderName() + ':' + (tab ? tab.id : 'page') + ':' + kind + ':' + Math.max(0, elements.indexOf(element)); } +function adviceRoot(provider) { + return document.querySelector('.advice-effectiveness-body[data-advice-provider="' + provider + '"]'); +} +function adviceConsentElements(provider) { + var root = adviceRoot(provider); + return { + root: root, + aggregate: root ? root.querySelector('[data-advice-consent-kind="aggregate"]') : null, + prompt: root ? root.querySelector('[data-advice-consent-kind="prompt"]') : null, + previewButton: root ? root.querySelector('[data-advice-action="preview"]') : null, + preview: root ? root.querySelector('[data-advice-preview]') : null, + consentStatus: root ? root.querySelector('[data-advice-consent-status]') : null, + }; +} +function adviceSyncConsentControls(provider) { + var elements = adviceConsentElements(provider); + if (!elements.root || !elements.aggregate || !elements.prompt) { return; } + var promptAvailable = elements.prompt.getAttribute('data-prompt-available') === 'true'; + elements.prompt.disabled = elements.aggregate.disabled || !elements.aggregate.checked || !promptAvailable; + if (elements.prompt.disabled) { elements.prompt.checked = false; } + if (elements.previewButton) { + elements.previewButton.disabled = elements.aggregate.disabled || !elements.aggregate.checked; + } +} +function adviceClearPreview(provider) { + var elements = adviceConsentElements(provider); + if (!elements.preview) { return; } + elements.preview.hidden = true; + elements.preview.open = false; + elements.preview.removeAttribute('data-snapshot-id'); + var body = elements.preview.querySelector('[data-advice-preview-body]'); + var digest = elements.preview.querySelector('[data-advice-preview-digest]'); + var mode = elements.preview.querySelector('[data-advice-preview-mode]'); + var contentType = elements.preview.querySelector('[data-advice-preview-content-type]'); + var bytes = elements.preview.querySelector('[data-advice-preview-bytes]'); + var count = elements.preview.querySelector('[data-advice-preview-count]'); + if (body) { body.textContent = ''; } + if (digest) { digest.textContent = ''; } + if (mode) { mode.textContent = ''; } + if (contentType) { contentType.textContent = ''; } + if (bytes) { bytes.textContent = ''; } + if (count) { count.textContent = ''; } +} +function adviceSetConsentPending(provider, pending) { + var elements = adviceConsentElements(provider); + if (elements.aggregate) { elements.aggregate.disabled = !!pending; } + if (elements.prompt) { elements.prompt.disabled = !!pending; } + if (elements.previewButton) { elements.previewButton.disabled = true; } + if (!pending) { adviceSyncConsentControls(provider); } +} +function restoreAdviceEffectivenessState() { + document.querySelectorAll('.advice-effectiveness-body[data-advice-provider]').forEach(function(root) { + var provider = root.getAttribute('data-advice-provider'); + if (provider) { adviceSyncConsentControls(provider); } + }); +} + // Keep expanded
    open across auto-refresh re-renders — a // data refresh shouldn't collapse something you're reading. Only a tab switch // resets to the default (see clearPersistedDetails in showTab). @@ -6505,6 +7280,7 @@ function restoreUi() { restoreActiveTab(); restoreSessionFilter(); restorePersistedDetails(); + restoreAdviceEffectivenessState(); restoreSessionDetails(); restoreTableSorts(); restoreChartMetrics(); @@ -7240,10 +8016,173 @@ window.syncChartBarSelection = syncChartBarSelection; window.closeAllHourlyDetails = closeAllHourlyDetails; window.closeAllMonthlyDetails = closeAllMonthlyDetails; +document.addEventListener('change', function(event) { + var input = event.target && event.target.closest + ? event.target.closest('[data-advice-consent-kind][data-provider]') + : null; + if (!input) { return; } + var provider = input.getAttribute('data-provider'); + if (provider !== 'claude' && provider !== 'codex') { return; } + var elements = adviceConsentElements(provider); + if (!elements.aggregate || !elements.prompt) { return; } + if (!elements.aggregate.checked) { elements.prompt.checked = false; } + adviceSyncConsentControls(provider); + adviceClearPreview(provider); + vscode.postMessage({ command: 'discardAdviceSnapshot', provider: provider }); + adviceSetConsentPending(provider, true); + vscode.postMessage({ + command: 'updateAdviceConsent', + provider: provider, + aggregateConsent: elements.aggregate.checked ? 'explicit' : 'not-granted', + promptSampleConsent: elements.prompt.checked ? 'explicit' : 'not-granted', + }); +}); + +document.addEventListener('click', function(event) { + var action = event.target && event.target.closest + ? event.target.closest('[data-advice-action][data-provider]') + : null; + if (!action) { return; } + var provider = action.getAttribute('data-provider'); + if (provider !== 'claude' && provider !== 'codex') { return; } + if (action.getAttribute('data-advice-action') === 'preview') { + event.preventDefault(); + var elements = adviceConsentElements(provider); + if (!elements.aggregate || !elements.prompt || !elements.aggregate.checked) { return; } + action.disabled = true; + if (elements.consentStatus) { elements.consentStatus.textContent = ''; } + vscode.postMessage({ + command: 'prepareAdviceSnapshot', + provider: provider, + aggregateConsent: 'explicit', + promptSampleConsent: elements.prompt.checked ? 'explicit' : 'not-granted', + }); + return; + } + if (action.getAttribute('data-advice-action') === 'feedback') { + event.preventDefault(); + var root = adviceRoot(provider); + if (!root) { return; } + root.querySelectorAll('[data-advice-action="feedback"]').forEach(function(button) { + button.disabled = true; + }); + var status = root.querySelector('[data-advice-feedback-status]'); + if (status) { status.textContent = ''; } + vscode.postMessage({ + command: 'recordAdviceFeedback', + provider: provider, + adviceId: action.getAttribute('data-advice-id'), + recommendationId: action.getAttribute('data-recommendation-id'), + kind: action.getAttribute('data-feedback-kind'), + }); + } +}); + // Handle messages from extension window.addEventListener('message', function(event) { const message = event.data; + if (message.command === 'adviceConsentResult') { + var consentProvider = message.provider; + var consentElements = adviceConsentElements(consentProvider); + if (consentElements.aggregate && consentElements.prompt) { + if (message.ok === true) { + consentElements.aggregate.checked = message.aggregateConsent === 'explicit'; + consentElements.prompt.checked = + consentElements.aggregate.checked && message.promptSampleConsent === 'explicit'; + adviceSetConsentPending(consentProvider, false); + if (consentElements.consentStatus) { consentElements.consentStatus.textContent = ''; } + } else { + consentElements.aggregate.checked = false; + consentElements.prompt.checked = false; + consentElements.aggregate.disabled = true; + consentElements.prompt.disabled = true; + if (consentElements.previewButton) { consentElements.previewButton.disabled = true; } + if (consentElements.consentStatus) { + consentElements.consentStatus.textContent = __adviceCopy.feedbackSaveFailed; + } + } + } + } + + if (message.command === 'adviceSnapshotResult') { + var snapshotProvider = message.provider; + var snapshotElements = adviceConsentElements(snapshotProvider); + if (snapshotElements.previewButton) { snapshotElements.previewButton.disabled = false; } + var validSnapshot = + message.ok === true && + typeof message.snapshotId === 'string' && /^snapshot-[a-f0-9]{24}$/.test(message.snapshotId) && + typeof message.body === 'string' && + typeof message.sha256 === 'string' && /^[a-f0-9]{64}$/.test(message.sha256) && + Number.isInteger(message.utf8Bytes) && message.utf8Bytes >= 0 && + Number.isInteger(message.promptSampleCount) && message.promptSampleCount >= 0 && + message.contentType === 'application/json' && + (message.dataMode === 'aggregates-only' || message.dataMode === 'aggregates-with-prompt-samples'); + if (!validSnapshot || !snapshotElements.preview) { + adviceClearPreview(snapshotProvider); + if (snapshotElements.consentStatus) { + snapshotElements.consentStatus.textContent = __adviceCopy.strictOutputRejected; + } + } else { + var preview = snapshotElements.preview; + var previewBody = preview.querySelector('[data-advice-preview-body]'); + var previewDigest = preview.querySelector('[data-advice-preview-digest]'); + var previewMode = preview.querySelector('[data-advice-preview-mode]'); + var previewContentType = preview.querySelector('[data-advice-preview-content-type]'); + var previewBytes = preview.querySelector('[data-advice-preview-bytes]'); + var previewCount = preview.querySelector('[data-advice-preview-count]'); + if (previewBody) { previewBody.textContent = message.body; } + if (previewDigest) { previewDigest.textContent = 'SHA-256 ' + message.sha256; } + if (previewContentType) { previewContentType.textContent = message.contentType; } + if (previewMode) { + previewMode.textContent = message.dataMode === 'aggregates-only' + ? __adviceCopy.aggregatesOnly + : __adviceCopy.aggregatesWithPromptSamples; + } + if (previewBytes) { + previewBytes.textContent = __adviceCopy.payloadBytes.replace('{bytes}', String(message.utf8Bytes)); + } + if (previewCount) { + previewCount.textContent = __adviceCopy.promptSamplesIncluded.replace( + '{count}', + String(message.promptSampleCount), + ); + } + preview.setAttribute('data-snapshot-id', message.snapshotId); + preview.hidden = false; + preview.open = true; + if (snapshotElements.consentStatus) { + snapshotElements.consentStatus.textContent = __adviceCopy.noNetworkTransport; + } + } + adviceSyncConsentControls(snapshotProvider); + } + + if (message.command === 'adviceFeedbackResult') { + var feedbackProvider = message.provider; + var feedbackRoot = adviceRoot(feedbackProvider); + if (feedbackRoot) { + var feedbackStatus = feedbackRoot.querySelector('[data-advice-feedback-status]'); + if (message.ok === true) { + feedbackRoot.querySelectorAll('[data-advice-action="feedback"]').forEach(function(button) { + var kind = button.getAttribute('data-feedback-kind'); + var selected = + (kind === 'helpful' && message.rating === 'helpful') || + (kind === 'not-helpful' && message.rating === 'not-helpful') || + (kind === 'applied' && message.applied === 'applied'); + button.disabled = false; + button.setAttribute('aria-pressed', selected ? 'true' : 'false'); + button.classList.toggle('is-selected', selected); + var marker = button.querySelector('[aria-hidden="true"]'); + if (marker) { marker.textContent = selected ? '✓' : '○'; } + }); + if (feedbackStatus) { feedbackStatus.textContent = __adviceCopy.feedbackLocalOnly; } + } else { + if (feedbackStatus) { feedbackStatus.textContent = __adviceCopy.feedbackSaveFailed; } + } + } + } + if (message.command === 'shareCardResult') { const prev = document.getElementById('scPreview'); if (prev) { diff --git a/tests/ui/advice-effectiveness.spec.mjs b/tests/ui/advice-effectiveness.spec.mjs new file mode 100644 index 00000000..1ae0e668 --- /dev/null +++ b/tests/ui/advice-effectiveness.spec.mjs @@ -0,0 +1,426 @@ +import { createHash } from 'node:crypto'; +import { createRequire } from 'node:module'; +import AxeBuilder from '@axe-core/playwright'; + +import { test, expect, openClaude, openCodex } from './support/app.mjs'; + +const require = createRequire(import.meta.url); +const { + PROMPT_SENTINEL, + buildAdviceEffectivenessFixture, +} = require('./support/advice-effectiveness-fixture.cjs'); + +const locales = ['en', 'de-DE', 'zh-TW', 'zh-CN', 'ja', 'ko', 'pt-BR', 'id']; + +async function openCandidate(page, provider, options = {}) { + const open = provider === 'claude' ? openClaude : openCodex; + await open(page, { ...options, fixture: 'advice-effectiveness' }); + await page.locator('#tab-content').click(); + const root = page.locator( + `.advice-effectiveness-body[data-advice-provider="${provider}"]`, + ); + await expect(root).toBeVisible(); + return root; +} + +async function dispatchHostMessage(page, message) { + await page.evaluate((data) => { + window.dispatchEvent(new MessageEvent('message', { data })); + }, message); +} + +async function postedMessages(page, command) { + return page.evaluate((expectedCommand) => + window.__ccuPostedMessages.filter((message) => message.command === expectedCommand), + command); +} + +async function expectPostedCount(page, command, count) { + await expect.poll(async () => (await postedMessages(page, command)).length).toBe(count); +} + +async function grantAggregateConsent(page) { + const aggregate = page.locator( + '[data-advice-provider="claude"] [data-advice-consent-kind="aggregate"]', + ); + await aggregate.focus(); + await aggregate.press('Space'); + await expectPostedCount(page, 'updateAdviceConsent', 1); + await dispatchHostMessage(page, { + command: 'adviceConsentResult', + ok: true, + provider: 'claude', + aggregateConsent: 'explicit', + promptSampleConsent: 'not-granted', + }); + await expect(aggregate).toBeChecked(); +} + +async function seriousOrCriticalViolations(page, include) { + const builder = new AxeBuilder({ page }); + if (include) builder.include(include); + const results = await builder.analyze(); + const violations = []; + for (const violation of results.violations + .filter((item) => item.impact === 'serious' || item.impact === 'critical')) { + const targets = []; + for (const node of violation.nodes) { + const contrast = node.any.map((check) => check.data).find((data) => data?.contrastRatio); + const themedAccent = violation.id === 'color-contrast' && contrast + ? await page.evaluate((target) => { + const element = document.querySelector(target.join(' ')); + if (!element) return null; + const style = getComputedStyle(element); + const probe = document.createElement('span'); + document.body.appendChild(probe); + const resolve = (value) => { + probe.style.color = value; + return getComputedStyle(probe).color; + }; + const root = getComputedStyle(document.documentElement); + const accentTokens = [ + '--vscode-charts-blue', '--vscode-charts-green', '--vscode-charts-orange', + '--vscode-charts-purple', '--vscode-charts-red', '--vscode-charts-yellow', + '--vscode-focusBorder', + ]; + const isAccent = accentTokens.some((token) => + resolve(root.getPropertyValue(token).trim()) === style.color); + probe.remove(); + const weight = Number.parseInt(style.fontWeight, 10) || 400; + const hasUnderline = style.textDecorationLine !== 'none' + || (Number.parseFloat(style.borderBottomWidth) >= 2 && style.borderBottomStyle !== 'none'); + const before = getComputedStyle(element, '::before').content; + const after = getComputedStyle(element, '::after').content; + const hasIcon = (before && before !== 'none' && before !== 'normal') + || (after && after !== 'none' && after !== 'normal') + || Boolean(element.querySelector('svg, .codicon, [aria-hidden="true"]')); + return { isAccent, hasNonColorCue: weight >= 600 || hasUnderline || hasIcon }; + }, node.target) + : null; + if (!themedAccent?.isAccent + || Number(contrast.contrastRatio) < 3 + || !themedAccent.hasNonColorCue) { + targets.push(node.target); + } + } + if (targets.length) violations.push({ + id: violation.id, + impact: violation.impact, + targets, + }); + } + return violations; +} + +test('candidate surface is absent from the default-off fixture', async ({ page }) => { + await openClaude(page, { fixture: 'advice-effectiveness-disabled' }); + await page.locator('#tab-content').click(); + await expect(page.locator('.advice-effectiveness-body')).toHaveCount(0); + + await openCodex(page, { fixture: 'advice-effectiveness-disabled' }); + await page.locator('#tab-content').click(); + await expect(page.locator('.advice-effectiveness-body')).toHaveCount(0); +}); + +test('candidate extends the existing provider card instead of creating a third card', async ({ page }) => { + const claudeRoot = await openCandidate(page, 'claude'); + const claudeCards = page.locator('#content > .action-card'); + await expect(claudeCards).toHaveCount(2); + await expect(claudeCards.first().locator('[data-advice-provider="claude"]')).toHaveCount(1); + await expect(claudeCards.nth(1).locator('.advice-effectiveness-body')).toHaveCount(0); + expect(await claudeRoot.evaluate((element) => + element.parentElement?.classList.contains('action-card'))).toBe(true); + + const codexRoot = await openCandidate(page, 'codex'); + const codexCards = page.locator('#content > .action-card'); + await expect(codexCards).toHaveCount(1); + await expect(codexCards.first().locator('[data-advice-provider="codex"]')).toHaveCount(1); + expect(await codexRoot.evaluate((element) => + element.parentElement?.classList.contains('action-card'))).toBe(true); +}); + +test('aggregate and prompt consent remain separate and prompt starts closed', async ({ page }) => { + const root = await openCandidate(page, 'claude'); + const aggregate = root.locator('[data-advice-consent-kind="aggregate"]'); + const prompt = root.locator('[data-advice-consent-kind="prompt"]'); + const preview = root.locator('[data-advice-action="preview"]'); + + await expect(aggregate).not.toBeChecked(); + await expect(aggregate).toBeEnabled(); + await expect(prompt).not.toBeChecked(); + await expect(prompt).toBeDisabled(); + await expect(preview).toBeDisabled(); + expect(await page.content()).not.toContain(PROMPT_SENTINEL); + + await aggregate.focus(); + await aggregate.press('Space'); + await expectPostedCount(page, 'updateAdviceConsent', 1); + expect((await postedMessages(page, 'updateAdviceConsent')).at(-1)).toEqual({ + command: 'updateAdviceConsent', + provider: 'claude', + aggregateConsent: 'explicit', + promptSampleConsent: 'not-granted', + }); + await dispatchHostMessage(page, { + command: 'adviceConsentResult', + ok: true, + provider: 'claude', + aggregateConsent: 'explicit', + promptSampleConsent: 'not-granted', + }); + + await expect(aggregate).toBeChecked(); + await expect(prompt).toBeEnabled(); + await expect(prompt).not.toBeChecked(); + await expect(preview).toBeEnabled(); + + await prompt.focus(); + await prompt.press('Space'); + await expectPostedCount(page, 'updateAdviceConsent', 2); + expect((await postedMessages(page, 'updateAdviceConsent')).at(-1)).toEqual({ + command: 'updateAdviceConsent', + provider: 'claude', + aggregateConsent: 'explicit', + promptSampleConsent: 'explicit', + }); + expect(await page.content()).not.toContain(PROMPT_SENTINEL); + await dispatchHostMessage(page, { + command: 'adviceConsentResult', + ok: true, + provider: 'claude', + aggregateConsent: 'explicit', + promptSampleConsent: 'explicit', + }); + await expect(prompt).toBeChecked(); + + await aggregate.focus(); + await aggregate.press('Space'); + await expectPostedCount(page, 'updateAdviceConsent', 3); + expect((await postedMessages(page, 'updateAdviceConsent')).at(-1)).toEqual({ + command: 'updateAdviceConsent', + provider: 'claude', + aggregateConsent: 'not-granted', + promptSampleConsent: 'not-granted', + }); +}); + +test('sealed preview renders the exact canonical body, UTF-8 size, and SHA-256', async ({ page }) => { + const fixture = buildAdviceEffectivenessFixture({ locale: 'en' }); + const networkAfterLoad = []; + await openCandidate(page, 'claude'); + page.on('request', (request) => networkAfterLoad.push(request.url())); + await grantAggregateConsent(page); + + const previewButton = page.locator( + '[data-advice-provider="claude"] [data-advice-action="preview"]', + ); + await previewButton.focus(); + await previewButton.press('Enter'); + await expectPostedCount(page, 'prepareAdviceSnapshot', 1); + expect((await postedMessages(page, 'prepareAdviceSnapshot')).at(-1)).toEqual({ + command: 'prepareAdviceSnapshot', + provider: 'claude', + aggregateConsent: 'explicit', + promptSampleConsent: 'not-granted', + }); + await dispatchHostMessage(page, fixture.snapshotMessages.aggregateOnly); + + const preview = page.locator('[data-advice-preview="claude"]'); + const body = preview.locator('[data-advice-preview-body]'); + await expect(preview).toBeVisible(); + await expect(preview).toHaveAttribute( + 'data-snapshot-id', + fixture.snapshotMessages.aggregateOnly.snapshotId, + ); + await expect(body).toHaveText(fixture.snapshotMessages.aggregateOnly.body); + await expect(preview.locator('[data-advice-preview-bytes]')).toContainText( + String(fixture.snapshotMessages.aggregateOnly.utf8Bytes), + ); + await expect(preview.locator('[data-advice-preview-count]')).toContainText('0'); + await expect(preview.locator('[data-advice-preview-digest]')).toHaveText( + `SHA-256 ${fixture.snapshotMessages.aggregateOnly.sha256}`, + ); + expect(Buffer.byteLength(fixture.snapshotMessages.aggregateOnly.body, 'utf8')).toBe( + fixture.snapshotMessages.aggregateOnly.utf8Bytes, + ); + expect(createHash('sha256') + .update(fixture.snapshotMessages.aggregateOnly.body, 'utf8') + .digest('hex')).toBe(fixture.snapshotMessages.aggregateOnly.sha256); + expect(fixture.snapshotMessages.aggregateOnly.body).not.toContain('promptSamples'); + expect(await page.content()).not.toContain(PROMPT_SENTINEL); + + const prompt = page.locator( + '[data-advice-provider="claude"] [data-advice-consent-kind="prompt"]', + ); + await prompt.focus(); + await prompt.press('Space'); + await expectPostedCount(page, 'updateAdviceConsent', 2); + await dispatchHostMessage(page, { + command: 'adviceConsentResult', + ok: true, + provider: 'claude', + aggregateConsent: 'explicit', + promptSampleConsent: 'explicit', + }); + expect(await page.content()).not.toContain(PROMPT_SENTINEL); + + await previewButton.focus(); + await previewButton.press('Enter'); + await expectPostedCount(page, 'prepareAdviceSnapshot', 2); + expect((await postedMessages(page, 'prepareAdviceSnapshot')).at(-1)).toEqual({ + command: 'prepareAdviceSnapshot', + provider: 'claude', + aggregateConsent: 'explicit', + promptSampleConsent: 'explicit', + }); + await dispatchHostMessage(page, fixture.snapshotMessages.withPromptSamples); + await expect(body).toHaveText(fixture.snapshotMessages.withPromptSamples.body); + await expect(body).toContainText(PROMPT_SENTINEL); + await expect(preview.locator('[data-advice-preview-bytes]')).toContainText( + String(fixture.snapshotMessages.withPromptSamples.utf8Bytes), + ); + await expect(preview.locator('[data-advice-preview-count]')).toContainText('1'); + await expect(preview.locator('[data-advice-preview-digest]')).toHaveText( + `SHA-256 ${fixture.snapshotMessages.withPromptSamples.sha256}`, + ); + expect(Buffer.byteLength(fixture.snapshotMessages.withPromptSamples.body, 'utf8')).toBe( + fixture.snapshotMessages.withPromptSamples.utf8Bytes, + ); + expect(createHash('sha256') + .update(fixture.snapshotMessages.withPromptSamples.body, 'utf8') + .digest('hex')).toBe(fixture.snapshotMessages.withPromptSamples.sha256); + expect(networkAfterLoad).toEqual([]); +}); + +test('candidate exposes no send control and feedback posts identifiers plus kind only', async ({ page }) => { + const root = await openCandidate(page, 'claude'); + await expect(root.locator('form, a[href]')).toHaveCount(0); + await expect(root.getByRole('button', { name: /send|upload|submit|transmit/i })).toHaveCount(0); + expect(await root.locator('[data-advice-action]').evaluateAll((elements) => + [...new Set(elements.map((element) => element.getAttribute('data-advice-action')))].sort())) + .toEqual(['feedback', 'preview']); + + const feedbackKinds = await root.locator('[data-advice-action="feedback"]') + .evaluateAll((buttons) => buttons.map((button) => button.getAttribute('data-feedback-kind'))); + expect(feedbackKinds).toEqual(['helpful', 'not-helpful', 'applied']); + + const helpful = root.locator('[data-feedback-kind="helpful"]'); + await helpful.focus(); + await helpful.press('Space'); + await expectPostedCount(page, 'recordAdviceFeedback', 1); + const message = (await postedMessages(page, 'recordAdviceFeedback')).at(-1); + expect(Object.keys(message).sort()).toEqual([ + 'adviceId', + 'command', + 'kind', + 'provider', + 'recommendationId', + ]); + expect(message).toEqual({ + command: 'recordAdviceFeedback', + provider: 'claude', + adviceId: 'advice-claude-ui-fixture', + recommendationId: 'recommendation-claude-clear-between-tasks', + kind: 'helpful', + }); + expect(JSON.stringify(message)).not.toMatch(/prompt|body|path|session|payload/i); + + await dispatchHostMessage(page, { + command: 'adviceFeedbackResult', + ok: true, + provider: 'claude', + adviceId: message.adviceId, + recommendationId: message.recommendationId, + rating: 'helpful', + applied: 'not-applied', + }); + await expect(helpful).toHaveAttribute('aria-pressed', 'true'); + await expect(helpful).toBeEnabled(); + await expect(root.locator('[data-advice-feedback-status]')).not.toBeEmpty(); +}); + +for (const provider of ['claude', 'codex']) { + test(`${provider} candidate has labelled structure and no serious Axe violations`, async ({ page }) => { + const root = await openCandidate(page, provider, { theme: 'dark' }); + await expect(root).toHaveAttribute('aria-labelledby', `advice-effectiveness-${provider}`); + await expect(root.locator(`#advice-effectiveness-${provider}`)).toBeVisible(); + await expect(root.locator('.advice-spine')).toHaveAttribute('aria-label', /.+/); + await expect(root.locator('.advice-spine > li')).toHaveCount(5); + await expect(root.locator('.advice-step-marker')).toHaveText(['1', '2', '3', '4', '5']); + await expect(root.locator('.advice-feedback-group')).toHaveAttribute('role', 'group'); + await expect(root.locator('[data-advice-feedback-status]')).toHaveAttribute('aria-live', 'polite'); + if (provider === 'claude') { + await expect(root.locator('.advice-consent legend')).not.toBeEmpty(); + await expect(root.locator('[data-advice-consent-status]')).toHaveAttribute('aria-live', 'polite'); + await expect(root.locator('[data-advice-preview-body]')).toHaveAttribute('tabindex', '0'); + await expect(root.locator('[data-advice-preview-body]')).toHaveAttribute('aria-label', /.+/); + } + expect(await seriousOrCriticalViolations( + page, + `.advice-effectiveness-body[data-advice-provider="${provider}"]`, + )).toEqual([]); + }); +} + +for (const locale of locales) { + test(`${locale} candidate adds no narrow-screen overflow beyond the shared shell`, async ({ page }) => { + for (const provider of ['claude', 'codex']) { + const open = provider === 'claude' ? openClaude : openCodex; + await open(page, { + fixture: 'advice-effectiveness-disabled', + locale, + width: 360, + height: 800, + }); + await page.locator('#tab-content').click(); + const baseline = await page.evaluate(() => ({ + documentWidth: document.documentElement.scrollWidth, + bodyWidth: document.body.scrollWidth, + })); + + const root = await openCandidate(page, provider, { + locale, + width: 360, + height: 800, + }); + const layout = await root.evaluate((element) => ({ + documentWidth: document.documentElement.scrollWidth, + bodyWidth: document.body.scrollWidth, + rootClientWidth: element.clientWidth, + rootScrollWidth: element.scrollWidth, + })); + // The pre-existing tab/header shell is intentionally one unwrapped row + // on narrow panels. The candidate must not widen that known baseline. + expect(layout.documentWidth, { baseline, layout }) + .toBeLessThanOrEqual(baseline.documentWidth); + expect(layout.bodyWidth, { baseline, layout }) + .toBeLessThanOrEqual(baseline.bodyWidth); + expect(layout.rootScrollWidth, layout).toBeLessThanOrEqual(layout.rootClientWidth); + } + }); +} + +for (const theme of ['light', 'dark']) { + test(`${theme} candidate keeps the sealed-snapshot visual hierarchy`, async ({ page }) => { + const root = await openCandidate(page, 'claude', { theme }); + const structure = await root.evaluate((element) => { + const style = getComputedStyle(element); + const spine = element.querySelector('.advice-spine'); + const payload = element.querySelector('.advice-payload-section'); + return { + background: style.backgroundColor, + borderStyle: style.borderTopStyle, + borderWidth: Number.parseFloat(style.borderTopWidth), + spineDisplay: spine ? getComputedStyle(spine).display : null, + payloadBorderStyle: payload ? getComputedStyle(payload).borderTopStyle : null, + payloadBorderWidth: payload ? Number.parseFloat(getComputedStyle(payload).borderTopWidth) : 0, + }; + }); + expect(structure.background).not.toBe('rgba(0, 0, 0, 0)'); + expect(structure.borderStyle).not.toBe('none'); + expect(structure.borderWidth).toBeGreaterThan(0); + expect(structure.spineDisplay).toBe('grid'); + expect(structure.payloadBorderStyle).not.toBe('none'); + expect(structure.payloadBorderWidth).toBeGreaterThan(0); + }); +} diff --git a/tests/ui/support/advice-effectiveness-fixture.cjs b/tests/ui/support/advice-effectiveness-fixture.cjs new file mode 100644 index 00000000..2d2794cd --- /dev/null +++ b/tests/ui/support/advice-effectiveness-fixture.cjs @@ -0,0 +1,179 @@ +'use strict'; + +const { + adaptClaudeAdvice, + adaptCodexLocalAdvice, +} = require('../../../out/adviceEffectiveness/adapters.js'); +const { + prepareAdviceSnapshot, +} = require('../../../out/adviceEffectiveness/integration.js'); +const { + buildAdviceAggregateSnapshot, +} = require('../../../out/adviceEffectiveness/payload.js'); + +const PROMPT_SENTINEL = 'SAFE_PROMPT_SENTINEL_測試_🚦'; +const GENERATED_AT = '2026-07-20T12:00:00.000Z'; + +function numericUsage() { + return { + totalInputTokens: 47_600, + totalOutputTokens: 8_300, + totalCacheCreationTokens: 17_900, + totalCacheReadTokens: 288_000, + totalCost: 3.05, + costBreakdown: { + input: 0.44, + output: 1.52, + cacheWrite: 0.82, + cacheRead: 0.27, + }, + messageCount: 25, + modelBreakdown: { + 'claude-sonnet-4-5-20250929': { + inputTokens: 38_200, + outputTokens: 6_100, + cacheCreationTokens: 14_800, + cacheReadTokens: 236_000, + cost: 2.74, + count: 18, + }, + 'claude-haiku-4-5-20251001': { + inputTokens: 9_400, + outputTokens: 2_200, + cacheCreationTokens: 3_100, + cacheReadTokens: 52_000, + cost: 0.31, + count: 7, + }, + }, + }; +} + +function requireAdapter(result, provider) { + if (!result.ok) { + throw new Error(`${provider} advice fixture failed: ${result.issues.join(', ')}`); + } + return result.value; +} + +function requireSnapshot(result, mode) { + if (!result.ok) { + throw new Error(`${mode} advice snapshot failed: ${result.reason}`); + } + return result.value; +} + +function snapshotMessage(snapshot, suffix) { + return { + command: 'adviceSnapshotResult', + ok: true, + snapshotId: `snapshot-${snapshot.preview.sha256.slice(suffix, suffix + 24)}`, + provider: snapshot.provider, + contentType: snapshot.preview.contentType, + dataMode: snapshot.preview.dataMode, + promptSampleCount: snapshot.preview.promptSampleCount, + utf8Bytes: snapshot.preview.utf8Bytes, + sha256: snapshot.preview.sha256, + body: snapshot.preview.body, + }; +} + +/** + * Test-only vertical fixture built through the production adapters and sealed + * snapshot helper. Inputs are numeric aggregates except for the single prompt + * sentinel, which remains host-side until the explicit prompt-opt-in test. + */ +function buildAdviceEffectivenessFixture({ locale = 'en' } = {}) { + const aggregate = buildAdviceAggregateSnapshot(numericUsage(), 'overall', 30); + const claude = requireAdapter(adaptClaudeAdvice({ + adviceId: 'advice-claude-ui-fixture', + generatedAt: GENERATED_AT, + locale, + aggregate, + sessionSummary: { + scope: 'overall', + windowDays: 30, + totalSessions: 8, + longSessionCount: 3, + largeContextSessionCount: 2, + }, + frameworkOverhead: { + frameworkEstimatedTokens: 4_760, + observedInputEstimatedTokens: 47_600, + classifiedEvents: 25, + }, + }), 'Claude'); + + const codex = requireAdapter(adaptCodexLocalAdvice({ + adviceId: 'advice-codex-ui-fixture', + generatedAt: GENERATED_AT, + locale, + scope: '30d', + insights: [ + { + kind: 'effort-comparison', + severity: 'normal', + scope: '30d', + evidence: { highEffortFresh: 700, lowMediumEffortFresh: 300 }, + proxy: true, + }, + { + kind: 'cache-context', + severity: 'info', + scope: '30d', + evidence: { processedToFreshRatio: 3, cachedInputShare: 0.7 }, + proxy: true, + }, + ], + behavior: { + childFreshShare: 0.2, + approvalReviewerFreshShare: 0.1, + highEffortFreshShare: 0.7, + processedToFreshRatio: 3, + cacheShare: 0.7, + postPatchToolCallsPerPatchCall: 4, + }, + quality: { + indexComplete: true, + identityComplete: true, + periodComplete: true, + qualityFlags: [], + }, + }), 'Codex'); + + const claudeState = { + provider: 'claude', + contract: claude.contract, + remotePreviewEligible: claude.remoteEvidenceEligible, + aggregate: claude.aggregate, + promptSamples: [{ text: PROMPT_SENTINEL }], + }; + const codexState = { + provider: 'codex', + contract: codex.contract, + remotePreviewEligible: codex.remoteEvidenceEligible, + promptSamples: [], + }; + const aggregateOnly = requireSnapshot(prepareAdviceSnapshot(claudeState, { + aggregate: 'explicit', + promptSamples: 'not-granted', + }), 'aggregates-only'); + const withPromptSamples = requireSnapshot(prepareAdviceSnapshot(claudeState, { + aggregate: 'explicit', + promptSamples: 'explicit', + }), 'aggregates-with-prompt-samples'); + + return { + promptSentinel: PROMPT_SENTINEL, + states: { claude: claudeState, codex: codexState }, + snapshotMessages: { + aggregateOnly: snapshotMessage(aggregateOnly, 0), + withPromptSamples: snapshotMessage(withPromptSamples, 8), + }, + }; +} + +module.exports = { + PROMPT_SENTINEL, + buildAdviceEffectivenessFixture, +}; diff --git a/tests/ui/support/render-harness.cjs b/tests/ui/support/render-harness.cjs index d8e83a2f..cdd59c0e 100644 --- a/tests/ui/support/render-harness.cjs +++ b/tests/ui/support/render-harness.cjs @@ -41,6 +41,9 @@ const { rootedTaskBeyondRecentRowCapFixture, rootlessCrossProjectCycleFixture, } = require('../../../out/test/codexFixtures.js'); +const { + buildAdviceEffectivenessFixture, +} = require('./advice-effectiveness-fixture.cjs'); Module._load = originalLoad; @@ -53,7 +56,6 @@ const THEMES = { '--vscode-font-size:13px;' + '--vscode-editor-font-family:Menlo,Monaco,"Courier New",monospace;' + '--vscode-editor-background:#ffffff;' + - '--vscode-editor-foreground:#000000;' + '--vscode-foreground:#616161;' + '--vscode-descriptionForeground:#717171;' + '--vscode-errorForeground:#A1260D;' + @@ -99,7 +101,6 @@ const THEMES = { '--vscode-font-size:13px;' + '--vscode-editor-font-family:Menlo,Monaco,"Courier New",monospace;' + '--vscode-editor-background:#1e1e1e;' + - '--vscode-editor-foreground:#D4D4D4;' + '--vscode-foreground:#CCCCCC;' + '--vscode-descriptionForeground:rgba(204,204,204,0.7);' + '--vscode-errorForeground:#F48771;' + @@ -142,21 +143,36 @@ const THEMES = { '}*,*::before,*::after{animation:none!important;transition:none!important}', }; -function settingsStore(autoRefresh = false, weeklyValue = true) { +function settingsStore({ + autoRefresh = false, + weeklyValue = true, + adviceEffectiveness = false, +} = {}) { const values = new Map(SETTINGS.map((definition) => [definition.key, definition.default])); values.set('codex.optimization.enabled', true); values.set('dashboardAutoRefresh', autoRefresh); values.set('showWeeklyEquivalentValue', weeklyValue); + values.set('advice.effectiveness.enabled', adviceEffectiveness); return { get: (key) => values.get(key), snapshot: () => SETTINGS.map((definition) => ({ ...definition, value: values.get(definition.key), - isDefault: true, + isDefault: values.get(definition.key) === definition.default, })), }; } +function memoryGlobalState() { + const values = new Map(); + return { + get: (key) => values.get(key), + update: async (key, value) => { + values.set(key, structuredClone(value)); + }, + }; +} + function claudeUsage(multiplier = 1) { const modelBreakdown = { 'claude-sonnet-4-5-20250929': { @@ -193,7 +209,7 @@ function claudeUsage(multiplier = 1) { }; } -function addClaudeData(provider, fixture = 'default') { +function addClaudeData(provider, { fixture = 'default', enableContent = false } = {}) { const today = claudeUsage(); const now = new Date(CODEX_WEBVIEW_NOW); const completedWeeklyFixture = fixture === 'weekly-claude-completed'; @@ -232,6 +248,19 @@ function addClaudeData(provider, fixture = 'default') { undefined, undefined, weeklyRecords, + [], + [], + enableContent + ? { + categories: [], + toolResultBreakdown: [], + totalEstimatedTokens: 0, + recentPrompts: [], + thinkingBySession: {}, + thinkingByDay: {}, + skillUses: [], + } + : null, ); if (completedWeeklyFixture) { provider.updateWeeklyQuotaHistory([{ @@ -293,7 +322,14 @@ function withoutInputSnapshot(snapshot) { }; } -exports.renderHarness = function renderHarness({ provider: selectedProvider = 'codex', locale = 'en', theme = 'light', fixture = 'default', autoRefresh = false, weeklyValue = true } = {}) { +exports.renderHarness = async function renderHarness({ + provider: selectedProvider = 'codex', + locale = 'en', + theme = 'light', + fixture = 'default', + autoRefresh = false, + weeklyValue = true, +} = {}) { I18n.setLanguage(locale); I18n.setTimezone('Asia/Hong_Kong'); vscodeHost.window.activeColorTheme.kind = theme === 'dark' ? 2 : 1; @@ -322,10 +358,25 @@ exports.renderHarness = function renderHarness({ provider: selectedProvider = 'c ? withoutInputSnapshot(baseSnapshot) : baseSnapshot; const view = buildCodexUsageView(snapshot, CODEX_WEBVIEW_NOW); - const provider = new UsageWebviewProvider({}); + const provider = new UsageWebviewProvider({ globalState: memoryGlobalState() }); + // The real extension loads globalState asynchronously before enabling any + // consent or feedback control. Let that same path settle in the harness. + await new Promise((resolve) => setImmediate(resolve)); const persistedDetailsFixture = fixture === 'persisted-details'; - provider.settings = settingsStore(autoRefresh, weeklyValue); - addClaudeData(provider, fixture); + const adviceEffectivenessFixture = fixture === 'advice-effectiveness'; + const adviceContentFixture = adviceEffectivenessFixture + || fixture === 'advice-effectiveness-disabled'; + provider.settings = settingsStore({ + autoRefresh, + weeklyValue, + adviceEffectiveness: adviceEffectivenessFixture, + }); + addClaudeData(provider, { fixture, enableContent: adviceContentFixture }); + if (adviceEffectivenessFixture) { + provider.updateAdviceEffectivenessData( + buildAdviceEffectivenessFixture({ locale }).states, + ); + } provider.updateProviderData( view, buildScopedCodexInsights(view), diff --git a/tests/ui/support/server.mjs b/tests/ui/support/server.mjs index 74eca816..1afa7675 100644 --- a/tests/ui/support/server.mjs +++ b/tests/ui/support/server.mjs @@ -6,7 +6,7 @@ const { renderHarness } = require('./render-harness.cjs'); const locales = new Set(['en', 'de-DE', 'zh-TW', 'zh-CN', 'ja', 'ko', 'pt-BR', 'id']); const uiTestPort = Number(process.env.CCU_UI_TEST_PORT ?? 4173); -const server = createServer((request, response) => { +const server = createServer(async (request, response) => { try { const url = new URL(request.url ?? '/', `http://127.0.0.1:${uiTestPort}`); if (url.pathname === '/health') { @@ -28,12 +28,30 @@ const server = createServer((request, response) => { : 'codex'; const theme = url.searchParams.get('theme') === 'dark' ? 'dark' : 'light'; const requestedFixture = url.searchParams.get('fixture') ?? 'default'; - const fixture = ['default', 'rootless-cycle', 'root-over-limit', 'persisted-details', 'weekly-usage-only', 'weekly-claude-completed', 'unknown-models', 'zero-input'].includes(requestedFixture) + const fixture = [ + 'default', + 'rootless-cycle', + 'root-over-limit', + 'persisted-details', + 'weekly-usage-only', + 'weekly-claude-completed', + 'unknown-models', + 'zero-input', + 'advice-effectiveness', + 'advice-effectiveness-disabled', + ].includes(requestedFixture) ? requestedFixture : 'default'; const autoRefresh = url.searchParams.get('autoRefresh') === 'true'; const weeklyValue = url.searchParams.get('weeklyValue') !== 'false'; - const html = renderHarness({ provider, locale, theme, fixture, autoRefresh, weeklyValue }); + const html = await renderHarness({ + provider, + locale, + theme, + fixture, + autoRefresh, + weeklyValue, + }); response.writeHead(200, { 'content-type': 'text/html; charset=utf-8', 'cache-control': 'no-store', From 19c52351f2fe42fde192b0303b57eab440d14121 Mon Sep 17 00:00:00 2001 From: Carl <74130028+Carl723000@users.noreply.github.com> Date: Thu, 27 Aug 2026 23:43:07 +0800 Subject: [PATCH 04/72] feat: prepare v2.3.1 evidence migration seams Co-authored-by: OpenAI Codex <215057067+openai-codex[bot]@users.noreply.github.com> --- AI-ADVICE-INTEGRATION-V2.3.1.md | 61 ++++ AI-ADVICE-INTEGRATION-V2.3.1.zh-CN.md | 49 ++++ .../evidencePreparation.ts | 166 +++++++++++ .../legacyPersonalization.ts | 276 ++++++++++++++++++ .../versionedPersistence.ts | 1 + src/test/adviceEvidencePreparation.test.ts | 144 +++++++++ src/test/adviceLegacyPersonalization.test.ts | 134 +++++++++ src/test/advicePersistence.test.ts | 23 ++ src/test/adviceRuntimePolicy.test.ts | 17 ++ 9 files changed, 871 insertions(+) create mode 100644 src/adviceEffectiveness/evidencePreparation.ts create mode 100644 src/adviceEffectiveness/legacyPersonalization.ts create mode 100644 src/test/adviceEvidencePreparation.test.ts create mode 100644 src/test/adviceLegacyPersonalization.test.ts diff --git a/AI-ADVICE-INTEGRATION-V2.3.1.md b/AI-ADVICE-INTEGRATION-V2.3.1.md index f5b72caf..667e7fcf 100644 --- a/AI-ADVICE-INTEGRATION-V2.3.1.md +++ b/AI-ADVICE-INTEGRATION-V2.3.1.md @@ -20,6 +20,24 @@ The current safety boundary is: The existing `Get AI Advice` and Prompt Optimizer remain separate legacy features. Their compatibility boundaries are documented in section 9; they must not be represented as satisfying the new path's sealed-payload or strict-parser guarantees. +### 2026-08-27 main-roadmap alignment + +This task has read and aligned with `feature-coverage-matrix.md`, +`parallel-development-handoff.md`, and `github-connected-roadmap.md` under the main +repository's `decision-reports/2026-08-24-release-roadmap/`. The actual commit chain is +R8 `48e5e77` → foundation cherry-pick `91739a8` → integration `20466a7`. Commit +`c7a95d9` on `codex/v2.3.1-ai-advice-validity-foundation` has the same patch as +`91739a8` but is based on old commit `0b4b750`; it must not be branch-merged or +cherry-picked again. + +The final Claude #87 incremental index is not on this branch. A clean candidate, +`2584173`, appeared on 2026-08-27, but it diverges substantially from R8 and also +changes `dataLoader.ts`, `extension.ts`, Codex backfill, and architecture documents. +It is not a small patch to import here. A final candidate must first inherit the +released v2.2.3 #87/#89/#92 work and stable v2.3.0, then selectively replay this +branch's contract, adapter, persistence, host-wiring, and UI layers. This prototype +is not the final integration baseline. + ## 2. Implementation map and ownership | Responsibility | Current integration point | Constraint | @@ -33,6 +51,8 @@ The existing `Get AI Advice` and Prompt Optimizer remain separate legacy feature | Reversible model experiment | `src/adviceEffectiveness/modelExperiment.ts` | One adjacent downward tier only, at least five pairs | | Strict model output | `src/adviceEffectiveness/structuredOutput.ts` | JSON-only, exact shape, known references, no repair/fallback | | Sole future BYOK seam | `src/adviceEffectiveness/legacyBridge.ts` | `backend: api` only; production code does not call it | +| #87 evidence preparation plan | `src/adviceEffectiveness/evidencePreparation.ts` | Pure and I/O-free; disabled and aggregates-only modes never request content scans, and a missing personalization snapshot only yields an explicit-refresh request | +| Legacy personalization migration preparation | `src/adviceEffectiveness/legacyPersonalization.ts` | userContext/prompts remain in a host-memory draft; no projection exists without the second consent; production does not call it | | Framework-origin classification | `src/promptOrigin.ts`, `src/dataLoader.ts` | Structural markers are aggregated immediately; no semantic or writing-quality judgment | | Thin host / UI wiring | `src/extension.ts`, `src/webview.ts` | Feature flag, host-only prompts, opaque snapshot, local feedback | @@ -62,6 +82,24 @@ The adapter type does not accept raw records, paths, session IDs, titles, or pro Claude's structural evidence is eligible for a sealed remote preview, but only after separate explicit aggregate consent. +### #87 incremental-index dependency + +When this R8 prototype is enabled, it still filters `cache.records` and recomputes +usage aggregates and session breakdowns. The default-off state adds no current-release +cost, but this path cannot ship in v2.3.1. The final integration must consume a +same-window aggregate, session summary, coverage, and quality state materialized by +the stable #87 `ClaudeUsageIndex`. Prompt samples and framework overhead must come +from the same per-file analysis contribution; advice must never trigger another JSONL +scan. + +`planAdviceEvidencePreparation` now records that boundary as an I/O-free typed seam. +Local rendering and aggregates-only mode consume existing data only. Prompt +personalization checks a content snapshot only after both consents are explicit and +the user asks to preview or send. A missing, stale, or mismatched snapshot yields a +`refresh-content-analysis` host-action request while `allowAutomaticScan` remains +fixed to `false`; the function never reads files or starts a refresh. It remains +unwired until the final stable index exists. + ### Codex `adaptCodexLocalAdvice` maps only existing R8 structural numeric signals such as fresh shares, cache share, processed/fresh multiples, and post-patch tool intensity. It uses only an insight's allowlisted `kind`, `scope`, and `proxy` fields plus a separate numeric behavior DTO; open-ended evidence objects and strings such as `observedEffort` do not enter the unified contract. @@ -112,6 +150,7 @@ Migration and degradation policy: - A valid v1 envelope migrates only validated feedback; feature mode, aggregate consent, and prompt consent all close, and comparable pairs start empty. - An unknown future version, extra field, duplicate ID, invalid enum, non-finite number, or storage error returns the all-closed state and does not overwrite the unknown/corrupt source. - After a write failure, the host enters degraded mode and rejects further consent or feedback mutations. +- A v2 state with explicit prompt consent but no aggregate consent is treated as corrupt, fails closed as a whole, and is never overwritten. The foundation's separate `feedback.ts` / `claudeCodeUsage.adviceEffectiveness.feedback.v1` event ledger remains for compatibility tests, but it is not the integrated UI's write target and is not silently merged into the v2 envelope. Any future migration needs a separately reviewed one-time import; code must not dual-write in the meantime. @@ -175,6 +214,17 @@ Comparison requires a local `applied` mark and a helpful/not-helpful rating, fol The legacy `getAdvice` / `adviceSummary` prompt behavior is a historical compatibility boundary and must not be described as the new path's privacy guarantee. The new aggregate and prompt-sample consents are not interchangeable with old Optimizer consent or any legacy configuration boolean. +`legacyPersonalization.ts` prepares only the A-20 migration seam and does not change +legacy runtime behavior. It rebuilds `advice.userContext` and candidate prompt +samples into a bounded, host-only draft, drops cwd, session IDs, and unknown fields, +and classifies sample age as within-window, older-than-window, or unknown. No content +means no migration; any content requires the new prompt-personalization consent. +`projectLegacyPersonalization` returns `undefined` without that consent and emits an +allowlisted, verbatim-previewable projection only after explicit consent. It also +revalidates the complete in-memory draft before projection, rejecting extra fields +and inconsistent metadata. The projection is not yet part of the canonical payload, +so A-20 remains partially prepared rather than wired. + ## 10. Strict parsing and the OAuth 403 boundary `parseStructuredAdviceOutput` accepts exactly one bounded, exact-shape JSON object. It rejects Markdown fences, surrounding prose, unknown fields, unknown observation/evidence references, duplicate IDs, oversized arrays/text, missing conditional actions, missing success criteria or quality guardrails, and unsupported versions. It does not strip fences, repair JSON, return a partial batch, or call legacy Markdown/Optimizer fallbacks. An empty `recommendations` array is a valid fail-closed result. @@ -223,14 +273,25 @@ Before anyone changes a default, adds a send button, or calls the feature shippe - Complete an F5 Extension Development Host smoke test, feasible Playwright/screenshot review, `npm run compile`, the full `npm test`, VSIX packaging, and the macOS/Linux installed-VSIX smoke required for a release candidate. - If this later becomes a user-visible release, update CHANGELOG and all seven README files as a separately authorized release task; that work is not implicitly authorized by this candidate branch. +The operation → full reload → persistence/invalidation gate is not yet complete. +Existing tests cover post-host-reply page state and independent `globalState` +read/write behavior, but not a real full-page replacement after consent or feedback, +nor host retrieval failure for an old opaque snapshot after consent withdrawal. The +final candidate must add those tests and must not report this gate as passed today. + ## 12. Current limitations and next step - The new panel has no production sender; a sealed preview does not mean data was sent. - Automatic comparable-task generation/matching and real quality-rubric collection are not implemented; only strict storage and pure comparison interfaces exist. +- Current Claude host wiring still recomputes from `cache.records`; final v2.3.1 must use the stable #87 materialized aggregate/coverage seam and prove advice causes no extra JSONL body reads. +- The remote parser strictly validates shape and references, but host-owned recommendation lineage/type and success policy are not yet locked. A model must not invent cross-day IDs or wrap unrelated evidence in a `/clear` recommendation. +- The Codex adapter may produce multiple recommendations, while the experimental UI currently closes feedback/comparison around the first only. The final candidate needs per-recommendation closure or one auditable host-owned selection rule. +- Full-reload persistence for consent/feedback and old-snapshot invalidation after consent withdrawal still lack end-to-end tests. - Topic drift and writing quality have no local semantic evidence. - Codex has no remote payload schema and remains local-only. - Legacy `Get AI Advice` and Prompt Optimizer have not migrated to the unified contract and retain their historical parsing behavior. - Framework origin is a structural proxy; log schema drift may reduce coverage and must not be filled with guesses. +- `advice.userContext` and sample age have only an unwired host-only migration seam; aggregates-only payloads still contain neither. - The separate feedback v1 ledger is not automatically imported into the v2 envelope. The minimum future wiring sequence is: first complete the gates above and a network privacy review; then let the host resolve the same Prepared object from its opaque snapshot ID and call only `requestStructuredAdviceViaLegacyByok`; after strict parsing, let the host assemble the complete `AdviceContract`. Do not wire through webview body text, `getAdvice`, `buildAdviceSummary`, `getUsageAdvice`, or Prompt Optimizer. diff --git a/AI-ADVICE-INTEGRATION-V2.3.1.zh-CN.md b/AI-ADVICE-INTEGRATION-V2.3.1.zh-CN.md index 0d2c6671..224fa441 100644 --- a/AI-ADVICE-INTEGRATION-V2.3.1.zh-CN.md +++ b/AI-ADVICE-INTEGRATION-V2.3.1.zh-CN.md @@ -20,6 +20,21 @@ 已有的 `Get AI Advice` 与 Prompt Optimizer 仍是独立的旧功能;它们的兼容边界见第 9 节,不能被视为满足新链路的密封 payload 或严格解析保证。 +### 2026-08-27 主仓库路线图对齐 + +本任务已逐项读取并对齐主仓库 `decision-reports/2026-08-24-release-roadmap/` 下的 +`feature-coverage-matrix.md`、`parallel-development-handoff.md` 与 +`github-connected-roadmap.md`。本分支的真实提交链是 R8 `48e5e77` → foundation +cherry-pick `91739a8` → integration `20466a7`。旧 foundation 分支 +`codex/v2.3.1-ai-advice-validity-foundation` 的 `c7a95d9` 与 `91739a8` 是同一 +patch,但基于旧 `0b4b750`;未来不得整支合并或重复 cherry-pick。 + +#87 的最终 Claude 增量索引不在本分支。2026-08-27 出现了 clean 候选 +`2584173`,但它与 R8/本分支大幅分叉,并同时修改 `dataLoader.ts`、 +`extension.ts`、Codex backfill 与架构文档,不能当成一个小补丁移入。最终候选必须先等 +v2.2.3 的 #87/#89/#92 正式落地,再从稳定 v2.3.0 选择性重放本分支的纯契约、适配、 +持久化、host wiring 和 UI 层;当前原型不是最终集成基线。 + ## 2. 实现地图与所有权 | 责任 | 当前接入点 | 约束 | @@ -33,6 +48,8 @@ | model 可逆实验 | `src/adviceEffectiveness/modelExperiment.ts` | 只允许相邻向下一档,至少五对任务 | | 严格模型输出 | `src/adviceEffectiveness/structuredOutput.ts` | JSON-only、exact shape、已知引用、无修复/fallback | | 唯一未来 BYOK 接线 | `src/adviceEffectiveness/legacyBridge.ts` | 仅 `backend: api`;当前生产代码未调用 | +| #87 证据准备计划 | `src/adviceEffectiveness/evidencePreparation.ts` | 纯函数、无 I/O;关闭态和 aggregates-only 永不请求内容扫描,个性化缺快照时只返回“需显式刷新” | +| 旧个性化迁移准备 | `src/adviceEffectiveness/legacyPersonalization.ts` | userContext/prompt 只保留于 host 内存草稿;无第二项 consent 时不产生投影,当前生产未调用 | | framework 来源分类 | `src/promptOrigin.ts`、`src/dataLoader.ts` | 只看结构标记并立即聚合,不做语义/写作质量判断 | | host / UI 薄接线 | `src/extension.ts`、`src/webview.ts` | feature flag、host-only prompt、opaque snapshot、本地反馈 | @@ -62,6 +79,20 @@ host 始终拥有观察、证据、隐私和来源。未来远程模型最多只 Claude 的结构化证据可用于密封远程预览,但仍需聚合数据的独立明确同意。 +### #87 增量索引依赖 + +当前 R8 原型在实验开关打开时,仍会从 `cache.records` 过滤窗口并重新计算 usage aggregate +和 session breakdown。默认关闭时不会增加当前发布开销,但这条路径不能进入正式 v2.3.1。 +未来必须消费稳定 #87 `ClaudeUsageIndex` 已物化的同窗口 aggregate、session summary、 +coverage 与 quality 状态;prompt 样本和 framework-overhead 也必须来自同一 per-file analysis +contribution,不能为了 advice 再扫描 JSONL。 + +`planAdviceEvidencePreparation` 已把这条边界固化为无 I/O 的 typed seam:本地展示和 +aggregates-only 只消费现成数据;prompt 个性化只有在两项 consent 均明确、且用户主动预览或 +发送时才检查内容快照。缺失、过期或窗口不一致只返回 +`refresh-content-analysis` 的 host action 请求,同时固定 `allowAutomaticScan: false`;函数本身 +永远不读文件或触发刷新。它在最终 stable index 接入前保持未接线。 + ### Codex `adaptCodexLocalAdvice` 只映射 R8 已有的结构数值信号,例如 fresh share、cache share、processed/fresh multiple 与 post-patch tool intensity。它只使用 insight 的 allowlisted `kind`、`scope`、`proxy` 和单独的数值 behavior DTO;开放式 evidence 对象以及 `observedEffort` 等字符串不会进入统一契约。 @@ -112,6 +143,7 @@ SHA-256 是快照身份/一致性摘要,不是签名,也不代表服务端 - 合法 v1 envelope 只迁移已验证反馈;feature、aggregate consent、prompt consent 全部重置为关闭,可比任务对从空数组开始。 - 未知未来版本、额外字段、重复 ID、非法 enum、非有限数或存储错误都返回全关闭状态,并且不覆盖原始未知/损坏数据。 - 写入失败后 host 进入 degraded 状态,拒绝继续修改 consent 或反馈。 +- v2 状态若出现 `promptSampleConsent=explicit` 但 aggregate consent 未授予,会作为损坏状态整体失败关闭,且不会覆盖原值。 基础提交中的独立 `feedback.ts` / `claudeCodeUsage.adviceEffectiveness.feedback.v1` event ledger 仍保留供兼容测试,但不是集成 UI 的写入点,也不会被静默合并进 v2 envelope。未来若决定迁移,必须单独设计可审计的一次性导入;在此之前不得双写。 @@ -175,6 +207,13 @@ model rightsizing 只是一条 typed、可逆实验 seam:允许 `opus → sonn 旧 `getAdvice` / `adviceSummary` 的 prompt 行为是历史兼容边界,不应被描述成新链路的隐私保证。新链路中的聚合同意和 prompt 样本同意不能与旧 Optimizer consent 或任何旧配置布尔值互相替代。 +`legacyPersonalization.ts` 只准备 A-20 的迁移接口,不改变旧 runtime:它把 +`advice.userContext` 与候选 prompt 样本重建为有界的 host-only 草稿,丢弃 cwd、session ID +和未知字段,并把样本年龄标为窗口内、早于窗口或未知。内容为空时无需迁移;有内容时必须使用 +新的 prompt-personalization consent。未授权时 `projectLegacyPersonalization` 返回 +`undefined`;授权后才产生可逐字预览的 allowlisted 投影。投影前还会重新验证整个内存草稿, +拒绝额外字段和不一致元数据。该投影尚未进入 canonical payload,因此 A-20 仍是部分完成而非已接线。 + ## 10. 严格解析与 OAuth 403 边界 `parseStructuredAdviceOutput` 只接受一个 bounded、exact-shape JSON object。它拒绝 Markdown fence、前后 prose、未知字段、未知 observation/evidence reference、重复 ID、过大数组/字符串、无条件动作、无成功判据、无 quality guardrail 以及不合版本的输出。它不剥 fence、不修 JSON、不返回部分 batch,也不调用旧 Markdown/Optimizer fallback;空 `recommendations` 是合法的 fail-closed 结果。 @@ -223,14 +262,24 @@ Claude Code OAuth 凭据仍只用于既有 usage/quota API。历史上的 OAuth - 完成 F5 Extension Development Host smoke、可行的 Playwright/截图检查、`npm run compile`、全量 `npm test`、VSIX 打包,以及发布候选所需的 macOS/Linux installed-VSIX smoke。 - 若最终成为用户可见发布内容,再按项目政策单独更新 CHANGELOG 和全部七份 README;这些不是本候选准备分支的隐式授权。 +其中“操作 → full reload → 状态保留/快照失效”的端到端验收当前尚未完成;现有测试只覆盖 +host 回执后的页面状态和独立的 `globalState` 读写。正式候选必须补真实 consent/feedback +操作后的 full-page replacement 测试,以及 consent 撤回后旧 opaque snapshot 不可再取回的 +host 测试,不能把当前证据写成已通过。 + ## 12. 当前已知限制与下一步 - 新面板没有生产 sender;密封预览不代表数据已经发送。 - 自动生成/匹配可比任务对与真实质量 rubric 采集仍未实现;当前只有严格存储和纯比较接口。 +- 当前 Claude host wiring 仍从 `cache.records` 重算;正式 v2.3.1 必须改为稳定 #87 的已物化 aggregate/coverage seam,并证明 advice 不增加 JSONL body read。 +- 远程 parser 已严格校验结构和引用,但 host-owned recommendation lineage/建议类型与成功政策尚未锁定;模型不能自行发明跨日 ID 或把无关 evidence 包装成 `/clear` 等建议。 +- Codex adapter 可以产生多条 recommendation,但实验 UI 当前只闭环第一条;正式候选必须逐条反馈/比较,或由 host 给出可审计的唯一选择规则。 +- consent/feedback 的 full-reload persistence 与 consent 撤回后的 snapshot 失效仍缺端到端测试。 - topic drift 和写作质量没有本地语义证据。 - Codex 没有远程 payload schema,保持 local-only。 - 旧 `Get AI Advice` 和 Prompt Optimizer 尚未迁移到统一 contract,且保留各自历史解析行为。 - framework origin 是结构 proxy,日志 schema 漂移可能降低覆盖;不得用猜测补齐。 +- `advice.userContext`/样本年龄只有未接线的 host-only 迁移 seam;aggregates-only payload 仍不会包含这些内容。 - 独立 feedback v1 ledger 没有自动导入 v2 envelope。 未来最小接线顺序应是:先完成上述验收与网络隐私评审,再让 host 通过 opaque snapshot ID 取回同一个 Prepared 对象,唯一调用 `requestStructuredAdviceViaLegacyByok`,严格解析后由 host 重新组装完整 `AdviceContract`;不要从 webview body、`getAdvice`、`buildAdviceSummary`、`getUsageAdvice` 或 Prompt Optimizer 绕接。 diff --git a/src/adviceEffectiveness/evidencePreparation.ts b/src/adviceEffectiveness/evidencePreparation.ts new file mode 100644 index 00000000..a615a4fc --- /dev/null +++ b/src/adviceEffectiveness/evidencePreparation.ts @@ -0,0 +1,166 @@ +/** + * Pure planning contract between v2.3.1 advice and the future stable #87 + * materialized Claude index. It deliberately performs no I/O and never starts + * a content scan. The host may interpret `refresh-content-analysis` only after + * the user explicitly requests prompt personalization. + */ + +export const ADVICE_EVIDENCE_PREPARATION_PLAN_VERSION = 1 as const; + +export type AdviceEvidencePreparationOperation = 'local-render' | 'preview' | 'send'; +export type AdviceEvidencePreparationDataMode = + | 'local-only' + | 'aggregates-only' + | 'aggregates-with-prompt-samples'; + +export interface AdviceContentSnapshotDescriptor { + status: 'ready' | 'stale'; + windowDays: number; + promptSampleCount: number; +} + +export interface AdviceEvidencePreparationRequest { + featureMode: 'disabled' | 'enabled'; + operation: AdviceEvidencePreparationOperation; + dataMode: AdviceEvidencePreparationDataMode; + aggregateAvailable: boolean; + aggregateConsent: 'not-granted' | 'explicit'; + promptPersonalizationConsent: 'not-granted' | 'explicit'; + requestedWindowDays: number; + contentSnapshot?: AdviceContentSnapshotDescriptor; +} + +export type AdviceEvidencePreparationReason = + | 'feature-disabled' + | 'invalid-request' + | 'local-evidence-ready' + | 'aggregate-consent-required' + | 'aggregate-unavailable' + | 'aggregate-evidence-ready' + | 'prompt-personalization-consent-required' + | 'content-snapshot-missing' + | 'content-snapshot-stale' + | 'content-window-mismatch' + | 'prompt-samples-unavailable' + | 'personalized-evidence-ready'; + +export interface AdviceEvidencePreparationPlan { + schemaVersion: typeof ADVICE_EVIDENCE_PREPARATION_PLAN_VERSION; + status: 'disabled' | 'ready' | 'blocked' | 'explicit-refresh-required'; + dataMode: AdviceEvidencePreparationDataMode; + reason: AdviceEvidencePreparationReason; + requiredHostAction: 'none' | 'refresh-content-analysis'; + /** This pure seam never authorizes a background or implicit JSONL scan. */ + allowAutomaticScan: false; +} + +function plan( + status: AdviceEvidencePreparationPlan['status'], + dataMode: AdviceEvidencePreparationDataMode, + reason: AdviceEvidencePreparationReason, + requiredHostAction: AdviceEvidencePreparationPlan['requiredHostAction'] = 'none', +): AdviceEvidencePreparationPlan { + return { + schemaVersion: ADVICE_EVIDENCE_PREPARATION_PLAN_VERSION, + status, + dataMode, + reason, + requiredHostAction, + allowAutomaticScan: false, + }; +} + +function validSnapshot(value: unknown): value is AdviceContentSnapshotDescriptor { + if (typeof value !== 'object' || value === null || Array.isArray(value)) return false; + const snapshot = value as Record; + const keys = Object.keys(snapshot).sort(); + if (keys.join('\0') !== ['promptSampleCount', 'status', 'windowDays'].join('\0')) return false; + return ( + (snapshot.status === 'ready' || snapshot.status === 'stale') && + typeof snapshot.windowDays === 'number' && + Number.isInteger(snapshot.windowDays) && + snapshot.windowDays >= 1 && + snapshot.windowDays <= 365 && + typeof snapshot.promptSampleCount === 'number' && + Number.isInteger(snapshot.promptSampleCount) && + snapshot.promptSampleCount >= 0 + ); +} + +function validRequest(value: AdviceEvidencePreparationRequest): boolean { + return ( + (value.operation === 'local-render' || value.operation === 'preview' || value.operation === 'send') && + (value.dataMode === 'local-only' || + value.dataMode === 'aggregates-only' || + value.dataMode === 'aggregates-with-prompt-samples') && + typeof value.aggregateAvailable === 'boolean' && + (value.aggregateConsent === 'not-granted' || value.aggregateConsent === 'explicit') && + (value.promptPersonalizationConsent === 'not-granted' || + value.promptPersonalizationConsent === 'explicit') && + Number.isInteger(value.requestedWindowDays) && + value.requestedWindowDays >= 1 && + value.requestedWindowDays <= 365 && + (value.contentSnapshot === undefined || validSnapshot(value.contentSnapshot)) + ); +} + +export function planAdviceEvidencePreparation( + request: AdviceEvidencePreparationRequest, +): AdviceEvidencePreparationPlan { + if (request?.featureMode === 'disabled') { + return plan('disabled', 'local-only', 'feature-disabled'); + } + if (request?.featureMode !== 'enabled' || !validRequest(request)) { + return plan('blocked', 'local-only', 'invalid-request'); + } + if (request.operation === 'local-render') { + return request.dataMode === 'local-only' + ? plan('ready', 'local-only', 'local-evidence-ready') + : plan('blocked', 'local-only', 'invalid-request'); + } + if (request.dataMode === 'local-only') { + return plan('blocked', 'local-only', 'invalid-request'); + } + if (request.aggregateConsent !== 'explicit') { + return plan('blocked', request.dataMode, 'aggregate-consent-required'); + } + if (!request.aggregateAvailable) { + return plan('blocked', request.dataMode, 'aggregate-unavailable'); + } + if (request.dataMode === 'aggregates-only') { + return plan('ready', request.dataMode, 'aggregate-evidence-ready'); + } + if (request.promptPersonalizationConsent !== 'explicit') { + return plan('blocked', request.dataMode, 'prompt-personalization-consent-required'); + } + + const snapshot = request.contentSnapshot; + if (!snapshot) { + return plan( + 'explicit-refresh-required', + request.dataMode, + 'content-snapshot-missing', + 'refresh-content-analysis', + ); + } + if (snapshot.status === 'stale') { + return plan( + 'explicit-refresh-required', + request.dataMode, + 'content-snapshot-stale', + 'refresh-content-analysis', + ); + } + if (snapshot.windowDays !== request.requestedWindowDays) { + return plan( + 'explicit-refresh-required', + request.dataMode, + 'content-window-mismatch', + 'refresh-content-analysis', + ); + } + if (snapshot.promptSampleCount === 0) { + return plan('blocked', request.dataMode, 'prompt-samples-unavailable'); + } + return plan('ready', request.dataMode, 'personalized-evidence-ready'); +} diff --git a/src/adviceEffectiveness/legacyPersonalization.ts b/src/adviceEffectiveness/legacyPersonalization.ts new file mode 100644 index 00000000..7a81fddc --- /dev/null +++ b/src/adviceEffectiveness/legacyPersonalization.ts @@ -0,0 +1,276 @@ +import { + MAX_PROMPT_SAMPLES, + MAX_PROMPT_SAMPLE_CHARS, + MAX_PROMPT_SAMPLE_TOTAL_CHARS, +} from './payload'; + +export const LEGACY_PERSONALIZATION_MIGRATION_VERSION = 1 as const; +export const MAX_LEGACY_USER_CONTEXT_CHARS = 1_000; + +export type LegacyPromptSampleAge = 'within-window' | 'older-than-window' | 'unknown'; + +export interface LegacyPersonalizationMigrationSummary { + requiresExplicitPromptPersonalizationConsent: boolean; + userContextChars: number; + userContextTruncated: boolean; + promptSampleCount: number; + promptSamplesTruncated: boolean; + sampleAge: { + withinWindow: number; + olderThanWindow: number; + unknown: number; + }; +} + +export interface LegacyPersonalizationDraft { + schemaVersion: typeof LEGACY_PERSONALIZATION_MIGRATION_VERSION; + windowDays: number; + userContext?: string; + promptSamples: { text: string; age: LegacyPromptSampleAge }[]; + /** Content-free metadata safe for migration/status UI. */ + migration: LegacyPersonalizationMigrationSummary; +} + +export interface LegacyPersonalizationProjection { + schemaVersion: typeof LEGACY_PERSONALIZATION_MIGRATION_VERSION; + consent: 'explicit'; + windowDays: number; + userContext?: string; + promptSamples: { text: string; age: LegacyPromptSampleAge }[]; +} + +export type BuildLegacyPersonalizationDraftResult = + | { ok: true; value: LegacyPersonalizationDraft } + | { ok: false; reason: 'invalid-legacy-personalization' }; + +interface LegacyPromptSampleInput { + text?: unknown; + observedAtEpochMs?: unknown; + [key: string]: unknown; +} + +export interface LegacyPersonalizationInput { + promptWindowDays: unknown; + userContext: unknown; + promptSamples: unknown; + nowEpochMs: unknown; +} + +function invalid(): BuildLegacyPersonalizationDraftResult { + return { ok: false, reason: 'invalid-legacy-personalization' }; +} + +function sampleAge( + observedAtEpochMs: number | undefined, + cutoffEpochMs: number, +): LegacyPromptSampleAge { + if (observedAtEpochMs === undefined) return 'unknown'; + return observedAtEpochMs >= cutoffEpochMs ? 'within-window' : 'older-than-window'; +} + +function hasExactKeys(value: Record, expected: string[]): boolean { + return Object.keys(value).sort().join('\0') === [...expected].sort().join('\0'); +} + +function validDraft(value: unknown): value is LegacyPersonalizationDraft { + if (typeof value !== 'object' || value === null || Array.isArray(value)) return false; + const draft = value as Record; + const rootKeys = ['migration', 'promptSamples', 'schemaVersion', 'windowDays']; + if (draft.userContext !== undefined) rootKeys.push('userContext'); + if (!hasExactKeys(draft, rootKeys)) return false; + if ( + draft.schemaVersion !== LEGACY_PERSONALIZATION_MIGRATION_VERSION || + typeof draft.windowDays !== 'number' || + !Number.isInteger(draft.windowDays) || + draft.windowDays < 1 || + draft.windowDays > 365 || + (draft.userContext !== undefined && + (typeof draft.userContext !== 'string' || + draft.userContext.length < 1 || + draft.userContext.length > MAX_LEGACY_USER_CONTEXT_CHARS || + draft.userContext.trim() !== draft.userContext)) || + !Array.isArray(draft.promptSamples) || + draft.promptSamples.length > MAX_PROMPT_SAMPLES + ) { + return false; + } + + const sampleAgeCounts = { withinWindow: 0, olderThanWindow: 0, unknown: 0 }; + let totalChars = 0; + for (const candidate of draft.promptSamples) { + if (typeof candidate !== 'object' || candidate === null || Array.isArray(candidate)) return false; + const sample = candidate as Record; + if ( + !hasExactKeys(sample, ['age', 'text']) || + typeof sample.text !== 'string' || + sample.text.length < 1 || + sample.text.length > MAX_PROMPT_SAMPLE_CHARS || + sample.text.trim() !== sample.text || + (sample.age !== 'within-window' && sample.age !== 'older-than-window' && sample.age !== 'unknown') + ) { + return false; + } + totalChars += sample.text.length; + if (totalChars > MAX_PROMPT_SAMPLE_TOTAL_CHARS) return false; + sampleAgeCounts[ + sample.age === 'within-window' + ? 'withinWindow' + : sample.age === 'older-than-window' + ? 'olderThanWindow' + : 'unknown' + ] += 1; + } + + if (typeof draft.migration !== 'object' || draft.migration === null || Array.isArray(draft.migration)) { + return false; + } + const migration = draft.migration as Record; + if (!hasExactKeys(migration, [ + 'promptSampleCount', + 'promptSamplesTruncated', + 'requiresExplicitPromptPersonalizationConsent', + 'sampleAge', + 'userContextChars', + 'userContextTruncated', + ])) { + return false; + } + if ( + migration.requiresExplicitPromptPersonalizationConsent !== + (draft.userContext !== undefined || draft.promptSamples.length > 0) || + migration.userContextChars !== (draft.userContext?.length ?? 0) || + typeof migration.userContextTruncated !== 'boolean' || + migration.promptSampleCount !== draft.promptSamples.length || + typeof migration.promptSamplesTruncated !== 'boolean' || + typeof migration.sampleAge !== 'object' || + migration.sampleAge === null || + Array.isArray(migration.sampleAge) + ) { + return false; + } + const ages = migration.sampleAge as Record; + return ( + hasExactKeys(ages, ['olderThanWindow', 'unknown', 'withinWindow']) && + ages.withinWindow === sampleAgeCounts.withinWindow && + ages.olderThanWindow === sampleAgeCounts.olderThanWindow && + ages.unknown === sampleAgeCounts.unknown + ); +} + +/** + * Rebuild the old settings/content into a host-only, bounded draft. Extra + * runtime fields (cwd, session IDs, paths) have no destination. No projection + * is returned until the new prompt-personalization consent is explicit. + */ +export function buildLegacyPersonalizationDraft( + input: LegacyPersonalizationInput, +): BuildLegacyPersonalizationDraftResult { + if ( + typeof input !== 'object' || + input === null || + typeof input.promptWindowDays !== 'number' || + !Number.isInteger(input.promptWindowDays) || + input.promptWindowDays < 1 || + input.promptWindowDays > 365 || + typeof input.nowEpochMs !== 'number' || + !Number.isFinite(input.nowEpochMs) || + input.nowEpochMs < 0 || + typeof input.userContext !== 'string' || + !Array.isArray(input.promptSamples) + ) { + return invalid(); + } + + const nowEpochMs = input.nowEpochMs; + const cutoffEpochMs = nowEpochMs - input.promptWindowDays * 86_400_000; + const rawContext = input.userContext.trim(); + const userContext = rawContext.slice(0, MAX_LEGACY_USER_CONTEXT_CHARS); + const promptSamples: LegacyPersonalizationDraft['promptSamples'] = []; + let totalChars = 0; + let promptSamplesTruncated = input.promptSamples.length > MAX_PROMPT_SAMPLES; + const ages = { withinWindow: 0, olderThanWindow: 0, unknown: 0 }; + + const candidates = input.promptSamples.slice(0, MAX_PROMPT_SAMPLES) as LegacyPromptSampleInput[]; + for (const candidate of candidates) { + if (typeof candidate !== 'object' || candidate === null || typeof candidate.text !== 'string') { + return invalid(); + } + let observedAtEpochMs: number | undefined; + if (candidate.observedAtEpochMs !== undefined) { + if ( + typeof candidate.observedAtEpochMs !== 'number' || + !Number.isFinite(candidate.observedAtEpochMs) || + candidate.observedAtEpochMs < 0 || + candidate.observedAtEpochMs > nowEpochMs + ) { + return invalid(); + } + observedAtEpochMs = candidate.observedAtEpochMs; + } + + const trimmed = candidate.text.trim(); + if (trimmed.length === 0) { + promptSamplesTruncated = true; + continue; + } + const individuallyBounded = trimmed.slice(0, MAX_PROMPT_SAMPLE_CHARS); + const remaining = MAX_PROMPT_SAMPLE_TOTAL_CHARS - totalChars; + if (remaining <= 0) { + promptSamplesTruncated = true; + break; + } + const text = individuallyBounded.slice(0, remaining); + if (text.length !== candidate.text.length || text.length !== trimmed.length) { + promptSamplesTruncated = true; + } + const age = sampleAge(observedAtEpochMs, cutoffEpochMs); + ages[age === 'within-window' ? 'withinWindow' : age === 'older-than-window' ? 'olderThanWindow' : 'unknown'] += 1; + promptSamples.push({ text, age }); + totalChars += text.length; + if (text.length < individuallyBounded.length) { + promptSamplesTruncated = true; + break; + } + } + if (promptSamples.length < input.promptSamples.length) promptSamplesTruncated = true; + + const effectiveContext = userContext.length > 0 ? userContext : undefined; + return { + ok: true, + value: { + schemaVersion: LEGACY_PERSONALIZATION_MIGRATION_VERSION, + windowDays: input.promptWindowDays, + ...(effectiveContext === undefined ? {} : { userContext: effectiveContext }), + promptSamples, + migration: { + requiresExplicitPromptPersonalizationConsent: + effectiveContext !== undefined || promptSamples.length > 0, + userContextChars: effectiveContext?.length ?? 0, + userContextTruncated: rawContext.length > MAX_LEGACY_USER_CONTEXT_CHARS, + promptSampleCount: promptSamples.length, + promptSamplesTruncated, + sampleAge: ages, + }, + }, + }; +} + +/** + * Create the exact future payload addendum only after separate consent. The + * current v2.3.1 prototype deliberately has no production caller. + */ +export function projectLegacyPersonalization( + draft: unknown, + consent: unknown, +): LegacyPersonalizationProjection | undefined { + if (consent !== 'explicit' || !validDraft(draft)) { + return undefined; + } + return { + schemaVersion: LEGACY_PERSONALIZATION_MIGRATION_VERSION, + consent: 'explicit', + windowDays: draft.windowDays, + ...(draft.userContext === undefined ? {} : { userContext: draft.userContext }), + promptSamples: draft.promptSamples.map((sample) => ({ ...sample })), + }; +} diff --git a/src/adviceEffectiveness/versionedPersistence.ts b/src/adviceEffectiveness/versionedPersistence.ts index 954d50dc..3afe88ab 100644 --- a/src/adviceEffectiveness/versionedPersistence.ts +++ b/src/adviceEffectiveness/versionedPersistence.ts @@ -313,6 +313,7 @@ function parseCurrentState(value: unknown): AdviceLocalState | undefined { !oneOf(value.featureMode, ['disabled', 'enabled'] as const) || !oneOf(value.aggregateConsent, ['not-granted', 'explicit'] as const) || !oneOf(value.promptSampleConsent, ['not-granted', 'explicit'] as const) || + (value.promptSampleConsent === 'explicit' && value.aggregateConsent !== 'explicit') || !Array.isArray(value.feedback) || value.feedback.length > MAX_PERSISTED_ADVICE_FEEDBACK || !Array.isArray(value.comparablePairs) || diff --git a/src/test/adviceEvidencePreparation.test.ts b/src/test/adviceEvidencePreparation.test.ts new file mode 100644 index 00000000..e2cd5190 --- /dev/null +++ b/src/test/adviceEvidencePreparation.test.ts @@ -0,0 +1,144 @@ +import { test } from 'node:test'; +import * as assert from 'node:assert/strict'; + +import { + AdviceEvidencePreparationRequest, + planAdviceEvidencePreparation, +} from '../adviceEffectiveness/evidencePreparation'; + +function request( + overrides: Partial = {}, +): AdviceEvidencePreparationRequest { + return { + featureMode: 'enabled', + operation: 'preview', + dataMode: 'aggregates-only', + aggregateAvailable: true, + aggregateConsent: 'explicit', + promptPersonalizationConsent: 'not-granted', + requestedWindowDays: 30, + ...overrides, + }; +} + +test('disabled advice preparation requests no aggregate or content work', () => { + const plan = planAdviceEvidencePreparation(request({ + featureMode: 'disabled', + dataMode: 'aggregates-with-prompt-samples', + aggregateAvailable: false, + aggregateConsent: 'not-granted', + promptPersonalizationConsent: 'not-granted', + })); + + assert.deepEqual(plan, { + schemaVersion: 1, + status: 'disabled', + dataMode: 'local-only', + reason: 'feature-disabled', + requiredHostAction: 'none', + allowAutomaticScan: false, + }); +}); + +test('local render and aggregates-only preview never require content analysis', () => { + const local = planAdviceEvidencePreparation(request({ + operation: 'local-render', + dataMode: 'local-only', + aggregateConsent: 'not-granted', + })); + assert.equal(local.status, 'ready'); + assert.equal(local.reason, 'local-evidence-ready'); + assert.equal(local.requiredHostAction, 'none'); + assert.equal(local.allowAutomaticScan, false); + + const aggregate = planAdviceEvidencePreparation(request({ + contentSnapshot: { + status: 'stale', + windowDays: 7, + promptSampleCount: 0, + }, + })); + assert.equal(aggregate.status, 'ready'); + assert.equal(aggregate.reason, 'aggregate-evidence-ready'); + assert.equal(aggregate.requiredHostAction, 'none'); + assert.equal(aggregate.allowAutomaticScan, false); +}); + +test('personalized preparation cannot request work before both consents', () => { + const noAggregateConsent = planAdviceEvidencePreparation(request({ + dataMode: 'aggregates-with-prompt-samples', + aggregateConsent: 'not-granted', + promptPersonalizationConsent: 'explicit', + })); + assert.equal(noAggregateConsent.status, 'blocked'); + assert.equal(noAggregateConsent.reason, 'aggregate-consent-required'); + assert.equal(noAggregateConsent.requiredHostAction, 'none'); + + const noPromptConsent = planAdviceEvidencePreparation(request({ + dataMode: 'aggregates-with-prompt-samples', + promptPersonalizationConsent: 'not-granted', + })); + assert.equal(noPromptConsent.status, 'blocked'); + assert.equal(noPromptConsent.reason, 'prompt-personalization-consent-required'); + assert.equal(noPromptConsent.requiredHostAction, 'none'); + assert.equal(noPromptConsent.allowAutomaticScan, false); +}); + +test('personalized preparation asks the host for one explicit refresh only when needed', () => { + const base = { + dataMode: 'aggregates-with-prompt-samples' as const, + promptPersonalizationConsent: 'explicit' as const, + }; + for (const [contentSnapshot, reason] of [ + [undefined, 'content-snapshot-missing'], + [{ status: 'stale', windowDays: 30, promptSampleCount: 3 }, 'content-snapshot-stale'], + [{ status: 'ready', windowDays: 7, promptSampleCount: 3 }, 'content-window-mismatch'], + ] as const) { + const plan = planAdviceEvidencePreparation(request({ ...base, contentSnapshot })); + assert.equal(plan.status, 'explicit-refresh-required'); + assert.equal(plan.reason, reason); + assert.equal(plan.requiredHostAction, 'refresh-content-analysis'); + assert.equal(plan.allowAutomaticScan, false); + } +}); + +test('personalized preparation is ready only for a matching materialized snapshot', () => { + const ready = planAdviceEvidencePreparation(request({ + operation: 'send', + dataMode: 'aggregates-with-prompt-samples', + promptPersonalizationConsent: 'explicit', + contentSnapshot: { + status: 'ready', + windowDays: 30, + promptSampleCount: 4, + }, + })); + assert.equal(ready.status, 'ready'); + assert.equal(ready.reason, 'personalized-evidence-ready'); + assert.equal(ready.requiredHostAction, 'none'); + assert.equal(ready.allowAutomaticScan, false); + + const empty = planAdviceEvidencePreparation(request({ + dataMode: 'aggregates-with-prompt-samples', + promptPersonalizationConsent: 'explicit', + contentSnapshot: { + status: 'ready', + windowDays: 30, + promptSampleCount: 0, + }, + })); + assert.equal(empty.status, 'blocked'); + assert.equal(empty.reason, 'prompt-samples-unavailable'); + assert.equal(empty.requiredHostAction, 'none'); +}); + +test('invalid preparation combinations fail closed without work', () => { + const invalid = planAdviceEvidencePreparation(request({ + operation: 'local-render', + dataMode: 'aggregates-only', + })); + assert.equal(invalid.status, 'blocked'); + assert.equal(invalid.reason, 'invalid-request'); + assert.equal(invalid.requiredHostAction, 'none'); + assert.equal(invalid.allowAutomaticScan, false); +}); diff --git a/src/test/adviceLegacyPersonalization.test.ts b/src/test/adviceLegacyPersonalization.test.ts new file mode 100644 index 00000000..016c042f --- /dev/null +++ b/src/test/adviceLegacyPersonalization.test.ts @@ -0,0 +1,134 @@ +import { test } from 'node:test'; +import * as assert from 'node:assert/strict'; + +import { + buildLegacyPersonalizationDraft, + LegacyPersonalizationDraft, + projectLegacyPersonalization, +} from '../adviceEffectiveness/legacyPersonalization'; + +const NOW = Date.parse('2026-08-27T12:00:00.000Z'); + +test('legacy user context and prompt samples remain host-only without separate consent', () => { + const draft = buildLegacyPersonalizationDraft({ + promptWindowDays: 30, + userContext: ' PRIVATE_CONTEXT ', + promptSamples: [{ + text: 'PRIVATE_PROMPT', + observedAtEpochMs: NOW - 2 * 86_400_000, + cwd: '/private/workspace', + sessionId: 'private-session', + }], + nowEpochMs: NOW, + }); + assert.equal(draft.ok, true); + if (!draft.ok) return; + + assert.equal(projectLegacyPersonalization(draft.value, 'not-granted'), undefined); + assert.equal(draft.value.migration.requiresExplicitPromptPersonalizationConsent, true); + assert.equal(draft.value.migration.userContextChars, 'PRIVATE_CONTEXT'.length); + assert.equal(draft.value.migration.promptSampleCount, 1); + assert.deepEqual(draft.value.migration.sampleAge, { + withinWindow: 1, + olderThanWindow: 0, + unknown: 0, + }); + assert.doesNotMatch(JSON.stringify(draft.value.migration), /PRIVATE_|workspace|session/); +}); + +test('explicit projection rebuilds an exact allowlisted personalization object', () => { + const draft = buildLegacyPersonalizationDraft({ + promptWindowDays: 30, + userContext: ' Project uses a strict review rubric. ', + promptSamples: [ + { text: ' Please review this change. ', observedAtEpochMs: NOW - 5 * 86_400_000 }, + { text: 'Older sample', observedAtEpochMs: NOW - 40 * 86_400_000 }, + { text: 'Unknown age' }, + ], + nowEpochMs: NOW, + }); + assert.equal(draft.ok, true); + if (!draft.ok) return; + + const projected = projectLegacyPersonalization(draft.value, 'explicit'); + assert.deepEqual(projected, { + schemaVersion: 1, + consent: 'explicit', + windowDays: 30, + userContext: 'Project uses a strict review rubric.', + promptSamples: [ + { text: 'Please review this change.', age: 'within-window' }, + { text: 'Older sample', age: 'older-than-window' }, + { text: 'Unknown age', age: 'unknown' }, + ], + }); +}); + +test('legacy personalization uses payload bounds and exposes truncation in migration metadata', () => { + const draft = buildLegacyPersonalizationDraft({ + promptWindowDays: 30, + userContext: `context-${'x'.repeat(2_000)}`, + promptSamples: Array.from({ length: 25 }, (_value, index) => ({ + text: `sample-${index}-${'y'.repeat(2_000)}`, + observedAtEpochMs: NOW, + })), + nowEpochMs: NOW, + }); + assert.equal(draft.ok, true); + if (!draft.ok) return; + + const projected = projectLegacyPersonalization(draft.value, 'explicit'); + assert.ok(projected); + assert.equal(projected.userContext?.length, 1_000); + assert.equal(projected.promptSamples.length, 12); + assert.equal( + projected.promptSamples.reduce((sum, sample) => sum + sample.text.length, 0), + 12_000, + ); + assert.equal(draft.value.migration.userContextTruncated, true); + assert.equal(draft.value.migration.promptSamplesTruncated, true); +}); + +test('legacy personalization rejects invalid windows and timestamps without retaining content', () => { + for (const input of [ + { + promptWindowDays: 0, + userContext: 'PRIVATE_CONTEXT', + promptSamples: [], + nowEpochMs: NOW, + }, + { + promptWindowDays: 30, + userContext: '', + promptSamples: [{ text: 'PRIVATE_PROMPT', observedAtEpochMs: Number.NaN }], + nowEpochMs: NOW, + }, + ]) { + const result = buildLegacyPersonalizationDraft(input); + assert.deepEqual(result, { ok: false, reason: 'invalid-legacy-personalization' }); + assert.doesNotMatch(JSON.stringify(result), /PRIVATE_/); + } +}); + +test('legacy personalization projection revalidates a hostile in-memory draft', () => { + const hostile = { + schemaVersion: 1, + windowDays: 30, + userContext: 'PRIVATE_CONTEXT', + promptSamples: [{ + text: 'PRIVATE_PROMPT', + age: 'within-window', + cwd: '/private/workspace', + }], + migration: { + requiresExplicitPromptPersonalizationConsent: true, + userContextChars: 15, + userContextTruncated: false, + promptSampleCount: 1, + promptSamplesTruncated: false, + sampleAge: { withinWindow: 1, olderThanWindow: 0, unknown: 0 }, + }, + } as unknown as LegacyPersonalizationDraft; + + assert.equal(projectLegacyPersonalization(hostile, 'explicit'), undefined); +}); diff --git a/src/test/advicePersistence.test.ts b/src/test/advicePersistence.test.ts index c6166a05..dd5728d8 100644 --- a/src/test/advicePersistence.test.ts +++ b/src/test/advicePersistence.test.ts @@ -105,6 +105,29 @@ test('unknown, corrupt, or identifying local data fails closed and is never over } }); +test('current local state rejects prompt consent without aggregate consent', async () => { + const inconsistent = { + ...createClosedAdviceLocalState(), + featureMode: 'enabled' as const, + aggregateConsent: 'not-granted' as const, + promptSampleConsent: 'explicit' as const, + }; + const storage = new MemoryStorage(); + storage.value = inconsistent; + + const loaded = await loadAndMigrateAdviceLocalState(storage); + assert.deepEqual(loaded, { + ok: false, + reason: 'invalid-local-data', + value: createClosedAdviceLocalState(), + }); + assert.equal(storage.updates.length, 0); + + const saved = await saveAdviceLocalState(storage, inconsistent); + assert.deepEqual(saved, { ok: false, reason: 'invalid-local-data' }); + assert.equal(storage.updates.length, 0); +}); + test('migration write failure returns a closed state without exposing the storage error', async () => { const storage = new MemoryStorage(); storage.value = { diff --git a/src/test/adviceRuntimePolicy.test.ts b/src/test/adviceRuntimePolicy.test.ts index bdb05a5b..5df6b659 100644 --- a/src/test/adviceRuntimePolicy.test.ts +++ b/src/test/adviceRuntimePolicy.test.ts @@ -22,3 +22,20 @@ test('extension advice and optimizer runtime stay API/BYOK-only', () => { assert.match(runtimeAdvice, /apiUrl:\s*config\.adviceApiUrl/); assert.doesNotMatch(runtimeAdvice, /getAccessToken\s*\(/); }); + +test('future advice seams remain unreachable from production host and webview code', () => { + const production = [ + readFileSync(path.join(root, 'src', 'extension.ts'), 'utf8'), + readFileSync(path.join(root, 'src', 'webview.ts'), 'utf8'), + ].join('\n'); + + for (const seam of [ + 'planAdviceEvidencePreparation', + 'buildLegacyPersonalizationDraft', + 'projectLegacyPersonalization', + 'requestStructuredAdviceViaLegacyByok', + 'appendStoredComparablePair', + ]) { + assert.doesNotMatch(production, new RegExp(`\\b${seam}\\b`)); + } +}); From 09d44c714862fbdc046ade443635465da31b26d9 Mon Sep 17 00:00:00 2001 From: Carl <74130028+Carl723000@users.noreply.github.com> Date: Sun, 30 Aug 2026 01:34:13 +0800 Subject: [PATCH 05/72] feat: complete v2.3.1 advice and history candidate Co-authored-by: OpenAI Codex <215057067+openai-codex[bot]@users.noreply.github.com> --- src/adviceEffectiveness/comparisonPairing.ts | 562 ++++++ .../comparisonProduction.ts | 413 ++++ src/adviceEffectiveness/comparisonResult.ts | 615 ++++++ src/adviceEffectiveness/contract.ts | 12 +- src/adviceEffectiveness/integration.ts | 29 + src/adviceEffectiveness/legacyBridge.ts | 14 +- src/adviceEffectiveness/payload.ts | 38 +- src/adviceEffectiveness/preparedRequest.ts | 255 +++ src/adviceEffectiveness/remoteAdvice.ts | 113 ++ .../versionedPersistence.ts | 422 ++-- src/advisor.ts | 243 +-- src/backgroundWorkState.ts | 451 +++++ src/claudeApiClient.ts | 49 +- src/claudeIncrementalIndex.ts | 136 +- src/codexViewComponents.ts | 11 + src/dataLoader.ts | 95 +- src/dateKeys.ts | 5 + src/extension.ts | 1352 ++++++++++--- src/httpClient.ts | 55 +- src/i18n.ts | 306 ++- src/optimizerRequest.ts | 133 ++ src/providers/codex/codexFilePassPool.ts | 44 +- src/providers/codex/codexIndex.ts | 445 ++++- src/providers/codex/codexIndexClient.ts | 103 +- src/providers/codex/codexIndexWorker.ts | 4 +- src/providers/codex/codexPeriodIndex.ts | 58 +- src/providers/codex/codexProvider.ts | 65 +- src/providers/codex/codexUsage.ts | 45 +- src/providers/codex/codexWorkerProtocol.ts | 3 + src/resourceOwnership.ts | 204 ++ src/settings.ts | 19 +- src/test/adviceComparisonPairing.test.ts | 148 ++ src/test/adviceComparisonProduction.test.ts | 239 +++ src/test/adviceComparisonResult.test.ts | 189 ++ src/test/adviceContract.test.ts | 12 + src/test/adviceIntegration.test.ts | 28 + src/test/adviceLegacyBridge.test.ts | 11 +- src/test/advicePayload.test.ts | 40 +- src/test/advicePersistence.test.ts | 120 ++ src/test/advicePreparedRequest.test.ts | 129 ++ src/test/adviceRemoteInvocation.test.ts | 106 ++ src/test/adviceRuntimePolicy.test.ts | 73 +- src/test/adviceStateSerialization.test.ts | 310 +++ src/test/backgroundWorkState.test.ts | 255 +++ src/test/claudeApiClient.test.ts | 30 + src/test/claudeIncrementalIndex.test.ts | 71 + src/test/codexIndex.test.ts | 231 ++- src/test/codexIndexClient.test.ts | 122 +- src/test/codexPeriodIndex.test.ts | 40 + src/test/codexUsage.test.ts | 121 +- src/test/codexWebviewFixtures.ts | 69 +- src/test/dataLoader.test.ts | 38 + src/test/dateKeys.test.ts | 10 + src/test/extensionRefreshRouting.test.ts | 10 +- src/test/extensionWindowActivity.test.ts | 817 +++++++- src/test/i18nTimezone.test.ts | 12 + src/test/optimizerPreparedFlow.test.ts | 84 + src/test/providerSelection.test.ts | 4 +- src/test/resourceOwnership.test.ts | 160 ++ src/test/resourceOwnershipIntegration.test.ts | 53 + src/types.ts | 2 +- src/webview.ts | 1692 +++++++++++++++-- tests/ui/advice-effectiveness.spec.mjs | 239 ++- tests/ui/codex-accessibility.spec.mjs | 7 +- tests/ui/codex-interactions.spec.mjs | 63 +- tests/ui/codex-responsive.spec.mjs | 22 +- tests/ui/codex-visual.spec.mjs | 40 +- .../support/advice-effectiveness-fixture.cjs | 4 + tests/ui/support/render-harness.cjs | 46 + tests/ui/support/server.mjs | 8 + 70 files changed, 10683 insertions(+), 1271 deletions(-) create mode 100644 src/adviceEffectiveness/comparisonPairing.ts create mode 100644 src/adviceEffectiveness/comparisonProduction.ts create mode 100644 src/adviceEffectiveness/comparisonResult.ts create mode 100644 src/adviceEffectiveness/preparedRequest.ts create mode 100644 src/adviceEffectiveness/remoteAdvice.ts create mode 100644 src/backgroundWorkState.ts create mode 100644 src/optimizerRequest.ts create mode 100644 src/resourceOwnership.ts create mode 100644 src/test/adviceComparisonPairing.test.ts create mode 100644 src/test/adviceComparisonProduction.test.ts create mode 100644 src/test/adviceComparisonResult.test.ts create mode 100644 src/test/advicePreparedRequest.test.ts create mode 100644 src/test/adviceRemoteInvocation.test.ts create mode 100644 src/test/adviceStateSerialization.test.ts create mode 100644 src/test/backgroundWorkState.test.ts create mode 100644 src/test/optimizerPreparedFlow.test.ts create mode 100644 src/test/resourceOwnership.test.ts create mode 100644 src/test/resourceOwnershipIntegration.test.ts diff --git a/src/adviceEffectiveness/comparisonPairing.ts b/src/adviceEffectiveness/comparisonPairing.ts new file mode 100644 index 00000000..7ed7d46b --- /dev/null +++ b/src/adviceEffectiveness/comparisonPairing.ts @@ -0,0 +1,562 @@ +import { createHash } from 'node:crypto'; + +import type { ComparableTaskPair } from './comparison'; +import { isAdviceIdentifier } from './contract'; + +export type StoredAdviceScope = 'overall' | 'project' | 'task-cohort'; +export type StoredProvider = 'claude' | 'codex'; +export type StoredTaskKind = 'small-change' | 'feature' | 'bug-fix' | 'review' | 'other'; +export type StoredComplexityBand = 'low' | 'medium' | 'high'; +export type StoredModelFamily = 'opus' | 'sonnet' | 'haiku' | 'fable' | 'other'; +export type StoredEffort = 'low' | 'medium' | 'high' | 'xhigh' | 'max' | 'ultra' | 'unknown'; +export type StoredMetricUnit = 'count' | 'tokens' | 'ratio' | 'multiple' | 'milliseconds'; +export type StoredMetricDirection = 'lower-is-better' | 'higher-is-better'; +export type StoredQualityPass = 'passed' | 'failed' | 'unknown'; +export type StoredCoverage = 'complete' | 'partial' | 'unknown'; +export type StoredConfidence = 'high' | 'medium' | 'low' | 'unknown'; + +export interface StoredComparableContext { + scope: StoredAdviceScope; + taskKind: StoredTaskKind; + complexityBand: StoredComplexityBand; + provider: StoredProvider; + modelFamily: StoredModelFamily; + effort: StoredEffort; + measurementProfileVersion: string; + metricDefinitionVersion: string; + qualityRubricId: string; +} + +export interface ComparableObservation { + context: StoredComparableContext; + metric: { + name: string; + unit: StoredMetricUnit; + direction: StoredMetricDirection; + value: number; + }; + quality: { + score: number; + passed: StoredQualityPass; + evidenceCount: number; + }; + evidence: { + coverage: StoredCoverage; + confidence: StoredConfidence; + qualityFlags: string[]; + }; + observedAtEpochMs: number; +} + +export interface ComparablePairBuildInput { + adviceId: string; + recommendationId: string; + recommendationVersion: string; + before: ComparableObservation; + after: ComparableObservation; + recordedAtEpochMs: number; +} + +export interface StoredComparablePair { + pairId: string; + adviceId: string; + recommendationId: string; + recommendationVersion: string; + context: StoredComparableContext; + metric: { + name: string; + unit: StoredMetricUnit; + direction: StoredMetricDirection; + beforeValue: number; + afterValue: number; + }; + quality: { + beforeScore: number; + afterScore: number; + beforePassed: StoredQualityPass; + afterPassed: StoredQualityPass; + beforeEvidenceCount: number; + afterEvidenceCount: number; + }; + evidence: { + beforeCoverage: StoredCoverage; + afterCoverage: StoredCoverage; + beforeConfidence: StoredConfidence; + afterConfidence: StoredConfidence; + beforeQualityFlags: string[]; + afterQualityFlags: string[]; + }; + recordedAtEpochMs: number; +} + +export type ComparablePairEvidenceIssue = + | 'scope-mismatch' + | 'task-kind-mismatch' + | 'complexity-mismatch' + | 'provider-mismatch' + | 'model-mismatch' + | 'effort-mismatch' + | 'measurement-profile-mismatch' + | 'metric-definition-mismatch' + | 'quality-rubric-mismatch' + | 'metric-name-mismatch' + | 'metric-unit-mismatch' + | 'metric-direction-mismatch' + | 'incomplete-coverage' + | 'insufficient-confidence' + | 'unresolved-quality-flags' + | 'unknown-quality-result' + | 'missing-quality-evidence' + | 'failed-quality-baseline' + | 'unknown-effort' + | 'zero-metric-baseline' + | 'invalid-observation-order'; + +export type ComparablePairBuildResult = + | { ok: true; value: StoredComparablePair } + | { + ok: false; + reason: 'invalid-input'; + issues: ['invalid-sanitized-observation']; + } + | { + ok: false; + reason: 'evidence-insufficient'; + issues: ComparablePairEvidenceIssue[]; + }; + +const BUILD_INPUT_KEYS = [ + 'adviceId', + 'after', + 'before', + 'recommendationId', + 'recommendationVersion', + 'recordedAtEpochMs', +] as const; +const OBSERVATION_KEYS = ['context', 'evidence', 'metric', 'observedAtEpochMs', 'quality'] as const; +const CONTEXT_KEYS = [ + 'complexityBand', + 'effort', + 'measurementProfileVersion', + 'metricDefinitionVersion', + 'modelFamily', + 'provider', + 'qualityRubricId', + 'scope', + 'taskKind', +] as const; +const OBSERVATION_METRIC_KEYS = ['direction', 'name', 'unit', 'value'] as const; +const OBSERVATION_QUALITY_KEYS = ['evidenceCount', 'passed', 'score'] as const; +const OBSERVATION_EVIDENCE_KEYS = ['confidence', 'coverage', 'qualityFlags'] as const; +const PAIR_KEYS = [ + 'adviceId', + 'context', + 'evidence', + 'metric', + 'pairId', + 'quality', + 'recommendationId', + 'recommendationVersion', + 'recordedAtEpochMs', +] as const; +const PAIR_METRIC_KEYS = ['afterValue', 'beforeValue', 'direction', 'name', 'unit'] as const; +const PAIR_QUALITY_KEYS = [ + 'afterEvidenceCount', + 'afterPassed', + 'afterScore', + 'beforeEvidenceCount', + 'beforePassed', + 'beforeScore', +] as const; +const PAIR_EVIDENCE_KEYS = [ + 'afterConfidence', + 'afterCoverage', + 'afterQualityFlags', + 'beforeConfidence', + 'beforeCoverage', + 'beforeQualityFlags', +] as const; + +function isObject(value: unknown): value is Record { + return typeof value === 'object' && value !== null && !Array.isArray(value); +} + +function hasExactKeys(value: Record, expected: readonly string[]): boolean { + const actual = Object.keys(value).sort(); + const keys = [...expected].sort(); + return actual.length === keys.length && actual.every((key, index) => key === keys[index]); +} + +function oneOf(value: unknown, values: readonly T[]): value is T { + return typeof value === 'string' && (values as readonly string[]).includes(value); +} + +function isEpochMs(value: unknown): value is number { + return typeof value === 'number' && Number.isInteger(value) && value >= 0; +} + +function isFiniteNonNegative(value: unknown): value is number { + return typeof value === 'number' && Number.isFinite(value) && value >= 0; +} + +function isScore(value: unknown): value is number { + return isFiniteNonNegative(value) && value <= 1; +} + +function parseQualityFlags(value: unknown): string[] | undefined { + if (!Array.isArray(value) || value.length > 32) return undefined; + const result: string[] = []; + const seen = new Set(); + for (const flag of value) { + if (typeof flag !== 'string' || !isAdviceIdentifier(flag) || seen.has(flag)) return undefined; + seen.add(flag); + result.push(flag); + } + return result; +} + +function parseContext(value: unknown): StoredComparableContext | undefined { + if (!isObject(value) || !hasExactKeys(value, CONTEXT_KEYS)) return undefined; + if ( + !oneOf(value.scope, ['overall', 'project', 'task-cohort'] as const) || + !oneOf(value.taskKind, ['small-change', 'feature', 'bug-fix', 'review', 'other'] as const) || + !oneOf(value.complexityBand, ['low', 'medium', 'high'] as const) || + !oneOf(value.provider, ['claude', 'codex'] as const) || + !oneOf(value.modelFamily, ['opus', 'sonnet', 'haiku', 'fable', 'other'] as const) || + !oneOf(value.effort, ['low', 'medium', 'high', 'xhigh', 'max', 'ultra', 'unknown'] as const) || + typeof value.measurementProfileVersion !== 'string' || + typeof value.metricDefinitionVersion !== 'string' || + typeof value.qualityRubricId !== 'string' || + !isAdviceIdentifier(value.measurementProfileVersion) || + !isAdviceIdentifier(value.metricDefinitionVersion) || + !isAdviceIdentifier(value.qualityRubricId) + ) { + return undefined; + } + return { + scope: value.scope, + taskKind: value.taskKind, + complexityBand: value.complexityBand, + provider: value.provider, + modelFamily: value.modelFamily, + effort: value.effort, + measurementProfileVersion: value.measurementProfileVersion, + metricDefinitionVersion: value.metricDefinitionVersion, + qualityRubricId: value.qualityRubricId, + }; +} + +function parseObservation(value: unknown): ComparableObservation | undefined { + if (!isObject(value) || !hasExactKeys(value, OBSERVATION_KEYS)) return undefined; + const context = parseContext(value.context); + if ( + !context || + !isObject(value.metric) || + !hasExactKeys(value.metric, OBSERVATION_METRIC_KEYS) || + !isObject(value.quality) || + !hasExactKeys(value.quality, OBSERVATION_QUALITY_KEYS) || + !isObject(value.evidence) || + !hasExactKeys(value.evidence, OBSERVATION_EVIDENCE_KEYS) || + !isEpochMs(value.observedAtEpochMs) + ) { + return undefined; + } + const metric = value.metric; + const quality = value.quality; + const evidence = value.evidence; + if ( + typeof metric.name !== 'string' || + !isAdviceIdentifier(metric.name) || + !oneOf(metric.unit, ['count', 'tokens', 'ratio', 'multiple', 'milliseconds'] as const) || + !oneOf(metric.direction, ['lower-is-better', 'higher-is-better'] as const) || + !isFiniteNonNegative(metric.value) || + !isScore(quality.score) || + !oneOf(quality.passed, ['passed', 'failed', 'unknown'] as const) || + !isEpochMs(quality.evidenceCount) || + !oneOf(evidence.coverage, ['complete', 'partial', 'unknown'] as const) || + !oneOf(evidence.confidence, ['high', 'medium', 'low', 'unknown'] as const) + ) { + return undefined; + } + const qualityFlags = parseQualityFlags(evidence.qualityFlags); + if (!qualityFlags) return undefined; + return { + context, + metric: { + name: metric.name, + unit: metric.unit, + direction: metric.direction, + value: metric.value, + }, + quality: { + score: quality.score, + passed: quality.passed, + evidenceCount: quality.evidenceCount, + }, + evidence: { + coverage: evidence.coverage, + confidence: evidence.confidence, + qualityFlags, + }, + observedAtEpochMs: value.observedAtEpochMs, + }; +} + +/** Strictly parse the content-free persisted representation and clone its arrays. */ +export function parseStoredComparablePair(value: unknown): StoredComparablePair | undefined { + if (!isObject(value) || !hasExactKeys(value, PAIR_KEYS)) return undefined; + if ( + typeof value.pairId !== 'string' || + typeof value.adviceId !== 'string' || + typeof value.recommendationId !== 'string' || + typeof value.recommendationVersion !== 'string' || + !isAdviceIdentifier(value.pairId) || + !isAdviceIdentifier(value.adviceId) || + !isAdviceIdentifier(value.recommendationId) || + !isAdviceIdentifier(value.recommendationVersion) || + !isEpochMs(value.recordedAtEpochMs) + ) { + return undefined; + } + const context = parseContext(value.context); + if ( + !context || + !isObject(value.metric) || + !hasExactKeys(value.metric, PAIR_METRIC_KEYS) || + !isObject(value.quality) || + !hasExactKeys(value.quality, PAIR_QUALITY_KEYS) || + !isObject(value.evidence) || + !hasExactKeys(value.evidence, PAIR_EVIDENCE_KEYS) + ) { + return undefined; + } + const metric = value.metric; + const quality = value.quality; + const evidence = value.evidence; + if ( + typeof metric.name !== 'string' || + !isAdviceIdentifier(metric.name) || + !oneOf(metric.unit, ['count', 'tokens', 'ratio', 'multiple', 'milliseconds'] as const) || + !oneOf(metric.direction, ['lower-is-better', 'higher-is-better'] as const) || + !isFiniteNonNegative(metric.beforeValue) || + !isFiniteNonNegative(metric.afterValue) || + !isScore(quality.beforeScore) || + !isScore(quality.afterScore) || + !oneOf(quality.beforePassed, ['passed', 'failed', 'unknown'] as const) || + !oneOf(quality.afterPassed, ['passed', 'failed', 'unknown'] as const) || + !isEpochMs(quality.beforeEvidenceCount) || + !isEpochMs(quality.afterEvidenceCount) || + !oneOf(evidence.beforeCoverage, ['complete', 'partial', 'unknown'] as const) || + !oneOf(evidence.afterCoverage, ['complete', 'partial', 'unknown'] as const) || + !oneOf(evidence.beforeConfidence, ['high', 'medium', 'low', 'unknown'] as const) || + !oneOf(evidence.afterConfidence, ['high', 'medium', 'low', 'unknown'] as const) + ) { + return undefined; + } + const beforeQualityFlags = parseQualityFlags(evidence.beforeQualityFlags); + const afterQualityFlags = parseQualityFlags(evidence.afterQualityFlags); + if (!beforeQualityFlags || !afterQualityFlags) return undefined; + return { + pairId: value.pairId, + adviceId: value.adviceId, + recommendationId: value.recommendationId, + recommendationVersion: value.recommendationVersion, + context, + metric: { + name: metric.name, + unit: metric.unit, + direction: metric.direction, + beforeValue: metric.beforeValue, + afterValue: metric.afterValue, + }, + quality: { + beforeScore: quality.beforeScore, + afterScore: quality.afterScore, + beforePassed: quality.beforePassed, + afterPassed: quality.afterPassed, + beforeEvidenceCount: quality.beforeEvidenceCount, + afterEvidenceCount: quality.afterEvidenceCount, + }, + evidence: { + beforeCoverage: evidence.beforeCoverage, + afterCoverage: evidence.afterCoverage, + beforeConfidence: evidence.beforeConfidence, + afterConfidence: evidence.afterConfidence, + beforeQualityFlags, + afterQualityFlags, + }, + recordedAtEpochMs: value.recordedAtEpochMs, + }; +} + +function mismatchIssues(before: ComparableObservation, after: ComparableObservation): ComparablePairEvidenceIssue[] { + const issues: ComparablePairEvidenceIssue[] = []; + const compare = (left: T, right: T, issue: ComparablePairEvidenceIssue): void => { + if (left !== right) issues.push(issue); + }; + compare(before.context.scope, after.context.scope, 'scope-mismatch'); + compare(before.context.taskKind, after.context.taskKind, 'task-kind-mismatch'); + compare(before.context.complexityBand, after.context.complexityBand, 'complexity-mismatch'); + compare(before.context.provider, after.context.provider, 'provider-mismatch'); + compare(before.context.modelFamily, after.context.modelFamily, 'model-mismatch'); + compare(before.context.effort, after.context.effort, 'effort-mismatch'); + compare( + before.context.measurementProfileVersion, + after.context.measurementProfileVersion, + 'measurement-profile-mismatch', + ); + compare( + before.context.metricDefinitionVersion, + after.context.metricDefinitionVersion, + 'metric-definition-mismatch', + ); + compare(before.context.qualityRubricId, after.context.qualityRubricId, 'quality-rubric-mismatch'); + compare(before.metric.name, after.metric.name, 'metric-name-mismatch'); + compare(before.metric.unit, after.metric.unit, 'metric-unit-mismatch'); + compare(before.metric.direction, after.metric.direction, 'metric-direction-mismatch'); + return issues; +} + +function reliabilityIssues( + before: ComparableObservation, + after: ComparableObservation, + recordedAtEpochMs: number, +): ComparablePairEvidenceIssue[] { + const issues: ComparablePairEvidenceIssue[] = []; + const add = (issue: ComparablePairEvidenceIssue): void => { + if (!issues.includes(issue)) issues.push(issue); + }; + for (const observation of [before, after]) { + if (observation.evidence.coverage !== 'complete') add('incomplete-coverage'); + if (observation.evidence.confidence === 'low' || observation.evidence.confidence === 'unknown') { + add('insufficient-confidence'); + } + if (observation.evidence.qualityFlags.length > 0) add('unresolved-quality-flags'); + if (observation.quality.passed === 'unknown') add('unknown-quality-result'); + if (observation.quality.evidenceCount < 1) add('missing-quality-evidence'); + } + if (before.quality.passed === 'failed') add('failed-quality-baseline'); + if (before.context.effort === 'unknown') add('unknown-effort'); + if (before.metric.value === 0) add('zero-metric-baseline'); + if ( + before.observedAtEpochMs >= after.observedAtEpochMs || + recordedAtEpochMs < after.observedAtEpochMs + ) { + add('invalid-observation-order'); + } + return issues; +} + +function pairDigest(value: Omit): string { + return createHash('sha256').update(JSON.stringify(value), 'utf8').digest('hex').slice(0, 32); +} + +/** + * Pair two already-sanitized observations. Comparability and reliability are + * proven from coarse machine fields; free text, paths, and session identifiers + * are not part of this boundary and make the exact parser fail. + */ +export function buildComparableTaskPair(input: unknown): ComparablePairBuildResult { + if (!isObject(input) || !hasExactKeys(input, BUILD_INPUT_KEYS)) { + return { ok: false, reason: 'invalid-input', issues: ['invalid-sanitized-observation'] }; + } + const before = parseObservation(input.before); + const after = parseObservation(input.after); + if ( + typeof input.adviceId !== 'string' || + typeof input.recommendationId !== 'string' || + typeof input.recommendationVersion !== 'string' || + !isAdviceIdentifier(input.adviceId) || + !isAdviceIdentifier(input.recommendationId) || + !isAdviceIdentifier(input.recommendationVersion) || + !isEpochMs(input.recordedAtEpochMs) || + !before || + !after + ) { + return { ok: false, reason: 'invalid-input', issues: ['invalid-sanitized-observation'] }; + } + const issues = [ + ...mismatchIssues(before, after), + ...reliabilityIssues(before, after, input.recordedAtEpochMs), + ]; + if (issues.length > 0) return { ok: false, reason: 'evidence-insufficient', issues }; + + const pairWithoutId: Omit = { + adviceId: input.adviceId, + recommendationId: input.recommendationId, + recommendationVersion: input.recommendationVersion, + context: { ...before.context }, + metric: { + name: before.metric.name, + unit: before.metric.unit, + direction: before.metric.direction, + beforeValue: before.metric.value, + afterValue: after.metric.value, + }, + quality: { + beforeScore: before.quality.score, + afterScore: after.quality.score, + beforePassed: before.quality.passed, + afterPassed: after.quality.passed, + beforeEvidenceCount: before.quality.evidenceCount, + afterEvidenceCount: after.quality.evidenceCount, + }, + evidence: { + beforeCoverage: before.evidence.coverage, + afterCoverage: after.evidence.coverage, + beforeConfidence: before.evidence.confidence, + afterConfidence: after.evidence.confidence, + beforeQualityFlags: [...before.evidence.qualityFlags], + afterQualityFlags: [...after.evidence.qualityFlags], + }, + recordedAtEpochMs: input.recordedAtEpochMs, + }; + return { + ok: true, + value: { pairId: `pair-${pairDigest(pairWithoutId)}`, ...pairWithoutId }, + }; +} + +/** Convert only state that has already passed the exact persistence parser. */ +export function toComparableTaskPairs(pairs: readonly StoredComparablePair[]): ComparableTaskPair[] { + return pairs.map((pair) => { + const context = { + taskKind: pair.context.taskKind, + complexityBand: pair.context.complexityBand, + provider: pair.context.provider, + modelFamily: pair.context.modelFamily, + effort: pair.context.effort, + metricDefinitionVersion: pair.context.metricDefinitionVersion, + qualityRubricId: pair.context.qualityRubricId, + }; + const outcome = (period: 'before' | 'after'): ComparableTaskPair['before'] => ({ + context: { ...context }, + primaryMetric: { + name: pair.metric.name, + unit: pair.metric.unit, + direction: pair.metric.direction, + value: period === 'before' ? pair.metric.beforeValue : pair.metric.afterValue, + }, + quality: { + score: period === 'before' ? pair.quality.beforeScore : pair.quality.afterScore, + passed: (() => { + const value = period === 'before' ? pair.quality.beforePassed : pair.quality.afterPassed; + return value === 'unknown' ? null : value === 'passed'; + })(), + evidenceCount: + period === 'before' ? pair.quality.beforeEvidenceCount : pair.quality.afterEvidenceCount, + }, + evidence: { + coverage: period === 'before' ? pair.evidence.beforeCoverage : pair.evidence.afterCoverage, + confidence: + period === 'before' ? pair.evidence.beforeConfidence : pair.evidence.afterConfidence, + qualityFlags: [ + ...(period === 'before' + ? pair.evidence.beforeQualityFlags + : pair.evidence.afterQualityFlags), + ], + }, + }); + return { pairId: pair.pairId, before: outcome('before'), after: outcome('after') }; + }); +} diff --git a/src/adviceEffectiveness/comparisonProduction.ts b/src/adviceEffectiveness/comparisonProduction.ts new file mode 100644 index 00000000..56e7dc5c --- /dev/null +++ b/src/adviceEffectiveness/comparisonProduction.ts @@ -0,0 +1,413 @@ +import { isAdviceIdentifier } from './contract'; +import { + ComparableObservation, + StoredComparablePair, + StoredEffort, + StoredModelFamily, + buildComparableTaskPair, +} from './comparisonPairing'; + +export const CODEX_COMPARISON_MEASUREMENT_PROFILE_VERSION = 'codex-task-structural-v1'; +export const CODEX_LOCAL_RECOMMENDATION_VERSION = 'codex-local-rule-v1'; + +export type ComparableCodexRecommendationId = + | 'recommendation-codex-multi-agent-share' + | 'recommendation-codex-effort-comparison' + | 'recommendation-codex-post-patch-tool-intensity' + | 'recommendation-codex-cache-context' + | 'recommendation-codex-approval-reviewer-share'; + +export interface CodexComparableTaskProjection { + observedAtEpochMs: number; + total: { + processed: number; + fresh: number; + }; + structural: { + patchCalls: number; + toolCalls: number; + postPatchToolCalls: number; + taskCompleteCount: number; + }; + childFreshShare: number; + approvalReviewerFreshShare: number; + modelFamilies: StoredModelFamily[]; + efforts: StoredEffort[]; + coverage: { + complete: boolean; + identityComplete: boolean; + qualityFlags: string[]; + }; +} + +export interface AppliedComparablePairsInput { + adviceId: string; + recommendationId: string; + recommendationVersion: string; + appliedAtEpochMs: number; + tasks: readonly CodexComparableTaskProjection[]; +} + +export type AppliedComparablePairsResult = + | { + ok: true; + status: 'comparable' | 'evidence-insufficient'; + pairs: StoredComparablePair[]; + } + | { ok: false; reason: 'invalid-input' }; + +const INPUT_KEYS = [ + 'adviceId', + 'appliedAtEpochMs', + 'recommendationId', + 'recommendationVersion', + 'tasks', +] as const; +const TASK_KEYS = [ + 'approvalReviewerFreshShare', + 'childFreshShare', + 'coverage', + 'efforts', + 'modelFamilies', + 'observedAtEpochMs', + 'structural', + 'total', +] as const; +const TOTAL_KEYS = ['fresh', 'processed'] as const; +const STRUCTURAL_KEYS = [ + 'patchCalls', + 'postPatchToolCalls', + 'taskCompleteCount', + 'toolCalls', +] as const; +const COVERAGE_KEYS = ['complete', 'identityComplete', 'qualityFlags'] as const; +const RECOMMENDATION_IDS = new Set([ + 'recommendation-codex-multi-agent-share', + 'recommendation-codex-effort-comparison', + 'recommendation-codex-post-patch-tool-intensity', + 'recommendation-codex-cache-context', + 'recommendation-codex-approval-reviewer-share', +]); + +function isObject(value: unknown): value is Record { + return typeof value === 'object' && value !== null && !Array.isArray(value); +} + +function hasExactKeys(value: Record, expected: readonly string[]): boolean { + const actual = Object.keys(value).sort(); + const keys = [...expected].sort(); + return actual.length === keys.length && actual.every((key, index) => key === keys[index]); +} + +function isEpochMs(value: unknown): value is number { + return typeof value === 'number' && Number.isInteger(value) && value >= 0; +} + +function isFiniteNonNegative(value: unknown): value is number { + return typeof value === 'number' && Number.isFinite(value) && value >= 0; +} + +function isCount(value: unknown): value is number { + return isFiniteNonNegative(value) && Number.isInteger(value); +} + +function isShare(value: unknown): value is number { + return isFiniteNonNegative(value) && value <= 1; +} + +function parseIdentifierList(value: unknown): string[] | undefined { + if (!Array.isArray(value) || value.length > 32) return undefined; + const result: string[] = []; + const seen = new Set(); + for (const item of value) { + if (typeof item !== 'string' || !isAdviceIdentifier(item) || seen.has(item)) return undefined; + seen.add(item); + result.push(item); + } + return result; +} + +function parseEnumList( + value: unknown, + allowed: readonly T[], + maximum: number, +): T[] | undefined { + if (!Array.isArray(value) || value.length > maximum) return undefined; + const result: T[] = []; + const seen = new Set(); + for (const item of value) { + if ( + typeof item !== 'string' || + !(allowed as readonly string[]).includes(item) || + seen.has(item) + ) { + return undefined; + } + seen.add(item); + result.push(item as T); + } + return result; +} + +function parseTask(value: unknown): CodexComparableTaskProjection | undefined { + if (!isObject(value) || !hasExactKeys(value, TASK_KEYS)) return undefined; + if ( + !isEpochMs(value.observedAtEpochMs) || + !isObject(value.total) || + !hasExactKeys(value.total, TOTAL_KEYS) || + !isObject(value.structural) || + !hasExactKeys(value.structural, STRUCTURAL_KEYS) || + !isObject(value.coverage) || + !hasExactKeys(value.coverage, COVERAGE_KEYS) || + !isFiniteNonNegative(value.total.processed) || + !isFiniteNonNegative(value.total.fresh) || + !isCount(value.structural.patchCalls) || + !isCount(value.structural.toolCalls) || + !isCount(value.structural.postPatchToolCalls) || + !isCount(value.structural.taskCompleteCount) || + !isShare(value.childFreshShare) || + !isShare(value.approvalReviewerFreshShare) || + typeof value.coverage.complete !== 'boolean' || + typeof value.coverage.identityComplete !== 'boolean' + ) { + return undefined; + } + const modelFamilies = parseEnumList( + value.modelFamilies, + ['opus', 'sonnet', 'haiku', 'fable', 'other'] as const, + 5, + ); + const efforts = parseEnumList( + value.efforts, + ['low', 'medium', 'high', 'xhigh', 'max', 'ultra', 'unknown'] as const, + 7, + ); + const qualityFlags = parseIdentifierList(value.coverage.qualityFlags); + if (!modelFamilies || !efforts || !qualityFlags) return undefined; + return { + observedAtEpochMs: value.observedAtEpochMs, + total: { processed: value.total.processed, fresh: value.total.fresh }, + structural: { + patchCalls: value.structural.patchCalls, + toolCalls: value.structural.toolCalls, + postPatchToolCalls: value.structural.postPatchToolCalls, + taskCompleteCount: value.structural.taskCompleteCount, + }, + childFreshShare: value.childFreshShare, + approvalReviewerFreshShare: value.approvalReviewerFreshShare, + modelFamilies, + efforts, + coverage: { + complete: value.coverage.complete, + identityComplete: value.coverage.identityComplete, + qualityFlags, + }, + }; +} + +export function isComparableCodexRecommendationId( + value: string, +): value is ComparableCodexRecommendationId { + return RECOMMENDATION_IDS.has(value as ComparableCodexRecommendationId); +} + +function complexityBand(task: CodexComparableTaskProjection): 'low' | 'medium' | 'high' { + if ( + task.structural.patchCalls === 1 && + task.structural.toolCalls <= 8 && + task.total.fresh <= 50_000 + ) { + return 'low'; + } + if ( + task.structural.patchCalls <= 2 && + task.structural.toolCalls <= 24 && + task.total.fresh <= 200_000 + ) { + return 'medium'; + } + return 'high'; +} + +function metric( + recommendationId: ComparableCodexRecommendationId, + task: CodexComparableTaskProjection, +): ComparableObservation['metric'] | undefined { + if (recommendationId === 'recommendation-codex-effort-comparison') { + return { + name: 'fresh-tokens', + unit: 'tokens', + direction: 'lower-is-better', + value: task.total.fresh, + }; + } + if (recommendationId === 'recommendation-codex-post-patch-tool-intensity') { + return { + name: 'post-patch-tool-intensity', + unit: 'multiple', + direction: 'lower-is-better', + value: task.structural.postPatchToolCalls / task.structural.patchCalls, + }; + } + if (recommendationId === 'recommendation-codex-multi-agent-share') { + return { + name: 'subagent-fresh-share', + unit: 'ratio', + direction: 'lower-is-better', + value: task.childFreshShare, + }; + } + if (recommendationId === 'recommendation-codex-approval-reviewer-share') { + return { + name: 'approval-reviewer-fresh-share', + unit: 'ratio', + direction: 'lower-is-better', + value: task.approvalReviewerFreshShare, + }; + } + if (task.total.fresh === 0) return undefined; + return { + name: 'processed-to-fresh-ratio', + unit: 'multiple', + direction: 'lower-is-better', + value: task.total.processed / task.total.fresh, + }; +} + +function observation( + recommendationId: ComparableCodexRecommendationId, + task: CodexComparableTaskProjection, +): ComparableObservation | undefined { + if ( + !task.coverage.complete || + !task.coverage.identityComplete || + task.coverage.qualityFlags.length > 0 || + task.structural.taskCompleteCount < 1 || + task.structural.patchCalls < 1 || + task.structural.patchCalls > 2 || + task.modelFamilies.length !== 1 || + task.efforts.length !== 1 || + task.efforts[0] === 'unknown' + ) { + return undefined; + } + const primaryMetric = metric(recommendationId, task); + if (!primaryMetric || !Number.isFinite(primaryMetric.value) || primaryMetric.value < 0) { + return undefined; + } + return { + context: { + scope: 'task-cohort', + taskKind: 'small-change', + complexityBand: complexityBand(task), + provider: 'codex', + modelFamily: task.modelFamilies[0], + effort: task.efforts[0], + measurementProfileVersion: CODEX_COMPARISON_MEASUREMENT_PROFILE_VERSION, + metricDefinitionVersion: `${primaryMetric.name}-v1`, + qualityRubricId: 'task-quality-rubric-v1', + }, + metric: primaryMetric, + quality: { + score: 1, + passed: 'passed', + evidenceCount: task.structural.taskCompleteCount, + }, + evidence: { coverage: 'complete', confidence: 'high', qualityFlags: [] }, + observedAtEpochMs: task.observedAtEpochMs, + }; +} + +function cohortKey(value: ComparableObservation): string { + return JSON.stringify({ + context: value.context, + metric: { + name: value.metric.name, + unit: value.metric.unit, + direction: value.metric.direction, + }, + }); +} + +/** + * Match already materialized, completed Codex task projections around one + * explicit applied event. No task key survives this boundary: observations are + * distinguished only by coarse context, numeric measurements, and timestamp. + */ +export function buildAppliedComparablePairs(input: unknown): AppliedComparablePairsResult { + if (!isObject(input) || !hasExactKeys(input, INPUT_KEYS)) { + return { ok: false, reason: 'invalid-input' }; + } + if ( + typeof input.adviceId !== 'string' || + typeof input.recommendationId !== 'string' || + typeof input.recommendationVersion !== 'string' || + !isAdviceIdentifier(input.adviceId) || + !isAdviceIdentifier(input.recommendationId) || + !isAdviceIdentifier(input.recommendationVersion) || + !isEpochMs(input.appliedAtEpochMs) || + !Array.isArray(input.tasks) || + input.tasks.length > 1_000 + ) { + return { ok: false, reason: 'invalid-input' }; + } + const tasks: CodexComparableTaskProjection[] = []; + for (const raw of input.tasks) { + const parsed = parseTask(raw); + if (!parsed) return { ok: false, reason: 'invalid-input' }; + tasks.push(parsed); + } + if (!isComparableCodexRecommendationId(input.recommendationId)) { + return { ok: true, status: 'evidence-insufficient', pairs: [] }; + } + + const seen = new Set(); + const observations = tasks + .map((task) => observation(input.recommendationId as ComparableCodexRecommendationId, task)) + .filter((item): item is ComparableObservation => item !== undefined) + .filter((item) => { + const key = JSON.stringify(item); + if (seen.has(key)) return false; + seen.add(key); + return true; + }); + const beforeByCohort = new Map(); + const afterByCohort = new Map(); + for (const item of observations) { + const target = item.observedAtEpochMs < input.appliedAtEpochMs + ? beforeByCohort + : item.observedAtEpochMs > input.appliedAtEpochMs + ? afterByCohort + : undefined; + if (!target) continue; + const key = cohortKey(item); + const cohort = target.get(key) ?? []; + cohort.push(item); + target.set(key, cohort); + } + + const pairs: StoredComparablePair[] = []; + for (const [key, after] of [...afterByCohort.entries()].sort(([left], [right]) => left.localeCompare(right))) { + const before = beforeByCohort.get(key) ?? []; + before.sort((left, right) => right.observedAtEpochMs - left.observedAtEpochMs); + after.sort((left, right) => left.observedAtEpochMs - right.observedAtEpochMs); + const count = Math.min(before.length, after.length); + for (let index = 0; index < count; index += 1) { + const paired = buildComparableTaskPair({ + adviceId: input.adviceId, + recommendationId: input.recommendationId, + recommendationVersion: input.recommendationVersion, + before: before[index], + after: after[index], + recordedAtEpochMs: after[index].observedAtEpochMs, + }); + if (paired.ok) pairs.push(paired.value); + } + } + pairs.sort((left, right) => left.pairId.localeCompare(right.pairId)); + return { + ok: true, + status: pairs.length > 0 ? 'comparable' : 'evidence-insufficient', + pairs, + }; +} diff --git a/src/adviceEffectiveness/comparisonResult.ts b/src/adviceEffectiveness/comparisonResult.ts new file mode 100644 index 00000000..09cd15cd --- /dev/null +++ b/src/adviceEffectiveness/comparisonResult.ts @@ -0,0 +1,615 @@ +import { createHash } from 'node:crypto'; + +import { + AdviceComparisonStats, + compareAdviceEffectiveness, +} from './comparison'; +import { isAdviceIdentifier } from './contract'; +import { + StoredAdviceScope, + StoredComparablePair, + StoredComplexityBand, + StoredEffort, + StoredMetricDirection, + StoredMetricUnit, + StoredModelFamily, + StoredProvider, + StoredTaskKind, + parseStoredComparablePair, + toComparableTaskPairs, +} from './comparisonPairing'; + +export const ADVICE_COMPARISON_RESULT_VERSION = 1 as const; +export const MAX_COMPARISON_RESULT_SAMPLE_PAIRS = 200; + +export interface AdviceComparisonGuardrail { + minComparablePairs: number; + minRelativeImprovement: number; + minAfterQualityScore: number; + maxMeanQualityRegression: number; + allowedQualityFlags: string[]; + qualityRubricId: string; +} + +export interface AdviceComparisonCohort { + scope: StoredAdviceScope; + taskKind: StoredTaskKind; + complexityBand: StoredComplexityBand; + modelFamily: StoredModelFamily; + effort: StoredEffort; + metricDefinitionVersion: string; + qualityRubricId: string; + metric: { + name: string; + unit: StoredMetricUnit; + direction: StoredMetricDirection; + }; +} + +export interface AdviceComparisonSample extends AdviceComparisonCohort { + pairIds: string[]; + pairCount: number; +} + +export type AdviceComparisonEvidenceReason = + | 'minimum-sample-not-met' + | 'incomparable-sample' + | 'unreliable-sample' + | 'comparison-inconclusive'; + +export type FrozenAdviceComparisonResult = + | { + status: 'evidence-insufficient'; + reasonCode: AdviceComparisonEvidenceReason; + comparablePairs: number; + } + | { + status: 'quality-guardrail-failed' | 'improved' | 'no-demonstrated-improvement'; + stats: AdviceComparisonStats; + }; + +export type AdviceComparisonReplayEligibility = + | { status: 'eligible'; reason: 'conclusive-result' } + | { status: 'ineligible'; reason: 'evidence-insufficient' }; + +export interface AdviceComparisonResultEnvelope { + schemaVersion: typeof ADVICE_COMPARISON_RESULT_VERSION; + comparisonId: string; + provider: StoredProvider; + recommendationId: string; + recommendationVersion: string; + measurementProfileVersion: string; + guardrail: AdviceComparisonGuardrail; + replayEligibility: AdviceComparisonReplayEligibility; + sample: AdviceComparisonSample; + result: FrozenAdviceComparisonResult; + recordedAtEpochMs: number; +} + +export interface AdviceComparisonResultBuildInput { + provider: StoredProvider; + recommendationId: string; + recommendationVersion: string; + measurementProfileVersion: string; + cohort: AdviceComparisonCohort; + guardrail: AdviceComparisonGuardrail; + pairs: readonly StoredComparablePair[]; + recordedAtEpochMs: number; +} + +export type AdviceComparisonResultBuildIssue = + | 'invalid-envelope-input' + | 'invalid-comparable-pair' + | 'duplicate-comparable-pair' + | 'comparison-precedes-sample'; + +export type AdviceComparisonResultBuildResult = + | { ok: true; value: AdviceComparisonResultEnvelope } + | { ok: false; reason: 'invalid-input'; issues: AdviceComparisonResultBuildIssue[] }; + +const BUILD_KEYS = [ + 'cohort', + 'guardrail', + 'measurementProfileVersion', + 'pairs', + 'provider', + 'recommendationId', + 'recommendationVersion', + 'recordedAtEpochMs', +] as const; +const ENVELOPE_KEYS = [ + 'comparisonId', + 'guardrail', + 'measurementProfileVersion', + 'provider', + 'recommendationId', + 'recommendationVersion', + 'recordedAtEpochMs', + 'replayEligibility', + 'result', + 'sample', + 'schemaVersion', +] as const; +const GUARDRAIL_KEYS = [ + 'allowedQualityFlags', + 'maxMeanQualityRegression', + 'minAfterQualityScore', + 'minComparablePairs', + 'minRelativeImprovement', + 'qualityRubricId', +] as const; +const COHORT_KEYS = [ + 'complexityBand', + 'effort', + 'metric', + 'metricDefinitionVersion', + 'modelFamily', + 'qualityRubricId', + 'scope', + 'taskKind', +] as const; +const SAMPLE_KEYS = [...COHORT_KEYS, 'pairCount', 'pairIds'] as const; +const METRIC_KEYS = ['direction', 'name', 'unit'] as const; +const REPLAY_KEYS = ['reason', 'status'] as const; +const INSUFFICIENT_RESULT_KEYS = ['comparablePairs', 'reasonCode', 'status'] as const; +const CONCLUSIVE_RESULT_KEYS = ['stats', 'status'] as const; +const STATS_KEYS = [ + 'comparablePairs', + 'meanAfterQuality', + 'meanAfterValue', + 'meanBeforeQuality', + 'meanBeforeValue', + 'meanQualityChange', + 'meanRelativeImprovement', +] as const; + +function isObject(value: unknown): value is Record { + return typeof value === 'object' && value !== null && !Array.isArray(value); +} + +function hasExactKeys(value: Record, expected: readonly string[]): boolean { + const actual = Object.keys(value).sort(); + const keys = [...expected].sort(); + return actual.length === keys.length && actual.every((key, index) => key === keys[index]); +} + +function oneOf(value: unknown, values: readonly T[]): value is T { + return typeof value === 'string' && (values as readonly string[]).includes(value); +} + +function isEpochMs(value: unknown): value is number { + return typeof value === 'number' && Number.isInteger(value) && value >= 0; +} + +function isFiniteNumber(value: unknown): value is number { + return typeof value === 'number' && Number.isFinite(value); +} + +function isFiniteNonNegative(value: unknown): value is number { + return isFiniteNumber(value) && value >= 0; +} + +function isScore(value: unknown): value is number { + return isFiniteNonNegative(value) && value <= 1; +} + +function parseIdentifierList(value: unknown, maximum: number): string[] | undefined { + if (!Array.isArray(value) || value.length > maximum) return undefined; + const result: string[] = []; + const seen = new Set(); + for (const item of value) { + if (typeof item !== 'string' || !isAdviceIdentifier(item) || seen.has(item)) return undefined; + seen.add(item); + result.push(item); + } + return result; +} + +function parseGuardrail(value: unknown): AdviceComparisonGuardrail | undefined { + if (!isObject(value) || !hasExactKeys(value, GUARDRAIL_KEYS)) return undefined; + if ( + !Number.isInteger(value.minComparablePairs) || + (value.minComparablePairs as number) < 2 || + !isFiniteNonNegative(value.minRelativeImprovement) || + !isScore(value.minAfterQualityScore) || + !isScore(value.maxMeanQualityRegression) || + typeof value.qualityRubricId !== 'string' || + !isAdviceIdentifier(value.qualityRubricId) + ) { + return undefined; + } + const allowedQualityFlags = parseIdentifierList(value.allowedQualityFlags, 32); + if (!allowedQualityFlags) return undefined; + return { + minComparablePairs: value.minComparablePairs as number, + minRelativeImprovement: value.minRelativeImprovement, + minAfterQualityScore: value.minAfterQualityScore, + maxMeanQualityRegression: value.maxMeanQualityRegression, + allowedQualityFlags, + qualityRubricId: value.qualityRubricId, + }; +} + +function parseMetric(value: unknown): AdviceComparisonCohort['metric'] | undefined { + if (!isObject(value) || !hasExactKeys(value, METRIC_KEYS)) return undefined; + if ( + typeof value.name !== 'string' || + !isAdviceIdentifier(value.name) || + !oneOf(value.unit, ['count', 'tokens', 'ratio', 'multiple', 'milliseconds'] as const) || + !oneOf(value.direction, ['lower-is-better', 'higher-is-better'] as const) + ) { + return undefined; + } + return { name: value.name, unit: value.unit, direction: value.direction }; +} + +function parseCohort(value: unknown): AdviceComparisonCohort | undefined { + if (!isObject(value) || !hasExactKeys(value, COHORT_KEYS)) return undefined; + const metric = parseMetric(value.metric); + if ( + !metric || + !oneOf(value.scope, ['overall', 'project', 'task-cohort'] as const) || + !oneOf(value.taskKind, ['small-change', 'feature', 'bug-fix', 'review', 'other'] as const) || + !oneOf(value.complexityBand, ['low', 'medium', 'high'] as const) || + !oneOf(value.modelFamily, ['opus', 'sonnet', 'haiku', 'fable', 'other'] as const) || + !oneOf(value.effort, ['low', 'medium', 'high', 'xhigh', 'max', 'ultra', 'unknown'] as const) || + typeof value.metricDefinitionVersion !== 'string' || + typeof value.qualityRubricId !== 'string' || + !isAdviceIdentifier(value.metricDefinitionVersion) || + !isAdviceIdentifier(value.qualityRubricId) + ) { + return undefined; + } + return { + scope: value.scope, + taskKind: value.taskKind, + complexityBand: value.complexityBand, + modelFamily: value.modelFamily, + effort: value.effort, + metricDefinitionVersion: value.metricDefinitionVersion, + qualityRubricId: value.qualityRubricId, + metric, + }; +} + +function parseSample(value: unknown): AdviceComparisonSample | undefined { + if (!isObject(value) || !hasExactKeys(value, SAMPLE_KEYS)) return undefined; + const cohortCandidate: Record = {}; + for (const key of COHORT_KEYS) cohortCandidate[key] = value[key]; + const cohort = parseCohort(cohortCandidate); + const pairIds = parseIdentifierList(value.pairIds, MAX_COMPARISON_RESULT_SAMPLE_PAIRS); + if ( + !cohort || + !pairIds || + !Number.isInteger(value.pairCount) || + (value.pairCount as number) < 0 || + value.pairCount !== pairIds.length + ) { + return undefined; + } + return { ...cohort, pairIds, pairCount: value.pairCount as number }; +} + +function parseStats(value: unknown): AdviceComparisonStats | undefined { + if (!isObject(value) || !hasExactKeys(value, STATS_KEYS)) return undefined; + if ( + !Number.isInteger(value.comparablePairs) || + (value.comparablePairs as number) < 0 || + !isFiniteNonNegative(value.meanBeforeValue) || + !isFiniteNonNegative(value.meanAfterValue) || + !isFiniteNumber(value.meanRelativeImprovement) || + !isScore(value.meanBeforeQuality) || + !isScore(value.meanAfterQuality) || + !isFiniteNumber(value.meanQualityChange) || + value.meanQualityChange < -1 || + value.meanQualityChange > 1 + ) { + return undefined; + } + return { + comparablePairs: value.comparablePairs as number, + meanBeforeValue: value.meanBeforeValue, + meanAfterValue: value.meanAfterValue, + meanRelativeImprovement: value.meanRelativeImprovement, + meanBeforeQuality: value.meanBeforeQuality, + meanAfterQuality: value.meanAfterQuality, + meanQualityChange: value.meanQualityChange, + }; +} + +function parseResult(value: unknown): FrozenAdviceComparisonResult | undefined { + if (!isObject(value) || typeof value.status !== 'string') return undefined; + if (value.status === 'evidence-insufficient') { + if ( + !hasExactKeys(value, INSUFFICIENT_RESULT_KEYS) || + !oneOf(value.reasonCode, [ + 'minimum-sample-not-met', + 'incomparable-sample', + 'unreliable-sample', + 'comparison-inconclusive', + ] as const) || + !Number.isInteger(value.comparablePairs) || + (value.comparablePairs as number) < 0 + ) { + return undefined; + } + return { + status: value.status, + reasonCode: value.reasonCode, + comparablePairs: value.comparablePairs as number, + }; + } + if ( + !oneOf(value.status, [ + 'quality-guardrail-failed', + 'improved', + 'no-demonstrated-improvement', + ] as const) || + !hasExactKeys(value, CONCLUSIVE_RESULT_KEYS) + ) { + return undefined; + } + const stats = parseStats(value.stats); + if (!stats) return undefined; + return { status: value.status, stats }; +} + +function parseReplayEligibility(value: unknown): AdviceComparisonReplayEligibility | undefined { + if (!isObject(value) || !hasExactKeys(value, REPLAY_KEYS)) return undefined; + if (value.status === 'eligible' && value.reason === 'conclusive-result') { + return { status: value.status, reason: value.reason }; + } + if (value.status === 'ineligible' && value.reason === 'evidence-insufficient') { + return { status: value.status, reason: value.reason }; + } + return undefined; +} + +function canonicalJson(value: unknown): string { + const canonicalize = (current: unknown): unknown => { + if (Array.isArray(current)) return current.map(canonicalize); + if (isObject(current)) { + const result: Record = {}; + for (const key of Object.keys(current).sort()) result[key] = canonicalize(current[key]); + return result; + } + return current; + }; + return JSON.stringify(canonicalize(value)); +} + +function comparisonId(value: Omit): string { + const digest = createHash('sha256').update(canonicalJson(value), 'utf8').digest('hex').slice(0, 32); + return `comparison-${digest}`; +} + +/** Parse, clone, and verify the entire frozen envelope, including its digest. */ +export function parseAdviceComparisonResultEnvelope( + value: unknown, +): AdviceComparisonResultEnvelope | undefined { + if (!isObject(value) || !hasExactKeys(value, ENVELOPE_KEYS)) return undefined; + if ( + value.schemaVersion !== ADVICE_COMPARISON_RESULT_VERSION || + typeof value.comparisonId !== 'string' || + typeof value.recommendationId !== 'string' || + typeof value.recommendationVersion !== 'string' || + typeof value.measurementProfileVersion !== 'string' || + !isAdviceIdentifier(value.comparisonId) || + !oneOf(value.provider, ['claude', 'codex'] as const) || + !isAdviceIdentifier(value.recommendationId) || + !isAdviceIdentifier(value.recommendationVersion) || + !isAdviceIdentifier(value.measurementProfileVersion) || + !isEpochMs(value.recordedAtEpochMs) + ) { + return undefined; + } + const guardrail = parseGuardrail(value.guardrail); + const sample = parseSample(value.sample); + const result = parseResult(value.result); + const replayEligibility = parseReplayEligibility(value.replayEligibility); + if ( + !guardrail || + !sample || + !result || + !replayEligibility || + guardrail.qualityRubricId !== sample.qualityRubricId || + (result.status === 'evidence-insufficient' + ? replayEligibility.status !== 'ineligible' || result.comparablePairs !== sample.pairCount + : replayEligibility.status !== 'eligible' || result.stats.comparablePairs !== sample.pairCount) + ) { + return undefined; + } + const withoutId: Omit = { + schemaVersion: ADVICE_COMPARISON_RESULT_VERSION, + provider: value.provider, + recommendationId: value.recommendationId, + recommendationVersion: value.recommendationVersion, + measurementProfileVersion: value.measurementProfileVersion, + guardrail, + replayEligibility, + sample, + result, + recordedAtEpochMs: value.recordedAtEpochMs, + }; + if (value.comparisonId !== comparisonId(withoutId)) return undefined; + return { comparisonId: value.comparisonId, ...withoutId }; +} + +function pairMatchesCohort( + pair: StoredComparablePair, + input: { + provider: StoredProvider; + recommendationId: string; + recommendationVersion: string; + measurementProfileVersion: string; + cohort: AdviceComparisonCohort; + }, +): boolean { + return ( + pair.context.provider === input.provider && + pair.recommendationId === input.recommendationId && + pair.recommendationVersion === input.recommendationVersion && + pair.context.measurementProfileVersion === input.measurementProfileVersion && + pair.context.scope === input.cohort.scope && + pair.context.taskKind === input.cohort.taskKind && + pair.context.complexityBand === input.cohort.complexityBand && + pair.context.modelFamily === input.cohort.modelFamily && + pair.context.effort === input.cohort.effort && + pair.context.metricDefinitionVersion === input.cohort.metricDefinitionVersion && + pair.context.qualityRubricId === input.cohort.qualityRubricId && + pair.metric.name === input.cohort.metric.name && + pair.metric.unit === input.cohort.metric.unit && + pair.metric.direction === input.cohort.metric.direction + ); +} + +function pairIsReliable(pair: StoredComparablePair, allowedFlags: ReadonlySet): boolean { + return ( + pair.evidence.beforeCoverage === 'complete' && + pair.evidence.afterCoverage === 'complete' && + pair.evidence.beforeConfidence !== 'low' && + pair.evidence.beforeConfidence !== 'unknown' && + pair.evidence.afterConfidence !== 'low' && + pair.evidence.afterConfidence !== 'unknown' && + pair.evidence.beforeQualityFlags.every((flag) => allowedFlags.has(flag)) && + pair.evidence.afterQualityFlags.every((flag) => allowedFlags.has(flag)) && + pair.quality.beforePassed === 'passed' && + pair.quality.afterPassed !== 'unknown' && + pair.quality.beforeEvidenceCount > 0 && + pair.quality.afterEvidenceCount > 0 && + pair.context.effort !== 'unknown' && + pair.metric.beforeValue > 0 + ); +} + +/** + * Freeze a comparison result from one strict recommendation/cohort lineage. + * The envelope stores only coarse cohort fields, aggregate statistics, and + * opaque pair IDs; comparator prose is deliberately not persisted. + */ +export function buildAdviceComparisonResultEnvelope( + input: unknown, +): AdviceComparisonResultBuildResult { + if (!isObject(input) || !hasExactKeys(input, BUILD_KEYS)) { + return { ok: false, reason: 'invalid-input', issues: ['invalid-envelope-input'] }; + } + const cohort = parseCohort(input.cohort); + const guardrail = parseGuardrail(input.guardrail); + if ( + !oneOf(input.provider, ['claude', 'codex'] as const) || + typeof input.recommendationId !== 'string' || + typeof input.recommendationVersion !== 'string' || + typeof input.measurementProfileVersion !== 'string' || + !isAdviceIdentifier(input.recommendationId) || + !isAdviceIdentifier(input.recommendationVersion) || + !isAdviceIdentifier(input.measurementProfileVersion) || + !cohort || + !guardrail || + cohort.qualityRubricId !== guardrail.qualityRubricId || + !Array.isArray(input.pairs) || + input.pairs.length > MAX_COMPARISON_RESULT_SAMPLE_PAIRS || + !isEpochMs(input.recordedAtEpochMs) + ) { + return { ok: false, reason: 'invalid-input', issues: ['invalid-envelope-input'] }; + } + const provider = input.provider; + const recommendationId = input.recommendationId; + const recommendationVersion = input.recommendationVersion; + const measurementProfileVersion = input.measurementProfileVersion; + const recordedAtEpochMs = input.recordedAtEpochMs; + + const pairs: StoredComparablePair[] = []; + const pairIds = new Set(); + for (const raw of input.pairs) { + const parsed = parseStoredComparablePair(raw); + if (!parsed) { + return { ok: false, reason: 'invalid-input', issues: ['invalid-comparable-pair'] }; + } + if (pairIds.has(parsed.pairId)) { + return { ok: false, reason: 'invalid-input', issues: ['duplicate-comparable-pair'] }; + } + if (parsed.recordedAtEpochMs > recordedAtEpochMs) { + return { ok: false, reason: 'invalid-input', issues: ['comparison-precedes-sample'] }; + } + pairIds.add(parsed.pairId); + pairs.push(parsed); + } + pairs.sort((left, right) => left.pairId.localeCompare(right.pairId)); + + const sample: AdviceComparisonSample = { + ...cohort, + pairIds: pairs.map((pair) => pair.pairId), + pairCount: pairs.length, + }; + let result: FrozenAdviceComparisonResult; + if (pairs.length < guardrail.minComparablePairs) { + result = { + status: 'evidence-insufficient', + reasonCode: 'minimum-sample-not-met', + comparablePairs: pairs.length, + }; + } else if ( + pairs.some((pair) => + !pairMatchesCohort(pair, { + provider, + recommendationId, + recommendationVersion, + measurementProfileVersion, + cohort, + }), + ) + ) { + result = { + status: 'evidence-insufficient', + reasonCode: 'incomparable-sample', + comparablePairs: pairs.length, + }; + } else if ( + pairs.some((pair) => !pairIsReliable(pair, new Set(guardrail.allowedQualityFlags))) + ) { + result = { + status: 'evidence-insufficient', + reasonCode: 'unreliable-sample', + comparablePairs: pairs.length, + }; + } else { + const compared = compareAdviceEffectiveness(toComparableTaskPairs(pairs), { + minComparablePairs: guardrail.minComparablePairs, + minRelativeImprovement: guardrail.minRelativeImprovement, + minAfterQualityScore: guardrail.minAfterQualityScore, + maxMeanQualityRegression: guardrail.maxMeanQualityRegression, + allowedQualityFlags: [...guardrail.allowedQualityFlags], + }); + result = compared.status === 'insufficient-evidence' + ? { + status: 'evidence-insufficient', + reasonCode: 'comparison-inconclusive', + comparablePairs: pairs.length, + } + : { status: compared.status, stats: { ...compared.stats } }; + } + const replayEligibility: AdviceComparisonReplayEligibility = + result.status === 'evidence-insufficient' + ? { status: 'ineligible', reason: 'evidence-insufficient' } + : { status: 'eligible', reason: 'conclusive-result' }; + const withoutId: Omit = { + schemaVersion: ADVICE_COMPARISON_RESULT_VERSION, + provider, + recommendationId, + recommendationVersion, + measurementProfileVersion, + guardrail, + replayEligibility, + sample, + result, + recordedAtEpochMs, + }; + const envelope: AdviceComparisonResultEnvelope = { + comparisonId: comparisonId(withoutId), + ...withoutId, + }; + const parsed = parseAdviceComparisonResultEnvelope(envelope); + return parsed + ? { ok: true, value: parsed } + : { ok: false, reason: 'invalid-input', issues: ['invalid-envelope-input'] }; +} diff --git a/src/adviceEffectiveness/contract.ts b/src/adviceEffectiveness/contract.ts index 2b8123f9..58b78620 100644 --- a/src/adviceEffectiveness/contract.ts +++ b/src/adviceEffectiveness/contract.ts @@ -106,7 +106,11 @@ export interface AdviceRecommendation { } export interface AdvicePrivacyInfo { - dataMode: 'local-only' | 'aggregates-only' | 'aggregates-with-prompt-samples'; + dataMode: + | 'local-only' + | 'aggregates-only' + | 'aggregates-with-personalization' + | 'aggregates-with-prompt-samples'; promptSampleConsent: 'not-applicable' | 'not-granted' | 'explicit'; promptSampleCount: number; feedbackStorage: 'local-only'; @@ -333,6 +337,12 @@ export function validateAdviceContract(contract: AdviceContract): string[] { ) { issues.push('prompt samples require explicit consent and a non-zero sample count'); } + if ( + privacy.dataMode === 'aggregates-with-personalization' && + (privacy.promptSampleConsent !== 'explicit' || privacy.promptSampleCount !== 0) + ) { + issues.push('personal context requires explicit consent and zero prompt samples'); + } if (!nonEmpty(contract.provenance.locale)) issues.push('provenance locale is required'); if (!Number.isFinite(Date.parse(contract.provenance.generatedAt))) { diff --git a/src/adviceEffectiveness/integration.ts b/src/adviceEffectiveness/integration.ts index 3dc94413..5ae0a6af 100644 --- a/src/adviceEffectiveness/integration.ts +++ b/src/adviceEffectiveness/integration.ts @@ -18,6 +18,8 @@ export interface AdviceEffectivenessProviderState { contract: AdviceContract; remotePreviewEligible: boolean; aggregate?: AdviceAggregateSnapshot; + /** Included only when the separate prompt-personalisation consent is explicit. */ + userContext?: string; promptSamples: readonly { text: string }[]; } @@ -30,6 +32,32 @@ export type AdviceSnapshotConsent = { promptSamples: 'explicit' | 'not-granted'; }; +/** Select only timestamped samples inside the current consent window. */ +export function selectAdvicePromptSamples( + samples: readonly { text: string; observedAtEpochMs: number }[], + nowEpochMs: number, + windowDays: number, +): { text: string }[] { + if ( + !Number.isFinite(nowEpochMs) || + !Number.isInteger(windowDays) || + windowDays < 1 || + windowDays > 365 + ) { + return []; + } + const cutoff = nowEpochMs - windowDays * 86_400_000; + return samples + .filter( + (sample) => + typeof sample.text === 'string' && + Number.isFinite(sample.observedAtEpochMs) && + sample.observedAtEpochMs >= cutoff && + sample.observedAtEpochMs <= nowEpochMs, + ) + .map((sample) => ({ text: sample.text })); +} + export interface PreparedAdviceSnapshot { provider: 'claude'; prepared: PreparedAdvicePayload; @@ -84,6 +112,7 @@ export function prepareAdviceSnapshot( ? { promptSamples: { consent: 'explicit' as const, + ...(state.userContext === undefined ? {} : { userContext: state.userContext }), samples: state.promptSamples.map((sample) => ({ text: sample.text })), }, } diff --git a/src/adviceEffectiveness/legacyBridge.ts b/src/adviceEffectiveness/legacyBridge.ts index dc67bbd6..259ae886 100644 --- a/src/adviceEffectiveness/legacyBridge.ts +++ b/src/adviceEffectiveness/legacyBridge.ts @@ -1,7 +1,3 @@ -import { - AdviceOptions, - callModel, -} from '../advisor'; import { PreparedAdvicePayload, sendPreparedAdvicePayload, @@ -33,10 +29,16 @@ export interface LegacyByokAdviceOptions { timeoutMs?: number; } +export interface LegacyByokCallOptions extends LegacyByokAdviceOptions { + backend: 'api'; + summary: ''; + language: ''; +} + export type LegacyByokModelCall = ( systemPrompt: string, userContent: string, - options: AdviceOptions, + options: LegacyByokCallOptions, ) => Promise; export type LegacyByokAdviceResult = @@ -55,7 +57,7 @@ export async function requestStructuredAdviceViaLegacyByok( prepared: PreparedAdvicePayload, references: StructuredAdviceReferences, options: LegacyByokAdviceOptions, - invoke: LegacyByokModelCall = callModel, + invoke: LegacyByokModelCall, ): Promise { if (!validLegacyByokOptions(options)) { return { diff --git a/src/adviceEffectiveness/payload.ts b/src/adviceEffectiveness/payload.ts index f07607ae..eebafcb5 100644 --- a/src/adviceEffectiveness/payload.ts +++ b/src/adviceEffectiveness/payload.ts @@ -12,6 +12,7 @@ export const ADVICE_REMOTE_PAYLOAD_VERSION = 1 as const; export const MAX_PROMPT_SAMPLES = 20; export const MAX_PROMPT_SAMPLE_CHARS = 1_000; export const MAX_PROMPT_SAMPLE_TOTAL_CHARS = 12_000; +export const MAX_USER_CONTEXT_CHARS = 1_000; /** Adding a metric here is an explicit remote-data/privacy review point. */ export const REMOTE_ADVICE_METRIC_ALLOWLIST = [ @@ -63,6 +64,8 @@ export interface AdviceAggregateSnapshot { export interface PromptSampleOptIn { consent: 'explicit'; + /** Configured context shares the same separate prompt-personalisation consent. */ + userContext?: string; /** Only text is accepted. Extra runtime fields such as cwd are discarded. */ samples: readonly { text: string }[]; } @@ -78,11 +81,14 @@ export interface PrepareAdvicePayloadInput { export interface PreparedAdvicePayload { contentType: 'application/json'; - dataMode: 'aggregates-only' | 'aggregates-with-prompt-samples'; + dataMode: + | 'aggregates-only' + | 'aggregates-with-personalization' + | 'aggregates-with-prompt-samples'; promptSampleCount: number; /** The one canonical serialization used by both preview and transport. */ serializedBody: string; - /** Exact UTF-8 bytes derived once from serializedBody. Future senders use these bytes directly. */ + /** Exact UTF-8 bytes derived once from serializedBody. The sender uses these bytes directly. */ canonicalBytes: Uint8Array; /** SHA-256 of canonicalBytes, lowercase hexadecimal. */ sha256: string; @@ -135,7 +141,9 @@ interface RemoteAdviceRequest { dataMode: PreparedAdvicePayload['dataMode']; promptSampleConsent: 'not-granted' | 'explicit'; promptSampleCount: number; + userContextIncluded: boolean; }; + userContext?: string; promptSamples?: { id: string; text: string }[]; } @@ -321,10 +329,18 @@ function promptSamples(optIn: PromptSampleOptIn | undefined): { id: string; text out.push({ id: `prompt-${out.length + 1}`, text: bounded }); totalChars += bounded.length; } - if (out.length === 0) throw new Error('explicit prompt opt-in must contain at least one non-empty sample'); return out; } +function userContext(optIn: PromptSampleOptIn | undefined): string | undefined { + if (optIn === undefined || optIn.userContext === undefined) return undefined; + if (optIn.consent !== 'explicit' || typeof optIn.userContext !== 'string') { + throw new Error('user context requires the separate prompt-personalisation opt-in'); + } + const text = optIn.userContext.trim().slice(0, MAX_USER_CONTEXT_CHARS); + return text.length > 0 ? text : undefined; +} + function canonicalize(value: unknown): unknown { if (value === null || typeof value === 'string' || typeof value === 'boolean') return value; if (typeof value === 'number') { @@ -353,8 +369,16 @@ export function prepareAdvicePayload(input: PrepareAdvicePayloadInput): Prepared validateAdviceAggregateSnapshot(input.aggregate); validateSignalGraph(input); const samples = promptSamples(input.promptSamples); + const context = userContext(input.promptSamples); + if (input.promptSamples !== undefined && samples.length === 0 && context === undefined) { + throw new Error('explicit prompt opt-in must contain a sample or user context'); + } const dataMode: PreparedAdvicePayload['dataMode'] = - samples.length > 0 ? 'aggregates-with-prompt-samples' : 'aggregates-only'; + samples.length > 0 + ? 'aggregates-with-prompt-samples' + : context !== undefined + ? 'aggregates-with-personalization' + : 'aggregates-only'; const request: RemoteAdviceRequest = { schemaVersion: ADVICE_REMOTE_PAYLOAD_VERSION, @@ -414,10 +438,12 @@ export function prepareAdvicePayload(input: PrepareAdvicePayloadInput): Prepared }, privacy: { dataMode, - promptSampleConsent: samples.length > 0 ? 'explicit' : 'not-granted', + promptSampleConsent: input.promptSamples !== undefined ? 'explicit' : 'not-granted', promptSampleCount: samples.length, + userContextIncluded: context !== undefined, }, }; + if (context !== undefined) request.userContext = context; if (samples.length > 0) request.promptSamples = samples; const serializedBody = stableAdvicePayloadStringify(request); @@ -444,7 +470,7 @@ function assertPreparedAdvicePayloadIntegrity(prepared: PreparedAdvicePayload): } } -/** Preview decodes the same canonical bytes that a future sender receives. */ +/** Preview decodes the same canonical bytes that the explicit sender receives. */ export function previewAdvicePayload(prepared: PreparedAdvicePayload): AdvicePayloadPreview { assertPreparedAdvicePayloadIntegrity(prepared); return { diff --git a/src/adviceEffectiveness/preparedRequest.ts b/src/adviceEffectiveness/preparedRequest.ts new file mode 100644 index 00000000..cf7bcc7c --- /dev/null +++ b/src/adviceEffectiveness/preparedRequest.ts @@ -0,0 +1,255 @@ +import { createHash } from 'node:crypto'; + +import type { AdviceFormat } from '../advisor'; +import { stableAdvicePayloadStringify } from './payload'; + +const AI_INVOCATION_SCHEMA_VERSION = 1 as const; +const AI_MAX_TOKENS = 16_000; +const DEFAULT_TIMEOUT_MS = 120_000; + +export type AiInvocationKind = 'advice' | 'optimizer'; +export type AiInvocationDataMode = + | 'aggregates-only' + | 'aggregates-with-personalization' + | 'aggregates-with-prompt-samples' + | 'user-draft-only'; + +export interface PrepareAiInvocationInput { + kind: AiInvocationKind; + apiFormat: AdviceFormat; + apiUrl: string; + model: string; + reasoningEffort?: string; + systemPrompt: string; + userContent: string; + dataMode: AiInvocationDataMode; + /** Opaque provider-index or draft revision; never a path/session/title. */ + sourceRevision: string; + consentGeneration: number; + createdAtEpochMs: number; + timeoutMs?: number; +} + +export interface PreparedAiInvocation { + readonly schemaVersion: typeof AI_INVOCATION_SCHEMA_VERSION; + readonly kind: AiInvocationKind; + readonly apiFormat: AdviceFormat; + readonly endpoint: string; + readonly model: string; + readonly dataMode: AiInvocationDataMode; + readonly sourceRevision: string; + readonly consentGeneration: number; + readonly createdAtEpochMs: number; + readonly timeoutMs: number; + readonly contentType: 'application/json'; + /** Canonical full provider HTTP body. It is never rebuilt after preparation. */ + readonly serializedBody: string; + readonly canonicalBytes: Uint8Array; + readonly sha256: string; +} + +export interface AiInvocationPreview { + kind: AiInvocationKind; + dataMode: AiInvocationDataMode; + contentType: 'application/json'; + body: string; + utf8Bytes: number; + sha256: string; +} + +export interface PreparedAiSendAuthorization { + backend: 'api'; + apiKey: string; + expectedSourceRevision?: string; + expectedConsentGeneration?: number; + signal?: AbortSignal; +} + +export interface PreparedAiTransportRequest { + endpoint: string; + apiFormat: AdviceFormat; + contentType: 'application/json'; + headers: Record; + /** Same object identity as PreparedAiInvocation.canonicalBytes. */ + canonicalBytes: Uint8Array; + timeoutMs: number; + signal?: AbortSignal; +} + +export type PreparedAiTransport = (request: PreparedAiTransportRequest) => Promise; + +function normalizeOpenAiUrl(value: string): string { + let url = value.trim().replace(/\/+$/, ''); + if (url === '') return 'https://api.deepseek.com/chat/completions'; + if (/api\.deepseek\.com\/v1(?:\/chat\/completions)?$/.test(url)) { + url = url.replace('/v1', ''); + } + return url.endsWith('/chat/completions') ? url : `${url}/chat/completions`; +} + +function normalizeAnthropicUrl(value: string): string { + const url = value.trim().replace(/\/+$/, ''); + if (url === '' || /api\.anthropic\.com$/.test(url) || /chat\/completions$/.test(url)) { + return 'https://api.anthropic.com/v1/messages'; + } + return url.endsWith('/v1/messages') ? url : `${url}/v1/messages`; +} + +function requireBoundedText(value: string, label: string, maxChars: number): string { + if (typeof value !== 'string') throw new Error(`${label} must be a string`); + const text = value.trim(); + if (text.length === 0 || text.length > maxChars) { + throw new Error(`${label} must contain 1-${maxChars} characters`); + } + return value; +} + +function requireOpaqueRevision(value: string): string { + if (!/^[A-Za-z0-9._:-]{1,128}$/.test(value)) { + throw new Error('sourceRevision must be an opaque revision'); + } + return value; +} + +function requestBody(input: PrepareAiInvocationInput): Record { + if (input.apiFormat === 'anthropic') { + return { + model: requireBoundedText(input.model, 'model', 256), + max_tokens: AI_MAX_TOKENS, + system: requireBoundedText(input.systemPrompt, 'systemPrompt', 40_000), + messages: [{ role: 'user', content: requireBoundedText(input.userContent, 'userContent', 200_000) }], + }; + } + const body: Record = { + model: requireBoundedText(input.model, 'model', 256), + stream: false, + messages: [ + { role: 'system', content: requireBoundedText(input.systemPrompt, 'systemPrompt', 40_000) }, + { role: 'user', content: requireBoundedText(input.userContent, 'userContent', 200_000) }, + ], + }; + const reasoningEffort = input.reasoningEffort?.trim(); + if (reasoningEffort) { + if (!/^[A-Za-z0-9_-]{1,32}$/.test(reasoningEffort)) { + throw new Error('reasoningEffort is invalid'); + } + body.reasoning_effort = reasoningEffort; + body.thinking = { type: 'enabled' }; + } + return body; +} + +/** Prepare the complete provider wire body exactly once. No credential enters it. */ +export function prepareAiInvocation(input: PrepareAiInvocationInput): PreparedAiInvocation { + if (input.kind !== 'advice' && input.kind !== 'optimizer') { + throw new Error('AI invocation kind is invalid'); + } + if (input.apiFormat !== 'anthropic' && input.apiFormat !== 'openai') { + throw new Error('AI invocation format is invalid'); + } + if (!Number.isInteger(input.consentGeneration) || input.consentGeneration < 0) { + throw new Error('consentGeneration must be a non-negative integer'); + } + if (!Number.isFinite(input.createdAtEpochMs) || input.createdAtEpochMs < 0) { + throw new Error('createdAtEpochMs must be finite and non-negative'); + } + const timeoutMs = input.timeoutMs ?? DEFAULT_TIMEOUT_MS; + if (!Number.isFinite(timeoutMs) || timeoutMs < 1_000 || timeoutMs > 10 * 60_000) { + throw new Error('timeoutMs is outside the supported range'); + } + const serializedBody = stableAdvicePayloadStringify(requestBody(input)); + const canonicalBytes = Buffer.from(serializedBody, 'utf8'); + const sha256 = createHash('sha256').update(canonicalBytes).digest('hex'); + return { + schemaVersion: AI_INVOCATION_SCHEMA_VERSION, + kind: input.kind, + apiFormat: input.apiFormat, + endpoint: input.apiFormat === 'anthropic' + ? normalizeAnthropicUrl(input.apiUrl) + : normalizeOpenAiUrl(input.apiUrl), + model: input.model.trim(), + dataMode: input.dataMode, + sourceRevision: requireOpaqueRevision(input.sourceRevision), + consentGeneration: input.consentGeneration, + createdAtEpochMs: input.createdAtEpochMs, + timeoutMs, + contentType: 'application/json', + serializedBody, + canonicalBytes, + sha256, + }; +} + +export function assertPreparedAiInvocationIntegrity(prepared: PreparedAiInvocation): void { + if ( + prepared.schemaVersion !== AI_INVOCATION_SCHEMA_VERSION || + !(prepared.canonicalBytes instanceof Uint8Array) || + prepared.contentType !== 'application/json' + ) { + throw new Error('prepared AI invocation integrity check failed'); + } + const decoded = Buffer.from(prepared.canonicalBytes).toString('utf8'); + const digest = createHash('sha256').update(prepared.canonicalBytes).digest('hex'); + if (decoded !== prepared.serializedBody || digest !== prepared.sha256) { + throw new Error('prepared AI invocation integrity check failed'); + } +} + +export function previewAiInvocation(prepared: PreparedAiInvocation): AiInvocationPreview { + assertPreparedAiInvocationIntegrity(prepared); + return { + kind: prepared.kind, + dataMode: prepared.dataMode, + contentType: prepared.contentType, + body: Buffer.from(prepared.canonicalBytes).toString('utf8'), + utf8Bytes: prepared.canonicalBytes.byteLength, + sha256: prepared.sha256, + }; +} + +/** + * The only send boundary. It validates current consent/source generation and + * gives transport the original canonical byte object; it cannot reserialize. + */ +export async function sendPreparedAiInvocation( + prepared: PreparedAiInvocation, + authorization: PreparedAiSendAuthorization, + transport: PreparedAiTransport, +): Promise { + assertPreparedAiInvocationIntegrity(prepared); + if (!authorization || authorization.backend !== 'api') { + throw new Error('Only the configured BYOK API backend may send AI requests'); + } + const apiKey = authorization.apiKey?.trim(); + if (!apiKey) throw new Error('A user-configured API key is required'); + if ( + (authorization.expectedSourceRevision !== undefined && + authorization.expectedSourceRevision !== prepared.sourceRevision) || + (authorization.expectedConsentGeneration !== undefined && + authorization.expectedConsentGeneration !== prepared.consentGeneration) + ) { + throw new Error('The prepared AI invocation is stale'); + } + if (authorization.signal?.aborted) { + throw new Error('The prepared AI invocation was cancelled'); + } + const headers: Record = prepared.apiFormat === 'anthropic' + ? { + 'Content-Type': prepared.contentType, + 'x-api-key': apiKey, + 'anthropic-version': '2023-06-01', + } + : { + 'Content-Type': prepared.contentType, + Authorization: `Bearer ${apiKey}`, + }; + return transport({ + endpoint: prepared.endpoint, + apiFormat: prepared.apiFormat, + contentType: prepared.contentType, + headers, + canonicalBytes: prepared.canonicalBytes, + timeoutMs: prepared.timeoutMs, + ...(authorization.signal ? { signal: authorization.signal } : {}), + }); +} diff --git a/src/adviceEffectiveness/remoteAdvice.ts b/src/adviceEffectiveness/remoteAdvice.ts new file mode 100644 index 00000000..9271fbd4 --- /dev/null +++ b/src/adviceEffectiveness/remoteAdvice.ts @@ -0,0 +1,113 @@ +import type { + PreparedAiTransportRequest, + PreparedAiInvocation, +} from './preparedRequest'; +import { prepareAiInvocation } from './preparedRequest'; +import type { PreparedAdvicePayload } from './payload'; +import { previewAdvicePayload } from './payload'; +import { parseStructuredAdviceOutput } from './structuredOutput'; +import type { + StructuredAdviceOutput, + StructuredAdviceParseErrorCode, + StructuredAdviceReferences, +} from './structuredOutput'; +import type { HttpResponse } from '../httpClient'; +import { sendPreparedModelRequest } from '../advisor'; + +export const STRUCTURED_ADVICE_SYSTEM_PROMPT = + 'Return exactly one strict JSON object matching the response contract in the user payload. ' + + 'Do not use Markdown fences, prose outside JSON, unknown fields, or evidence IDs absent ' + + 'from that payload. Return an empty recommendations array when the evidence is insufficient.'; + +export interface StructuredAdviceInvocationConfig { + apiFormat: 'anthropic' | 'openai'; + apiUrl: string; + model: string; + reasoningEffort?: string; + sourceRevision: string; + consentGeneration: number; + createdAtEpochMs: number; + timeoutMs?: number; +} + +export interface StructuredAdviceSendOptions { + apiKey: string; + expectedSourceRevision?: string; + expectedConsentGeneration?: number; + signal?: AbortSignal; + transport?: (request: PreparedAiTransportRequest) => Promise; +} + +export type StructuredAdviceRequestResult = + | { ok: true; value: StructuredAdviceOutput } + | { + ok: false; + code: StructuredAdviceParseErrorCode | 'transport-error' | 'invalid-prepared-request'; + issues: string[]; + }; + +/** + * Wrap the sealed evidence body in the provider's complete HTTP body exactly + * once. The returned object, not the evidence-only inner payload, is what both + * preview and transport consume. + */ +export function prepareStructuredAdviceInvocation( + evidence: PreparedAdvicePayload, + config: StructuredAdviceInvocationConfig, +): PreparedAiInvocation { + // This assertion catches any mismatch between the inner evidence preview and + // its bytes before those exact bytes are embedded in the provider body. + previewAdvicePayload(evidence); + return prepareAiInvocation({ + kind: 'advice', + apiFormat: config.apiFormat, + apiUrl: config.apiUrl, + model: config.model, + ...(config.reasoningEffort ? { reasoningEffort: config.reasoningEffort } : {}), + systemPrompt: STRUCTURED_ADVICE_SYSTEM_PROMPT, + userContent: evidence.serializedBody, + dataMode: evidence.dataMode, + sourceRevision: config.sourceRevision, + consentGeneration: config.consentGeneration, + createdAtEpochMs: config.createdAtEpochMs, + ...(config.timeoutMs === undefined ? {} : { timeoutMs: config.timeoutMs }), + }); +} + +/** + * Strict advice send boundary. Only an already-prepared advice invocation may + * cross it; malformed output is rejected without prose repair or fallback. + */ +export async function requestStructuredAdvice( + prepared: PreparedAiInvocation, + references: StructuredAdviceReferences, + options: StructuredAdviceSendOptions, +): Promise { + if (prepared.kind !== 'advice' || prepared.dataMode === 'user-draft-only') { + return { + ok: false, + code: 'invalid-prepared-request', + issues: ['prepared request is not structured advice'], + }; + } + let raw: string; + try { + raw = await sendPreparedModelRequest(prepared, options.apiKey, { + ...(options.expectedSourceRevision === undefined + ? {} + : { expectedSourceRevision: options.expectedSourceRevision }), + ...(options.expectedConsentGeneration === undefined + ? {} + : { expectedConsentGeneration: options.expectedConsentGeneration }), + ...(options.signal ? { signal: options.signal } : {}), + ...(options.transport ? { transport: options.transport } : {}), + }); + } catch { + return { + ok: false, + code: 'transport-error', + issues: ['configured BYOK transport failed'], + }; + } + return parseStructuredAdviceOutput(raw, references); +} diff --git a/src/adviceEffectiveness/versionedPersistence.ts b/src/adviceEffectiveness/versionedPersistence.ts index 3afe88ab..b6d71136 100644 --- a/src/adviceEffectiveness/versionedPersistence.ts +++ b/src/adviceEffectiveness/versionedPersistence.ts @@ -1,11 +1,37 @@ -import type { ComparableTaskPair } from './comparison'; import { isAdviceIdentifier } from './contract'; +import { + StoredComparablePair, + parseStoredComparablePair, + toComparableTaskPairs, +} from './comparisonPairing'; +import { + AdviceComparisonResultEnvelope, + parseAdviceComparisonResultEnvelope, +} from './comparisonResult'; + +export type { + StoredAdviceScope, + StoredComparableContext, + StoredComparablePair, + StoredComplexityBand, + StoredConfidence, + StoredCoverage, + StoredEffort, + StoredMetricDirection, + StoredMetricUnit, + StoredModelFamily, + StoredProvider, + StoredQualityPass, + StoredTaskKind, +} from './comparisonPairing'; +export { toComparableTaskPairs } from './comparisonPairing'; /** Stable key: schema versions migrate in-place instead of changing the key. */ export const ADVICE_LOCAL_STATE_KEY = 'ccu.adviceEffectiveness.localState'; export const ADVICE_LOCAL_STATE_VERSION = 2 as const; export const MAX_PERSISTED_ADVICE_FEEDBACK = 500; export const MAX_PERSISTED_COMPARABLE_PAIRS = 200; +export const MAX_PERSISTED_ADVICE_COMPARISON_RESULTS = 200; export type AdviceFeatureMode = 'disabled' | 'enabled'; export type AdviceConsentState = 'not-granted' | 'explicit'; @@ -18,59 +44,11 @@ export interface PersistedAdviceFeedback { recommendationId: string; rating: PersistedAdviceRating; applied: PersistedAdviceApplied; + /** Stable intervention boundary; later rating changes never move it. */ + appliedAtEpochMs: number | null; updatedAtEpochMs: number; } -export type StoredTaskKind = - | 'small-change' - | 'feature' - | 'bug-fix' - | 'review' - | 'other'; -export type StoredModelFamily = 'opus' | 'sonnet' | 'haiku' | 'fable' | 'other'; -export type StoredEffort = 'low' | 'medium' | 'high' | 'xhigh' | 'max' | 'ultra' | 'unknown'; -export type StoredMetricUnit = 'count' | 'tokens' | 'ratio' | 'multiple' | 'milliseconds'; -export type StoredQualityPass = 'passed' | 'failed' | 'unknown'; - -export interface StoredComparablePair { - pairId: string; - adviceId: string; - recommendationId: string; - context: { - taskKind: StoredTaskKind; - complexityBand: 'low' | 'medium' | 'high'; - provider: 'claude' | 'codex'; - modelFamily: StoredModelFamily; - effort: StoredEffort; - metricDefinitionVersion: string; - qualityRubricId: string; - }; - metric: { - name: string; - unit: StoredMetricUnit; - direction: 'lower-is-better' | 'higher-is-better'; - beforeValue: number; - afterValue: number; - }; - quality: { - beforeScore: number; - afterScore: number; - beforePassed: StoredQualityPass; - afterPassed: StoredQualityPass; - beforeEvidenceCount: number; - afterEvidenceCount: number; - }; - evidence: { - beforeCoverage: 'complete' | 'partial' | 'unknown'; - afterCoverage: 'complete' | 'partial' | 'unknown'; - beforeConfidence: 'high' | 'medium' | 'low' | 'unknown'; - afterConfidence: 'high' | 'medium' | 'low' | 'unknown'; - beforeQualityFlags: string[]; - afterQualityFlags: string[]; - }; - recordedAtEpochMs: number; -} - export interface AdviceLocalState { schemaVersion: typeof ADVICE_LOCAL_STATE_VERSION; featureMode: AdviceFeatureMode; @@ -78,6 +56,7 @@ export interface AdviceLocalState { promptSampleConsent: AdviceConsentState; feedback: PersistedAdviceFeedback[]; comparablePairs: StoredComparablePair[]; + comparisonResults: AdviceComparisonResultEnvelope[]; } export interface AdviceLocalStateStorage { @@ -98,6 +77,15 @@ export type AdviceLocalStateMutationResult = | { ok: false; reason: 'invalid-local-data' | 'invalid-input' }; const STATE_KEYS = [ + 'aggregateConsent', + 'comparablePairs', + 'comparisonResults', + 'featureMode', + 'feedback', + 'promptSampleConsent', + 'schemaVersion', +] as const; +const LEGACY_V2_STATE_KEYS = [ 'aggregateConsent', 'comparablePairs', 'featureMode', @@ -108,45 +96,17 @@ const STATE_KEYS = [ const FEEDBACK_KEYS = [ 'adviceId', 'applied', + 'appliedAtEpochMs', 'rating', 'recommendationId', 'updatedAtEpochMs', ] as const; -const PAIR_KEYS = [ +const LEGACY_FEEDBACK_KEYS = [ 'adviceId', - 'context', - 'evidence', - 'metric', - 'pairId', - 'quality', + 'applied', + 'rating', 'recommendationId', - 'recordedAtEpochMs', -] as const; -const CONTEXT_KEYS = [ - 'complexityBand', - 'effort', - 'metricDefinitionVersion', - 'modelFamily', - 'provider', - 'qualityRubricId', - 'taskKind', -] as const; -const METRIC_KEYS = ['afterValue', 'beforeValue', 'direction', 'name', 'unit'] as const; -const QUALITY_KEYS = [ - 'afterEvidenceCount', - 'afterPassed', - 'afterScore', - 'beforeEvidenceCount', - 'beforePassed', - 'beforeScore', -] as const; -const EVIDENCE_KEYS = [ - 'afterConfidence', - 'afterCoverage', - 'afterQualityFlags', - 'beforeConfidence', - 'beforeCoverage', - 'beforeQualityFlags', + 'updatedAtEpochMs', ] as const; function isObject(value: unknown): value is Record { @@ -163,30 +123,10 @@ function isEpochMs(value: unknown): value is number { return typeof value === 'number' && Number.isInteger(value) && value >= 0; } -function isFiniteNonNegative(value: unknown): value is number { - return typeof value === 'number' && Number.isFinite(value) && value >= 0; -} - -function isScore(value: unknown): value is number { - return isFiniteNonNegative(value) && value <= 1; -} - function oneOf(value: unknown, values: readonly T[]): value is T { return typeof value === 'string' && (values as readonly string[]).includes(value); } -function parseOpaqueFlags(value: unknown): string[] | undefined { - if (!Array.isArray(value) || value.length > 32) return undefined; - const result: string[] = []; - const seen = new Set(); - for (const flag of value) { - if (typeof flag !== 'string' || !isAdviceIdentifier(flag) || seen.has(flag)) return undefined; - seen.add(flag); - result.push(flag); - } - return result; -} - function parseFeedback(value: unknown): PersistedAdviceFeedback | undefined { if (!isObject(value) || !hasExactKeys(value, FEEDBACK_KEYS)) return undefined; if ( @@ -196,6 +136,9 @@ function parseFeedback(value: unknown): PersistedAdviceFeedback | undefined { !isAdviceIdentifier(value.recommendationId) || !oneOf(value.rating, ['unrated', 'helpful', 'not-helpful'] as const) || !oneOf(value.applied, ['not-applied', 'applied'] as const) || + !(value.appliedAtEpochMs === null || isEpochMs(value.appliedAtEpochMs)) || + (value.applied === 'applied' && value.appliedAtEpochMs === null) || + (value.applied === 'not-applied' && value.appliedAtEpochMs !== null) || !isEpochMs(value.updatedAtEpochMs) ) { return undefined; @@ -205,107 +148,11 @@ function parseFeedback(value: unknown): PersistedAdviceFeedback | undefined { recommendationId: value.recommendationId, rating: value.rating, applied: value.applied, + appliedAtEpochMs: value.appliedAtEpochMs, updatedAtEpochMs: value.updatedAtEpochMs, }; } -function parseComparablePair(value: unknown): StoredComparablePair | undefined { - if (!isObject(value) || !hasExactKeys(value, PAIR_KEYS)) return undefined; - if ( - typeof value.pairId !== 'string' || - typeof value.adviceId !== 'string' || - typeof value.recommendationId !== 'string' || - !isAdviceIdentifier(value.pairId) || - !isAdviceIdentifier(value.adviceId) || - !isAdviceIdentifier(value.recommendationId) || - !isEpochMs(value.recordedAtEpochMs) || - !isObject(value.context) || - !hasExactKeys(value.context, CONTEXT_KEYS) || - !isObject(value.metric) || - !hasExactKeys(value.metric, METRIC_KEYS) || - !isObject(value.quality) || - !hasExactKeys(value.quality, QUALITY_KEYS) || - !isObject(value.evidence) || - !hasExactKeys(value.evidence, EVIDENCE_KEYS) - ) { - return undefined; - } - const context = value.context; - const metric = value.metric; - const quality = value.quality; - const evidence = value.evidence; - if ( - !oneOf(context.taskKind, ['small-change', 'feature', 'bug-fix', 'review', 'other'] as const) || - !oneOf(context.complexityBand, ['low', 'medium', 'high'] as const) || - !oneOf(context.provider, ['claude', 'codex'] as const) || - !oneOf(context.modelFamily, ['opus', 'sonnet', 'haiku', 'fable', 'other'] as const) || - !oneOf(context.effort, ['low', 'medium', 'high', 'xhigh', 'max', 'ultra', 'unknown'] as const) || - typeof context.metricDefinitionVersion !== 'string' || - typeof context.qualityRubricId !== 'string' || - !isAdviceIdentifier(context.metricDefinitionVersion) || - !isAdviceIdentifier(context.qualityRubricId) || - typeof metric.name !== 'string' || - !isAdviceIdentifier(metric.name) || - !oneOf(metric.unit, ['count', 'tokens', 'ratio', 'multiple', 'milliseconds'] as const) || - !oneOf(metric.direction, ['lower-is-better', 'higher-is-better'] as const) || - !isFiniteNonNegative(metric.beforeValue) || - !isFiniteNonNegative(metric.afterValue) || - !isScore(quality.beforeScore) || - !isScore(quality.afterScore) || - !oneOf(quality.beforePassed, ['passed', 'failed', 'unknown'] as const) || - !oneOf(quality.afterPassed, ['passed', 'failed', 'unknown'] as const) || - !isEpochMs(quality.beforeEvidenceCount) || - !isEpochMs(quality.afterEvidenceCount) || - !oneOf(evidence.beforeCoverage, ['complete', 'partial', 'unknown'] as const) || - !oneOf(evidence.afterCoverage, ['complete', 'partial', 'unknown'] as const) || - !oneOf(evidence.beforeConfidence, ['high', 'medium', 'low', 'unknown'] as const) || - !oneOf(evidence.afterConfidence, ['high', 'medium', 'low', 'unknown'] as const) - ) { - return undefined; - } - const beforeQualityFlags = parseOpaqueFlags(evidence.beforeQualityFlags); - const afterQualityFlags = parseOpaqueFlags(evidence.afterQualityFlags); - if (!beforeQualityFlags || !afterQualityFlags) return undefined; - return { - pairId: value.pairId, - adviceId: value.adviceId, - recommendationId: value.recommendationId, - context: { - taskKind: context.taskKind, - complexityBand: context.complexityBand, - provider: context.provider, - modelFamily: context.modelFamily, - effort: context.effort, - metricDefinitionVersion: context.metricDefinitionVersion, - qualityRubricId: context.qualityRubricId, - }, - metric: { - name: metric.name, - unit: metric.unit, - direction: metric.direction, - beforeValue: metric.beforeValue, - afterValue: metric.afterValue, - }, - quality: { - beforeScore: quality.beforeScore, - afterScore: quality.afterScore, - beforePassed: quality.beforePassed, - afterPassed: quality.afterPassed, - beforeEvidenceCount: quality.beforeEvidenceCount, - afterEvidenceCount: quality.afterEvidenceCount, - }, - evidence: { - beforeCoverage: evidence.beforeCoverage, - afterCoverage: evidence.afterCoverage, - beforeConfidence: evidence.beforeConfidence, - afterConfidence: evidence.afterConfidence, - beforeQualityFlags, - afterQualityFlags, - }, - recordedAtEpochMs: value.recordedAtEpochMs, - }; -} - function parseCurrentState(value: unknown): AdviceLocalState | undefined { if (!isObject(value) || !hasExactKeys(value, STATE_KEYS)) return undefined; if ( @@ -317,7 +164,9 @@ function parseCurrentState(value: unknown): AdviceLocalState | undefined { !Array.isArray(value.feedback) || value.feedback.length > MAX_PERSISTED_ADVICE_FEEDBACK || !Array.isArray(value.comparablePairs) || - value.comparablePairs.length > MAX_PERSISTED_COMPARABLE_PAIRS + value.comparablePairs.length > MAX_PERSISTED_COMPARABLE_PAIRS || + !Array.isArray(value.comparisonResults) || + value.comparisonResults.length > MAX_PERSISTED_ADVICE_COMPARISON_RESULTS ) { return undefined; } @@ -334,11 +183,19 @@ function parseCurrentState(value: unknown): AdviceLocalState | undefined { const comparablePairs: StoredComparablePair[] = []; const pairIds = new Set(); for (const raw of value.comparablePairs) { - const item = parseComparablePair(raw); + const item = parseStoredComparablePair(raw); if (!item || pairIds.has(item.pairId)) return undefined; pairIds.add(item.pairId); comparablePairs.push(item); } + const comparisonResults: AdviceComparisonResultEnvelope[] = []; + const comparisonIds = new Set(); + for (const raw of value.comparisonResults) { + const item = parseAdviceComparisonResultEnvelope(raw); + if (!item || comparisonIds.has(item.comparisonId)) return undefined; + comparisonIds.add(item.comparisonId); + comparisonResults.push(item); + } return { schemaVersion: ADVICE_LOCAL_STATE_VERSION, featureMode: value.featureMode, @@ -346,9 +203,38 @@ function parseCurrentState(value: unknown): AdviceLocalState | undefined { promptSampleConsent: value.promptSampleConsent, feedback, comparablePairs, + comparisonResults, }; } +function migrateLegacyV2(value: Record): AdviceLocalState | undefined { + const oldRoot = hasExactKeys(value, LEGACY_V2_STATE_KEYS); + const currentRoot = hasExactKeys(value, STATE_KEYS); + if ( + (!oldRoot && !currentRoot) || + value.schemaVersion !== ADVICE_LOCAL_STATE_VERSION || + !Array.isArray(value.feedback) || + (oldRoot && (!Array.isArray(value.comparablePairs) || value.comparablePairs.length !== 0)) + ) { + return undefined; + } + const feedback: PersistedAdviceFeedback[] = []; + for (const raw of value.feedback) { + if (!isObject(raw) || !hasExactKeys(raw, LEGACY_FEEDBACK_KEYS)) return undefined; + const parsed = parseFeedback({ + ...raw, + appliedAtEpochMs: raw.applied === 'applied' ? raw.updatedAtEpochMs : null, + }); + if (!parsed) return undefined; + feedback.push(parsed); + } + return parseCurrentState({ + ...value, + feedback, + comparisonResults: currentRoot ? value.comparisonResults : [], + }); +} + interface LegacyFeedbackV1 { adviceId: string; recommendationId: string; @@ -401,6 +287,7 @@ function migrateV1(value: Record): AdviceLocalState | undefined recommendationId: legacy.recommendationId, rating: legacy.rating, applied: legacy.applied ? 'applied' : 'not-applied', + appliedAtEpochMs: legacy.applied ? legacy.updatedAtEpochMs : null, updatedAtEpochMs: legacy.updatedAtEpochMs, }); } @@ -412,6 +299,7 @@ function migrateV1(value: Record): AdviceLocalState | undefined promptSampleConsent: 'not-granted', feedback, comparablePairs: [], + comparisonResults: [], }; } @@ -423,6 +311,15 @@ export function createClosedAdviceLocalState(): AdviceLocalState { promptSampleConsent: 'not-granted', feedback: [], comparablePairs: [], + comparisonResults: [], + }; +} + +/** User-visible privacy reset: keep the enabled surface, close consent, and erase its local ledger. */ +export function createClearedAdviceLocalState(): AdviceLocalState { + return { + ...createClosedAdviceLocalState(), + featureMode: 'enabled', }; } @@ -440,10 +337,14 @@ export async function loadAndMigrateAdviceLocalState( if (raw === undefined) return { ok: true, migrated: false, value: closed }; const current = parseCurrentState(raw); if (current) return { ok: true, migrated: false, value: current }; - if (!isObject(raw) || raw.schemaVersion !== 1) { + if (!isObject(raw)) { return { ok: false, reason: 'invalid-local-data', value: closed }; } - const migrated = migrateV1(raw); + const migrated = raw.schemaVersion === ADVICE_LOCAL_STATE_VERSION + ? migrateLegacyV2(raw) + : raw.schemaVersion === 1 + ? migrateV1(raw) + : undefined; if (!migrated) return { ok: false, reason: 'invalid-local-data', value: closed }; try { await storage.update(ADVICE_LOCAL_STATE_KEY, migrated); @@ -491,18 +392,34 @@ export function upsertAdviceLocalFeedback( const existing = current.feedback.find( (item) => item.adviceId === input.adviceId && item.recommendationId === input.recommendationId, ); + let rating = existing?.rating ?? 'unrated'; + let applied = existing?.applied ?? 'not-applied'; + let appliedAtEpochMs = existing?.appliedAtEpochMs ?? null; + if (input.kind === 'helpful') { + rating = rating === 'helpful' ? 'unrated' : 'helpful'; + } else if (input.kind === 'not-helpful') { + rating = rating === 'not-helpful' ? 'unrated' : 'not-helpful'; + } else if (applied === 'applied') { + applied = 'not-applied'; + appliedAtEpochMs = null; + } else { + applied = 'applied'; + appliedAtEpochMs = input.updatedAtEpochMs; + } const next: PersistedAdviceFeedback = { adviceId: input.adviceId, recommendationId: input.recommendationId, - rating: input.kind === 'applied' ? existing?.rating ?? 'unrated' : input.kind, - applied: input.kind === 'applied' ? 'applied' : existing?.applied ?? 'not-applied', + rating, + applied, + appliedAtEpochMs, updatedAtEpochMs: input.updatedAtEpochMs, }; - const feedback = current.feedback - .filter( - (item) => item.adviceId !== input.adviceId || item.recommendationId !== input.recommendationId, - ) - .concat(next) + const withoutTarget = current.feedback.filter( + (item) => item.adviceId !== input.adviceId || item.recommendationId !== input.recommendationId, + ); + const feedback = (rating === 'unrated' && applied === 'not-applied' + ? withoutTarget + : withoutTarget.concat(next)) .slice(-MAX_PERSISTED_ADVICE_FEEDBACK); return { ok: true, value: { ...current, feedback } }; } @@ -513,7 +430,7 @@ export function appendStoredComparablePair( ): AdviceLocalStateMutationResult { const current = parseCurrentState(state); if (!current) return { ok: false, reason: 'invalid-local-data' }; - const pair = parseComparablePair(input); + const pair = parseStoredComparablePair(input); if (!pair || current.comparablePairs.some((item) => item.pairId === pair.pairId)) { return { ok: false, reason: 'invalid-input' }; } @@ -528,6 +445,30 @@ export function appendStoredComparablePair( }; } +export function appendAdviceComparisonResult( + state: AdviceLocalState, + input: unknown, +): AdviceLocalStateMutationResult { + const current = parseCurrentState(state); + if (!current) return { ok: false, reason: 'invalid-local-data' }; + const result = parseAdviceComparisonResultEnvelope(input); + if ( + !result || + current.comparisonResults.some((item) => item.comparisonId === result.comparisonId) + ) { + return { ok: false, reason: 'invalid-input' }; + } + return { + ok: true, + value: { + ...current, + comparisonResults: current.comparisonResults + .concat(result) + .slice(-MAX_PERSISTED_ADVICE_COMPARISON_RESULTS), + }, + }; +} + export interface AdviceRecommendationLineage { provider: 'claude' | 'codex'; recommendationId: string; @@ -556,58 +497,3 @@ export function selectStoredComparablePairLineage( pair.recommendationId === lineage.recommendationId, ); } - -/** Convert only state that has already passed the exact persistence parser. */ -export function toComparableTaskPairs( - pairs: readonly StoredComparablePair[], -): ComparableTaskPair[] { - return pairs.map((pair) => { - const context = { - taskKind: pair.context.taskKind, - complexityBand: pair.context.complexityBand, - provider: pair.context.provider, - modelFamily: pair.context.modelFamily, - effort: pair.context.effort, - metricDefinitionVersion: pair.context.metricDefinitionVersion, - qualityRubricId: pair.context.qualityRubricId, - }; - const outcome = ( - period: 'before' | 'after', - ): ComparableTaskPair['before'] => ({ - context: { ...context }, - primaryMetric: { - name: pair.metric.name, - unit: pair.metric.unit, - direction: pair.metric.direction, - value: period === 'before' ? pair.metric.beforeValue : pair.metric.afterValue, - }, - quality: { - score: period === 'before' ? pair.quality.beforeScore : pair.quality.afterScore, - passed: (() => { - const value = period === 'before' ? pair.quality.beforePassed : pair.quality.afterPassed; - return value === 'unknown' ? null : value === 'passed'; - })(), - evidenceCount: - period === 'before' - ? pair.quality.beforeEvidenceCount - : pair.quality.afterEvidenceCount, - }, - evidence: { - coverage: - period === 'before' - ? pair.evidence.beforeCoverage - : pair.evidence.afterCoverage, - confidence: - period === 'before' - ? pair.evidence.beforeConfidence - : pair.evidence.afterConfidence, - qualityFlags: [ - ...(period === 'before' - ? pair.evidence.beforeQualityFlags - : pair.evidence.afterQualityFlags), - ], - }, - }); - return { pairId: pair.pairId, before: outcome('before'), after: outcome('after') }; - }); -} diff --git a/src/advisor.ts b/src/advisor.ts index b7689dcb..6868c4d1 100644 --- a/src/advisor.ts +++ b/src/advisor.ts @@ -1,76 +1,17 @@ -// Optional "usage advice" feature: sends a usage summary (and, optionally, a -// sample of the developer's own prompts) to a model and returns advice on how -// to use Claude Code more effectively. -// -// Transport history (v2.1 Phase 9): the exported entry point still contains a -// dormant 'subscription' prototype, but production never selects it. Direct -// Messages calls with Claude Code subscription credentials later returned 403 -// and are not an approved/default AI backend; those credentials remain scoped -// to the quota API. Supported production paths are user-configured BYOK: -// 1. 'api' + apiFormat 'anthropic' (the default for a configured key) — -// x-api-key against /v1/messages. -// 2. 'api' + apiFormat 'openai' — the OpenAI chat-completions shape -// (DeepSeek etc.), kept for compatibility. -// Anthropic is the default shape across the extension; OpenAI is opt-in. +// Shared AI helpers for the two explicit user-triggered workflows. Production +// can send only an already-previewed PreparedAiInvocation through the user's +// configured BYOK endpoint. Anthropic Messages and OpenAI-compatible chat are +// wire formats; neither implies subscription/OAuth credential reuse. import { HttpResponse, requestViaCurl, requestViaFetch } from './httpClient'; +import { + PreparedAiInvocation, + PreparedAiTransportRequest, + sendPreparedAiInvocation, +} from './adviceEffectiveness/preparedRequest'; -export type AdviceBackend = 'subscription' | 'api'; export type AdviceFormat = 'anthropic' | 'openai'; -export interface AdviceOptions { - // Transport selection. Defaults (when omitted) keep the pre-2.1 behaviour: - // backend 'api' + format 'openai'. - backend?: AdviceBackend; - apiFormat?: AdviceFormat; - // backend 'api': - apiKey: string; - apiUrl: string; - model: string; - // backend 'subscription': the cheap model to spend a little quota on, plus a - // provider for a valid OAuth access token (ClaudeApiClient.getAccessToken). - subscriptionModel?: string; - getSubscriptionToken?: () => Promise; - // The usage digest sent as the user turn (built by adviceSummary.ts). - summary: string; - // Localised name of the user's UI language, e.g. "简体中文 (Simplified Chinese)". - language: string; - // '', 'high' or 'max' — passed as reasoning_effort for OpenAI-format models. - reasoningEffort?: string; - // Free-text background about the user/project; when set, the reply ends with - // a "Personalised for this project" section calibrated against it. - userContext?: string; - // Abort the request after this long (reasoning models can take minutes). - timeoutMs?: number; -} - -const DEFAULT_TIMEOUT_MS = 120_000; -const ADVICE_MAX_TOKENS = 16_000; - -function buildSystemPrompt(language: string, userContext?: string): string { - let prompt = - 'You are a coaching advisor that helps a developer use the Claude Code AI ' + - 'coding agent more effectively. You are given a breakdown of their usage and ' + - 'a sample of their actual prompts. Your PRIMARY goal: advise how to write ' + - 'clearer, more complete and more precise instructions so tasks are completed ' + - 'correctly and efficiently — point at concrete weaknesses in the sample ' + - 'prompts and show better rewrites. SECONDARY goal: where it does not hurt ' + - 'clarity, suggest ways to reduce token consumption. Be concrete and ' + - 'actionable, use short sections and bullet points. '; - const ctx = (userContext || '').trim(); - if (ctx !== '') { - prompt += - 'The user provided this background about themself and the project: ' + - `"${ctx.slice(0, 1000)}". End your reply with a final section titled ` + - '"Personalised for this project" that calibrates the advice against this ' + - 'background instead of generic best practice. '; - } - prompt += - `IMPORTANT: write your entire reply in ${language}, regardless of the ` + - 'language(s) used in the sample prompts.'; - return prompt; -} - /** Normalise an OpenAI-compatible chat endpoint URL. */ function normalizeOpenAiUrl(url: string): string { let u = (url || '').trim().replace(/\/+$/, ''); @@ -99,10 +40,11 @@ function normalizeAnthropicUrl(url: string): string { async function send( url: string, headers: Record, - body: string, - timeoutMs: number + body: string | Uint8Array, + timeoutMs: number, + signal?: AbortSignal, ): Promise { - const reqOpts = { method: 'POST', headers, body }; + const reqOpts = { method: 'POST', headers, body, ...(signal ? { signal } : {}) }; const curl = (): Promise => requestViaCurl(url, { ...reqOpts, timeoutSec: Math.ceil(timeoutMs / 1000) }); try { @@ -114,133 +56,92 @@ async function send( } return r; } catch { + if (signal?.aborted) throw new Error('Request cancelled'); try { return await requestViaFetch(url, { ...reqOpts, timeoutMs }); } catch { + if (signal?.aborted) throw new Error('Request cancelled'); return curl(); } } } -/** - * Send one system+user turn to whichever backend is configured and return the - * assistant text. Shared by the advice feature and the Usage Optimizer. - */ -export async function callModel( - systemPrompt: string, - userContent: string, - options: AdviceOptions -): Promise { - const timeoutMs = options.timeoutMs ?? DEFAULT_TIMEOUT_MS; - const backend: AdviceBackend = options.backend ?? 'api'; - const format: AdviceFormat = options.apiFormat ?? 'openai'; - - // --- Anthropic Messages shape (subscription, or api+anthropic) --- - if (backend === 'subscription' || format === 'anthropic') { - let headers: Record; - let model: string; - let url = normalizeAnthropicUrl(options.apiUrl); - if (backend === 'subscription') { - const token = options.getSubscriptionToken ? await options.getSubscriptionToken() : null; - if (!token) { - throw new Error( - 'No Claude subscription session found — sign in to Claude Code, or configure an API key.' - ); - } - url = 'https://api.anthropic.com/v1/messages'; - model = options.subscriptionModel || 'claude-haiku-4-5'; - headers = { - 'Content-Type': 'application/json', - Authorization: `Bearer ${token}`, - 'anthropic-version': '2023-06-01', - 'anthropic-beta': 'oauth-2025-04-20', - }; - } else { - model = options.model; - headers = { - 'Content-Type': 'application/json', - 'x-api-key': options.apiKey, - 'anthropic-version': '2023-06-01', - }; - } - const body = JSON.stringify({ - model, - max_tokens: ADVICE_MAX_TOKENS, - system: systemPrompt, - messages: [{ role: 'user', content: userContent }], - }); - const response = await send(url, headers, body, timeoutMs); - if (response.status < 200 || response.status >= 300) { - throw new Error(`API ${response.status}: ${response.body.slice(0, 300)}`); - } - let data: { content?: { type?: string; text?: string }[] }; - try { - data = JSON.parse(response.body); - } catch { - throw new Error(`The API returned a non-JSON response: ${response.body.slice(0, 200)}`); - } - const text = (data.content || []) - .filter((b) => b.type === 'text' && typeof b.text === 'string') - .map((b) => b.text as string) - .join(''); - if (!text.trim()) { - throw new Error('The model returned an empty response.'); - } - return text; - } - - // --- OpenAI chat-completions shape (api + openai) --- - const body: Record = { - model: options.model, - stream: false, - messages: [ - { role: 'system', content: systemPrompt }, - { role: 'user', content: userContent }, - ], - }; - if (options.reasoningEffort && options.reasoningEffort.trim() !== '') { - body.reasoning_effort = options.reasoningEffort.trim(); - body.thinking = { type: 'enabled' }; - } - const url = normalizeOpenAiUrl(options.apiUrl); - const headers = { - 'Content-Type': 'application/json', - Authorization: `Bearer ${options.apiKey}`, - }; - const response = await send(url, headers, JSON.stringify(body), timeoutMs); +function parsePreparedModelResponse( + prepared: PreparedAiInvocation, + response: HttpResponse, +): string { if (response.status < 200 || response.status >= 300) { - const hint = - response.status === 404 - ? ' (check advice.apiUrl — for DeepSeek it is https://api.deepseek.com/chat/completions)' - : ''; + const hint = prepared.apiFormat === 'openai' && response.status === 404 + ? ' (check advice.apiUrl — for DeepSeek it is https://api.deepseek.com/chat/completions)' + : ''; throw new Error(`API ${response.status}${hint}: ${response.body.slice(0, 300)}`); } - let data: { choices?: { message?: { content?: string } }[] }; + let data: unknown; try { data = JSON.parse(response.body); } catch { throw new Error(`The API returned a non-JSON response: ${response.body.slice(0, 200)}`); } - const content = data.choices?.[0]?.message?.content; - if (!content || content.trim() === '') { - throw new Error('The model returned an empty response.'); + let text = ''; + if (prepared.apiFormat === 'anthropic') { + const blocks = (data as { content?: { type?: string; text?: string }[] }).content; + text = (blocks ?? []) + .filter((block) => block.type === 'text' && typeof block.text === 'string') + .map((block) => block.text as string) + .join(''); + } else { + text = (data as { choices?: { message?: { content?: string } }[] }) + .choices?.[0]?.message?.content ?? ''; } - return content; + if (!text.trim()) throw new Error('The model returned an empty response.'); + return text; } /** - * Request usage advice. Returns the advice text (markdown). + * Production transport for an already-previewed invocation. The full wire + * body is the original canonical byte object on every fetch retry/curl + * fallback; this function never calls JSON.stringify. */ -export async function getUsageAdvice(options: AdviceOptions): Promise { - const systemPrompt = buildSystemPrompt(options.language || 'English', options.userContext); - return callModel(systemPrompt, options.summary, options); +export async function sendPreparedModelRequest( + prepared: PreparedAiInvocation, + apiKey: string, + options: { + expectedSourceRevision?: string; + expectedConsentGeneration?: number; + signal?: AbortSignal; + transport?: (request: PreparedAiTransportRequest) => Promise; + } = {}, +): Promise { + const response = await sendPreparedAiInvocation( + prepared, + { + backend: 'api', + apiKey, + ...(options.expectedSourceRevision !== undefined + ? { expectedSourceRevision: options.expectedSourceRevision } + : {}), + ...(options.expectedConsentGeneration !== undefined + ? { expectedConsentGeneration: options.expectedConsentGeneration } + : {}), + ...(options.signal ? { signal: options.signal } : {}), + }, + options.transport ?? ((request) => send( + request.endpoint, + request.headers, + request.canonicalBytes, + request.timeoutMs, + request.signal, + )), + ); + return parsePreparedModelResponse(prepared, response); } // === Usage Optimizer (Phase 9c) === // The optimizer turns a rough pasted request into one tight, paste-ready prompt // plus a settings recommendation. The system prompt and the reply parser are // pure functions here (no VS Code dependency) so they can be unit-tested; the -// VS Code glue (consent modal, config, webview round-trip) lives in extension.ts. +// VS Code glue (exact preview, explicit send, config, webview round-trip) lives +// in extension.ts and webview.ts. export interface OptimizerLenses { resolve: boolean; // flag ambiguous references diff --git a/src/backgroundWorkState.ts b/src/backgroundWorkState.ts new file mode 100644 index 00000000..6d41484d --- /dev/null +++ b/src/backgroundWorkState.ts @@ -0,0 +1,451 @@ +export const BACKGROUND_WORK_STATE_SCHEMA_VERSION = 1 as const; + +export const BACKGROUND_WORK_STATUSES = [ + 'eligible', + 'running', + 'cooldown', + 'paused', + 'complete', +] as const; + +export const BACKGROUND_WORK_REASONS = [ + 'first-index', + 'parser-migration', + 'period-migration', + 'hourly-history', + 'history-backfill', + 'rule-migration', + 'resume', +] as const; + +export const BACKGROUND_WORK_PAUSED_REASONS = [ + 'failure-backoff', + 'no-progress', + 'user', + 'corrupt-state', +] as const; + +export const BACKGROUND_WORK_TRIGGERS = ['automatic', 'manual'] as const; + +export type BackgroundWorkStatus = typeof BACKGROUND_WORK_STATUSES[number]; +export type BackgroundWorkReason = typeof BACKGROUND_WORK_REASONS[number]; +export type BackgroundWorkPausedReason = typeof BACKGROUND_WORK_PAUSED_REASONS[number]; +export type BackgroundWorkTrigger = typeof BACKGROUND_WORK_TRIGGERS[number]; + +/** A monotonic, aggregate-only watermark. It is not a scanner cursor: the + * provider index remains responsible for exact resume positions. */ +export interface BackgroundWorkProgress { + completedUnits: number; + totalUnits: number; + completedBytes: number; + totalBytes: number; +} + +export interface BackgroundWorkState { + schemaVersion: typeof BACKGROUND_WORK_STATE_SCHEMA_VERSION; + measurementVersion: number; + status: BackgroundWorkStatus; + reason: BackgroundWorkReason; + progress: BackgroundWorkProgress; + failStreak: number; + nextEligibleAt: number | null; + pausedReason: BackgroundWorkPausedReason | null; + updatedAt: number; +} + +export interface BackgroundWorkBackoffPolicy { + failureBaseMs: number; + failureMaxMs: number; + noProgressCooldownMs: number; +} + +export const DEFAULT_BACKGROUND_WORK_BACKOFF_POLICY: Readonly = + Object.freeze({ + failureBaseMs: 60_000, + failureMaxMs: 3_600_000, + noProgressCooldownMs: 300_000, + }); + +export interface CreateBackgroundWorkStateOptions { + measurementVersion: number; + reason: BackgroundWorkReason; + now: number; + progress?: BackgroundWorkProgress; +} + +export type BackgroundWorkRestoreDisposition = + | 'new' + | 'valid' + | 'measurement-changed' + | 'corrupt'; + +export interface BackgroundWorkRestoreResult { + disposition: BackgroundWorkRestoreDisposition; + state: BackgroundWorkState; +} + +export interface BackgroundWorkStartOptions { + trigger: BackgroundWorkTrigger; + now: number; + reason?: BackgroundWorkReason; +} + +export interface BackgroundWorkStartResult { + started: boolean; + state: BackgroundWorkState; +} + +const STATE_FIELDS = [ + 'schemaVersion', + 'measurementVersion', + 'status', + 'reason', + 'progress', + 'failStreak', + 'nextEligibleAt', + 'pausedReason', + 'updatedAt', +] as const; + +const PROGRESS_FIELDS = [ + 'completedUnits', + 'totalUnits', + 'completedBytes', + 'totalBytes', +] as const; + +const statusSet = new Set(BACKGROUND_WORK_STATUSES); +const reasonSet = new Set(BACKGROUND_WORK_REASONS); +const pausedReasonSet = new Set(BACKGROUND_WORK_PAUSED_REASONS); +const triggerSet = new Set(BACKGROUND_WORK_TRIGGERS); + +function isRecord(value: unknown): value is Record { + return typeof value === 'object' && value !== null && !Array.isArray(value); +} + +function hasExactFields(value: Record, fields: readonly string[]): boolean { + const keys = Object.keys(value); + return keys.length === fields.length && fields.every((field) => + Object.prototype.hasOwnProperty.call(value, field)); +} + +function isSafeNonNegativeInteger(value: unknown): value is number { + return typeof value === 'number' && Number.isSafeInteger(value) && value >= 0; +} + +function assertSafeNonNegativeInteger(value: number, field: string): void { + if (!isSafeNonNegativeInteger(value)) { + throw new RangeError(`${field} must be a non-negative safe integer`); + } +} + +function assertReason(reason: BackgroundWorkReason): void { + if (!reasonSet.has(reason)) throw new TypeError('reason must be a fixed BackgroundWorkReason'); +} + +function assertTrigger(trigger: BackgroundWorkTrigger): void { + if (!triggerSet.has(trigger)) throw new TypeError('trigger must be automatic or manual'); +} + +function isProgress(value: unknown): value is BackgroundWorkProgress { + if (!isRecord(value) || !hasExactFields(value, PROGRESS_FIELDS)) return false; + if (!isSafeNonNegativeInteger(value.completedUnits) || + !isSafeNonNegativeInteger(value.totalUnits) || + !isSafeNonNegativeInteger(value.completedBytes) || + !isSafeNonNegativeInteger(value.totalBytes)) { + return false; + } + return value.completedUnits <= value.totalUnits && value.completedBytes <= value.totalBytes; +} + +function cloneProgress(progress: BackgroundWorkProgress): BackgroundWorkProgress { + if (!isProgress(progress)) throw new TypeError('progress must contain fixed non-negative numeric fields'); + return { + completedUnits: progress.completedUnits, + totalUnits: progress.totalUnits, + completedBytes: progress.completedBytes, + totalBytes: progress.totalBytes, + }; +} + +function emptyProgress(): BackgroundWorkProgress { + return { + completedUnits: 0, + totalUnits: 0, + completedBytes: 0, + totalBytes: 0, + }; +} + +function validStatusFields(value: Record): boolean { + switch (value.status) { + case 'eligible': + return isSafeNonNegativeInteger(value.nextEligibleAt) && value.pausedReason === null; + case 'running': + case 'complete': + return value.nextEligibleAt === null && value.pausedReason === null; + case 'cooldown': + return isSafeNonNegativeInteger(value.nextEligibleAt) && + (value.pausedReason === 'failure-backoff' || value.pausedReason === 'no-progress'); + case 'paused': + return value.nextEligibleAt === null && + (value.pausedReason === 'user' || value.pausedReason === 'corrupt-state'); + default: + return false; + } +} + +export function isBackgroundWorkState(value: unknown): value is BackgroundWorkState { + if (!isRecord(value) || !hasExactFields(value, STATE_FIELDS)) return false; + return value.schemaVersion === BACKGROUND_WORK_STATE_SCHEMA_VERSION && + isSafeNonNegativeInteger(value.measurementVersion) && value.measurementVersion > 0 && + statusSet.has(value.status) && + reasonSet.has(value.reason) && + isProgress(value.progress) && + isSafeNonNegativeInteger(value.failStreak) && + (value.nextEligibleAt === null || isSafeNonNegativeInteger(value.nextEligibleAt)) && + (value.pausedReason === null || pausedReasonSet.has(value.pausedReason)) && + isSafeNonNegativeInteger(value.updatedAt) && + validStatusFields(value); +} + +function cloneState(value: BackgroundWorkState): BackgroundWorkState { + return { + schemaVersion: BACKGROUND_WORK_STATE_SCHEMA_VERSION, + measurementVersion: value.measurementVersion, + status: value.status, + reason: value.reason, + progress: cloneProgress(value.progress), + failStreak: value.failStreak, + nextEligibleAt: value.nextEligibleAt, + pausedReason: value.pausedReason, + updatedAt: value.updatedAt, + }; +} + +export function createBackgroundWorkState( + options: CreateBackgroundWorkStateOptions, +): BackgroundWorkState { + assertSafeNonNegativeInteger(options.measurementVersion, 'measurementVersion'); + if (options.measurementVersion === 0) throw new RangeError('measurementVersion must be positive'); + assertSafeNonNegativeInteger(options.now, 'now'); + assertReason(options.reason); + return { + schemaVersion: BACKGROUND_WORK_STATE_SCHEMA_VERSION, + measurementVersion: options.measurementVersion, + status: 'eligible', + reason: options.reason, + progress: options.progress === undefined ? emptyProgress() : cloneProgress(options.progress), + failStreak: 0, + nextEligibleAt: options.now, + pausedReason: null, + updatedAt: options.now, + }; +} + +export function restoreBackgroundWorkState( + value: unknown, + options: CreateBackgroundWorkStateOptions, +): BackgroundWorkRestoreResult { + const fresh = (): BackgroundWorkState => createBackgroundWorkState(options); + if (value === undefined || value === null) return { disposition: 'new', state: fresh() }; + if (!isBackgroundWorkState(value)) { + const state = fresh(); + return { + disposition: 'corrupt', + state: { + ...state, + status: 'paused', + nextEligibleAt: null, + pausedReason: 'corrupt-state', + }, + }; + } + if (value.measurementVersion !== options.measurementVersion) { + return { disposition: 'measurement-changed', state: fresh() }; + } + return { disposition: 'valid', state: cloneState(value) }; +} + +export function canStartBackgroundWork( + state: BackgroundWorkState, + options: Pick, +): boolean { + assertTrigger(options.trigger); + assertSafeNonNegativeInteger(options.now, 'now'); + if (!isBackgroundWorkState(state)) return false; + if (state.status === 'complete' || state.status === 'running') return false; + if (state.status === 'paused') { + return state.pausedReason === 'user' && options.trigger === 'manual'; + } + if (options.trigger === 'manual') return true; + return state.nextEligibleAt !== null && options.now >= state.nextEligibleAt; +} + +export function beginBackgroundWork( + state: BackgroundWorkState, + options: BackgroundWorkStartOptions, +): BackgroundWorkStartResult { + if (options.reason !== undefined) assertReason(options.reason); + if (!canStartBackgroundWork(state, options)) return { started: false, state }; + return { + started: true, + state: { + ...cloneState(state), + status: 'running', + reason: options.reason ?? state.reason, + nextEligibleAt: null, + pausedReason: null, + updatedAt: options.now, + }, + }; +} + +function assertRunningState(state: BackgroundWorkState): void { + if (!isBackgroundWorkState(state) || state.status !== 'running') { + throw new Error('background work attempt must be running'); + } +} + +function assertBackoffPolicy(policy: BackgroundWorkBackoffPolicy): void { + assertSafeNonNegativeInteger(policy.failureBaseMs, 'failureBaseMs'); + assertSafeNonNegativeInteger(policy.failureMaxMs, 'failureMaxMs'); + assertSafeNonNegativeInteger(policy.noProgressCooldownMs, 'noProgressCooldownMs'); + if (policy.failureBaseMs === 0 || policy.failureMaxMs < policy.failureBaseMs || + policy.noProgressCooldownMs === 0) { + throw new RangeError('background work backoff durations must be positive and ordered'); + } +} + +export function backgroundWorkFailureBackoffMs( + failStreak: number, + policy: BackgroundWorkBackoffPolicy = DEFAULT_BACKGROUND_WORK_BACKOFF_POLICY, +): number { + assertSafeNonNegativeInteger(failStreak, 'failStreak'); + assertBackoffPolicy(policy); + if (failStreak === 0) return 0; + const exponent = Math.min(52, failStreak - 1); + return Math.min(policy.failureMaxMs, policy.failureBaseMs * Math.pow(2, exponent)); +} + +function safeTimestampAfter(now: number, delay: number): number { + return Math.min(Number.MAX_SAFE_INTEGER, now + delay); +} + +export function recordBackgroundWorkFailure( + state: BackgroundWorkState, + options: { now: number }, + policy: BackgroundWorkBackoffPolicy = DEFAULT_BACKGROUND_WORK_BACKOFF_POLICY, +): BackgroundWorkState { + assertRunningState(state); + assertSafeNonNegativeInteger(options.now, 'now'); + assertBackoffPolicy(policy); + const failStreak = Math.min(Number.MAX_SAFE_INTEGER, state.failStreak + 1); + return { + ...cloneState(state), + status: 'cooldown', + failStreak, + nextEligibleAt: safeTimestampAfter( + options.now, + backgroundWorkFailureBackoffMs(failStreak, policy), + ), + pausedReason: 'failure-backoff', + updatedAt: options.now, + }; +} + +/** Return an interrupted in-flight attempt to the immediately eligible state. + * Cancellation is neither a failure nor proof of no progress: the provider + * index owns the exact durable cursor and the next permitted coordinator pass + * may resume it from there. */ +export function interruptBackgroundWork( + state: BackgroundWorkState, + options: { now: number }, +): BackgroundWorkState { + assertRunningState(state); + assertSafeNonNegativeInteger(options.now, 'now'); + return { + ...cloneState(state), + status: 'eligible', + reason: 'resume', + nextEligibleAt: options.now, + pausedReason: null, + updatedAt: options.now, + }; +} + +function mergedProgress( + current: BackgroundWorkProgress, + incoming: BackgroundWorkProgress, +): BackgroundWorkProgress { + const checked = cloneProgress(incoming); + const completedUnits = Math.max(current.completedUnits, checked.completedUnits); + const completedBytes = Math.max(current.completedBytes, checked.completedBytes); + return { + completedUnits, + totalUnits: Math.max(current.totalUnits, checked.totalUnits, completedUnits), + completedBytes, + totalBytes: Math.max(current.totalBytes, checked.totalBytes, completedBytes), + }; +} + +export function recordBackgroundWorkProgress( + state: BackgroundWorkState, + options: { now: number; complete: boolean; progress: BackgroundWorkProgress }, + policy: BackgroundWorkBackoffPolicy = DEFAULT_BACKGROUND_WORK_BACKOFF_POLICY, +): BackgroundWorkState { + assertRunningState(state); + assertSafeNonNegativeInteger(options.now, 'now'); + assertBackoffPolicy(policy); + const progress = mergedProgress(state.progress, options.progress); + const advanced = progress.completedUnits > state.progress.completedUnits || + progress.completedBytes > state.progress.completedBytes; + + if (options.complete) { + return { + ...cloneState(state), + status: 'complete', + progress, + failStreak: 0, + nextEligibleAt: null, + pausedReason: null, + updatedAt: options.now, + }; + } + if (advanced) { + return { + ...cloneState(state), + status: 'eligible', + progress, + failStreak: 0, + nextEligibleAt: options.now, + pausedReason: null, + updatedAt: options.now, + }; + } + return { + ...cloneState(state), + status: 'cooldown', + progress, + nextEligibleAt: safeTimestampAfter(options.now, policy.noProgressCooldownMs), + pausedReason: 'no-progress', + updatedAt: options.now, + }; +} + +export function pauseBackgroundWork( + state: BackgroundWorkState, + options: { now: number }, +): BackgroundWorkState { + assertSafeNonNegativeInteger(options.now, 'now'); + if (!isBackgroundWorkState(state)) throw new TypeError('background work state is invalid'); + if (state.status === 'complete' || state.pausedReason === 'corrupt-state') return state; + if (state.status === 'paused' && state.pausedReason === 'user') return state; + return { + ...cloneState(state), + status: 'paused', + nextEligibleAt: null, + pausedReason: 'user', + updatedAt: options.now, + }; +} diff --git a/src/claudeApiClient.ts b/src/claudeApiClient.ts index d54878dd..081e1b5a 100644 --- a/src/claudeApiClient.ts +++ b/src/claudeApiClient.ts @@ -189,15 +189,21 @@ export class ClaudeApiClient { return Date.now() >= credentials.claudeAiOauth.expiresAt - 60 * 1000; } - private async refreshAccessToken(credentials: ClaudeCredentials): Promise { + private async refreshAccessToken( + credentials: ClaudeCredentials, + signal?: AbortSignal, + ): Promise { + if (signal?.aborted) throw new Error('Request cancelled'); const r = await this.request('https://console.anthropic.com/v1/oauth/token', { method: 'POST', headers: { 'Content-Type': 'application/json' }, body: JSON.stringify({ refresh_token: credentials.claudeAiOauth.refreshToken, grant_type: 'refresh_token' - }) + }), + signal, }); + if (signal?.aborted) throw new Error('Request cancelled'); if (r.status !== 200) { throw new Error(`Token refresh failed: ${r.status}`); } @@ -214,7 +220,8 @@ export class ClaudeApiClient { return updated; } - private async getValidCredentials(): Promise { + private async getValidCredentials(signal?: AbortSignal): Promise { + if (signal?.aborted) throw new Error('Request cancelled'); // Re-read from disk/keychain on every call so switching Claude accounts is // honoured without a window reload. #45: a switched-in account has a *valid* // (non-expired) token, so the expiry-only re-read below never noticed it and @@ -222,6 +229,7 @@ export class ClaudeApiClient { // loadCredentials refreshes this.credentials; fall back to the cached copy // only if the fresh read transiently fails. let credentials = (await this.loadCredentials()) || this.credentials; + if (signal?.aborted) throw new Error('Request cancelled'); if (!credentials) { return null; } @@ -238,7 +246,7 @@ export class ClaudeApiClient { } this.log('token: expired, refreshing'); try { - credentials = await this.refreshAccessToken(fresh || credentials); + credentials = await this.refreshAccessToken(fresh || credentials, signal); } catch (e) { this.log(`token: refresh failed: ${(e as Error).message}`); return null; @@ -262,8 +270,14 @@ export class ClaudeApiClient { * rejection ("403 Request not allowed"). */ private async request( url: string, - opts: { method?: string; headers?: Record; body?: string } + opts: { + method?: string; + headers?: Record; + body?: string; + signal?: AbortSignal; + } ): Promise { + if (opts.signal?.aborted) throw new Error('Request cancelled'); if (!this.preferCurl) { try { const r = await requestViaFetch(url, opts); @@ -275,22 +289,25 @@ export class ClaudeApiClient { return r; } } catch (e) { + if (opts.signal?.aborted) throw new Error('Request cancelled'); this.log(`fetch: error ${(e as Error).message} → trying curl`); } } + if (opts.signal?.aborted) throw new Error('Request cancelled'); const r = await requestViaCurl(url, opts, (line) => this.log(line)); this.log(`curl: ${r.status} ${url}`); return r; } - private callUsageApi(accessToken: string): Promise { + private callUsageApi(accessToken: string, signal?: AbortSignal): Promise { return this.request('https://api.anthropic.com/api/oauth/usage', { method: 'GET', headers: { Authorization: `Bearer ${accessToken}`, 'anthropic-beta': 'oauth-2025-04-20', 'Content-Type': 'application/json' - } + }, + signal, }); } @@ -300,19 +317,24 @@ export class ClaudeApiClient { * or when anything else goes wrong. All decisions are logged to the * "Claude Code Usage" output channel for diagnosis. */ - async fetchUsageLimits(): Promise { + async fetchUsageLimits(signal?: AbortSignal): Promise { + if (signal?.aborted) return null; if (Date.now() < this.rateLimitedUntil) { this.log(`skip: cooling down for ${Math.round((this.rateLimitedUntil - Date.now()) / 1000)}s after 429`); return null; } try { - const credentials = await this.getValidCredentials(); + const credentials = await this.getValidCredentials(signal); if (!credentials) { return null; } - let response = await this.callUsageApi(credentials.claudeAiOauth.accessToken); + if (signal?.aborted) return null; + let response = await this.callUsageApi( + credentials.claudeAiOauth.accessToken, + signal, + ); if (response.status === 429) { // 60 s cool-down. The old flat 5-minute cool-down made a single 429 @@ -328,8 +350,11 @@ export class ClaudeApiClient { if (response.status === 401) { this.log('401: forcing token refresh and retrying once'); try { - const refreshed = await this.refreshAccessToken(credentials); - response = await this.callUsageApi(refreshed.claudeAiOauth.accessToken); + const refreshed = await this.refreshAccessToken(credentials, signal); + response = await this.callUsageApi( + refreshed.claudeAiOauth.accessToken, + signal, + ); } catch (e) { this.log(`401 retry: refresh failed: ${(e as Error).message}`); return null; diff --git a/src/claudeIncrementalIndex.ts b/src/claudeIncrementalIndex.ts index 56ddc260..398cbf24 100644 --- a/src/claudeIncrementalIndex.ts +++ b/src/claudeIncrementalIndex.ts @@ -16,7 +16,14 @@ import { newAnalysisAcc, validateUsageRecord, } from './dataLoader'; -import { dayKeyInZone, monthKeyInZone } from './dateKeys'; +import { + dayKeyInZone, + formatHourLabel, + hourKeyInZone, + monthKeyInZone, + resolveTimeZone, + rollingDayKeys, +} from './dateKeys'; import { I18n } from './i18n'; import { isRetryDuplicatePrompt } from './promptDedup'; import { @@ -102,6 +109,8 @@ interface CopyOnWriteKeys { } export interface ClaudeUsageIndex { + /** Canonical configured zone used by every materialized calendar bucket. */ + timeZone: string; files: Map; manifest: UsageManifest | null; candidatesByMessage: Map>; @@ -175,6 +184,14 @@ export interface ClaudeUsageDashboardSnapshot extends ClaudeUsageAggregateSnapsh workflows: WorkflowUsage[]; costliestMessages: CostlyMessage[]; context: ContextWindowInfo | null; + /** Present only for the default-off advice capability; built from materialized aggregates. */ + adviceWindow?: { + windowDays: number; + aggregate: UsageData; + totalSessions: number; + longSessionCount: number; + largeContextSessionCount: number; + }; } interface FilePlan { @@ -241,6 +258,7 @@ function emptyCopyOnWriteKeys(): CopyOnWriteKeys { export function createClaudeUsageIndex(): ClaudeUsageIndex { return { + timeZone: resolveTimeZone(I18n.getTimezone()), files: new Map(), manifest: null, candidatesByMessage: new Map(), @@ -610,9 +628,14 @@ function cloneUsageData(value: UsageData): UsageData { }; } -function localDayKey(date: Date): string { - if (Number.isNaN(date.getTime())) return ''; - return `${date.getFullYear()}-${String(date.getMonth() + 1).padStart(2, '0')}-${String(date.getDate()).padStart(2, '0')}`; +function configuredTimeKeys( + index: ClaudeUsageIndex, + date: Date, +): { day: string; month: string; hour: string } { + const day = dayKeyInZone(date, index.timeZone); + const month = monthKeyInZone(date, index.timeZone); + const hour = formatHourLabel(hourKeyInZone(date, index.timeZone)); + return { day, month, hour }; } function addUsageData(target: UsageData, source: UsageData, sign: 1 | -1): void { @@ -669,29 +692,36 @@ function applyBucket( else buckets.set(key, value); } +function applyConfiguredTimeAggregate( + index: ClaudeUsageIndex, + record: ClaudeUsageRecord, + contribution: UsageData, + sign: 1 | -1, +): void { + const date = new Date(record.timestamp); + const { day, month, hour } = configuredTimeKeys(index, date); + const dayHour = day && hour + ? `${day}\0${hour}` + : ''; + applyBucket(index.aggregates.byDay, day, contribution, sign); + applyBucket(index.aggregates.byMonth, month, contribution, sign); + applyBucket(index.aggregates.byLocalDay, day, contribution, sign); + applyBucket(index.aggregates.byLocalHour, dayHour, contribution, sign); +} + function applyAggregate(index: ClaudeUsageIndex, record: ClaudeUsageRecord, sign: 1 | -1): void { const contribution = ClaudeDataLoader.calculateUsageData([record]); addUsageData(index.aggregates.allTime, contribution, sign); - const date = new Date(record.timestamp); - const day = dayKeyInZone(date, I18n.getTimezone()); - const month = monthKeyInZone(date, I18n.getTimezone()); - const localDay = localDayKey(date); - const localHour = localDay - ? `${localDay}\0${String(date.getHours()).padStart(2, '0')}:00` - : ''; + applyConfiguredTimeAggregate(index, record, contribution, sign); const project = record._projectPath || record._projectName || 'unknown'; const branch = record._gitBranch && record._gitBranch.trim() ? record._gitBranch : '-'; - applyBucket(index.aggregates.byDay, day, contribution, sign); - applyBucket(index.aggregates.byMonth, month, contribution, sign); - applyBucket(index.aggregates.byLocalDay, localDay, contribution, sign); - applyBucket(index.aggregates.byLocalHour, localHour, contribution, sign); applyBucket(index.aggregates.bySession, record._sessionId || 'unknown', contribution, sign); applyBucket(index.aggregates.byProject, project.toLowerCase(), contribution, sign); applyBucket(index.aggregates.byBranch, `${record._projectName || 'unknown'}\0${branch}`, contribution, sign); applyBucket(index.aggregates.byWorkflow, record._workflowId, contribution, sign); } -function membershipKeys(record: ClaudeUsageRecord): { +function membershipKeys(index: ClaudeUsageIndex, record: ClaudeUsageRecord): { session: string; project: string; branch: string; @@ -704,7 +734,13 @@ function membershipKeys(record: ClaudeUsageRecord): { const branchName = record._gitBranch && record._gitBranch.trim() ? record._gitBranch : '-'; const branch = `${record._projectName || 'unknown'}\0${branchName}`; const workflow = record._workflowId || (record._agentId ? `adhoc:${session}` : undefined); - return { session, project, branch, workflow, localDay: localDayKey(new Date(record.timestamp)) }; + return { + session, + project, + branch, + workflow, + localDay: dayKeyInZone(new Date(record.timestamp), index.timeZone), + }; } function writableSet( @@ -749,7 +785,7 @@ function removeMembership( } function markDirty(index: ClaudeUsageIndex, record: ClaudeUsageRecord): void { - const keys = membershipKeys(record); + const keys = membershipKeys(index, record); index.dirtyGroups.sessions.add(keys.session); index.dirtyGroups.projects.add(keys.project); index.dirtyGroups.branches.add(keys.branch); @@ -757,7 +793,7 @@ function markDirty(index: ClaudeUsageIndex, record: ClaudeUsageRecord): void { } function addVisibleMembership(index: ClaudeUsageIndex, visibleKey: string, record: ClaudeUsageRecord): void { - const keys = membershipKeys(record); + const keys = membershipKeys(index, record); addMembership(index.visibleKeysBySession, keys.session, visibleKey, index.copyOnWrite.visibleSessions); addMembership(index.visibleKeysByProject, keys.project, visibleKey, index.copyOnWrite.visibleProjects); addMembership(index.visibleKeysByBranch, keys.branch, visibleKey, index.copyOnWrite.visibleBranches); @@ -767,7 +803,7 @@ function addVisibleMembership(index: ClaudeUsageIndex, visibleKey: string, recor } function removeVisibleMembership(index: ClaudeUsageIndex, visibleKey: string, record: ClaudeUsageRecord): void { - const keys = membershipKeys(record); + const keys = membershipKeys(index, record); removeMembership(index.visibleKeysBySession, keys.session, visibleKey, index.copyOnWrite.visibleSessions); removeMembership(index.visibleKeysByProject, keys.project, visibleKey, index.copyOnWrite.visibleProjects); removeMembership(index.visibleKeysByBranch, keys.branch, visibleKey, index.copyOnWrite.visibleBranches); @@ -797,6 +833,26 @@ function setVisible( return 1; } +/** Rebucket already-materialized records after an explicit timezone change. */ +function rebuildConfiguredTimeAggregates(index: ClaudeUsageIndex): void { + index.aggregates.byDay = new Map(); + index.aggregates.byMonth = new Map(); + index.aggregates.byLocalDay = new Map(); + index.aggregates.byLocalHour = new Map(); + index.visibleKeysByLocalDay = new Map(); + index.copyOnWrite.visibleLocalDays = new Set(); + for (const [visibleKey, record] of index.visibleRecords) { + const contribution = ClaudeDataLoader.calculateUsageData([record]); + applyConfiguredTimeAggregate(index, record, contribution, 1); + addMembership( + index.visibleKeysByLocalDay, + dayKeyInZone(new Date(record.timestamp), index.timeZone), + visibleKey, + index.copyOnWrite.visibleLocalDays, + ); + } +} + function candidateMessageId(value: IndexedRecord): string | null { const messageId = value.record.message.id; return messageId ? String(messageId) : null; @@ -1189,8 +1245,8 @@ export function claudeUsageAggregateSnapshot( index: ClaudeUsageIndex, now: Date = new Date(), ): ClaudeUsageAggregateSnapshot { - const localToday = localDayKey(now); - const configuredMonth = monthKeyInZone(now, I18n.getTimezone()); + const localToday = dayKeyInZone(now, index.timeZone); + const configuredMonth = monthKeyInZone(now, index.timeZone); const dailyForMonth = [...index.aggregates.byDay.entries()] .filter(([day]) => day.startsWith(configuredMonth)) .map(([date, data]) => ({ date, data: cloneUsageData(data) })) @@ -1218,6 +1274,7 @@ export function claudeUsageDashboardSnapshot( workspacePath?: string; projectGroupingMode?: 'git' | 'folder' | 'flat'; contextWindowOverride?: number; + adviceWindowDays?: number; now?: Date; } = {}, ): ClaudeUsageDashboardSnapshot { @@ -1242,7 +1299,7 @@ export function claudeUsageDashboardSnapshot( if (options.workspacePath) { const todayRecords = recordsForKeys( index, - index.visibleKeysByLocalDay.get(localDayKey(now)), + index.visibleKeysByLocalDay.get(dayKeyInZone(now, index.timeZone)), ).filter((record) => recordMatchesWorkspace(record, options.workspacePath!)); workspaceToday = ClaudeDataLoader.calculateUsageData(todayRecords); } @@ -1271,6 +1328,30 @@ export function claudeUsageDashboardSnapshot( .flat() .sort((left, right) => right.cost - left.cost) .slice(0, 10); + let adviceWindow: ClaudeUsageDashboardSnapshot['adviceWindow']; + if (options.adviceWindowDays !== undefined) { + const windowDays = Math.min(365, Math.max(1, Math.round(options.adviceWindowDays))); + const dayKeys = new Set(rollingDayKeys(now.getTime(), index.timeZone, windowDays)); + const aggregate = emptyUsageData(); + for (const day of dayKeys) { + const value = index.aggregates.byDay.get(day); + if (value) addUsageData(aggregate, value, 1); + } + const windowSessions = [...index.sessionRows.values()].filter((row) => + dayKeys.has(dayKeyInZone(row.endTime, index.timeZone)), + ); + adviceWindow = { + windowDays, + aggregate, + totalSessions: windowSessions.length, + longSessionCount: windowSessions.filter( + (row) => row.endTime.getTime() - row.startTime.getTime() >= 8 * 60 * 60 * 1000, + ).length, + largeContextSessionCount: windowSessions.filter( + (row) => row.peakContextTokens >= 150_000, + ).length, + }; + } return { ...aggregates, session, @@ -1285,6 +1366,7 @@ export function claudeUsageDashboardSnapshot( .slice(0, 50), costliestMessages, context, + ...(adviceWindow ? { adviceWindow } : {}), }; } @@ -1294,6 +1376,8 @@ export async function updateClaudeUsageIndex( options: ClaudeUsageIndexUpdateOptions = {}, ): Promise { const started = performance.now(); + const configuredTimeZone = resolveTimeZone(I18n.getTimezone()); + const timeZoneChanged = previous.timeZone !== configuredTimeZone; const analyzeContent = options.analyzeContent !== false; const windowDays = Math.min(365, Math.max(1, Math.round(options.windowDays ?? 30))); const analysisCutoffMs = analyzeContent && previous.analyzeContent && @@ -1431,6 +1515,10 @@ export async function updateClaudeUsageIndex( } const next = cloneIndexForCommit(previous); + next.timeZone = configuredTimeZone; + if (timeZoneChanged) { + rebuildConfiguredTimeAggregates(next); + } next.manifest = manifest; next.analyzeContent = analyzeContent; next.windowDays = windowDays; @@ -1438,7 +1526,7 @@ export async function updateClaudeUsageIndex( const affectedMessages = new Set(); const affectedDirect = new Set(); const affectedSessions = new Set(); - let aggregateMutations = 0; + let aggregateMutations = timeZoneChanged ? next.visibleRecords.size : 0; const removeFile = (file: ClaudeUsageFileContribution): void => { next.files.delete(file.fileId); affectedSessions.add(ClaudeDataLoader.parseSessionInfo(file.path).sessionId); diff --git a/src/codexViewComponents.ts b/src/codexViewComponents.ts index 3e340574..7703b2c1 100644 --- a/src/codexViewComponents.ts +++ b/src/codexViewComponents.ts @@ -2,6 +2,7 @@ import { CodexInsightEvidenceKey, CodexInsightKind, } from './providers/codex/codexInsights'; +import type { BackgroundWorkReason } from './backgroundWorkState'; /** Copy contract retained for provider-aware renderers in webview.ts. */ export const CODEX_QUALITY_FLAGS = [ @@ -149,6 +150,7 @@ export interface CodexViewCopy { indexedAllTime: string; indexedSubtotal: string; indexingInProgress: string; + indexingReasons: Record; updatedAt: string; claudeTokenAccounting: string; codexTokenAccounting: string; @@ -289,6 +291,15 @@ export const CODEX_COPY_EN: CodexViewCopy = { observedSessionDuration: 'Elapsed span between the first and last observed events; a proxy, not actual active time.', indexedSubtotal: 'Indexed subtotal', indexingInProgress: 'Indexing is still in progress; unverified legacy totals are excluded.', + indexingReasons: { + 'first-index': 'First local history setup', + 'parser-migration': 'Updating the local parser index', + 'period-migration': 'Updating date history', + 'hourly-history': 'Building recent hourly history', + 'history-backfill': 'Completing local history', + 'rule-migration': 'Updating measurement rules', + resume: 'Resuming local history', + }, indexedLogEntries: 'Indexed log entries', indexedStorage: 'Indexed storage', indexedAllTime: 'Indexed all time', diff --git a/src/dataLoader.ts b/src/dataLoader.ts index 4c7297c6..f76cb531 100644 --- a/src/dataLoader.ts +++ b/src/dataLoader.ts @@ -6,7 +6,12 @@ import * as path from 'path'; // Removed zod dependency - using native validation instead import { calculateCostBreakdown, getModelRatesPerMillion } from './pricing'; import { isRetryDuplicatePrompt } from './promptDedup'; -import { dayKeyInZone, monthKeyInZone } from './dateKeys'; +import { + dayKeyInZone, + formatHourLabel, + hourKeyInZone, + monthKeyInZone, +} from './dateKeys'; import { I18n } from './i18n'; import { DayUsage } from './heatmap'; import { ShareInput, ShareRange, rangeLabel as shareRangeLabel } from './shareCard'; @@ -103,7 +108,7 @@ export interface AnalysisAcc { toolIdToName: Record; seenUuids: Set; cutoffMs: number; - prompts: { cwd: string; text: string }[]; + prompts: { cwd: string; text: string; observedAtEpochMs: number }[]; // Estimated thinking vs. total assistant-output tokens, per session and per // local day ("YYYY-MM-DD") — feeds the thinking-share column / Today line. thinkingBySession: Record; @@ -163,7 +168,12 @@ function localDayKey(timestamp: unknown): string { } // Collect an actual user prompt (capped + truncated) for the AI-advice feature. -function collectPrompt(acc: AnalysisAcc, cwd: string, text: string): void { +function collectPrompt( + acc: AnalysisAcc, + cwd: string, + text: string, + observedAtEpochMs: number, +): void { const trimmed = text.trim(); if (trimmed.length < 4) { return; @@ -171,7 +181,11 @@ function collectPrompt(acc: AnalysisAcc, cwd: string, text: string): void { // The caller has already applied classifyPromptTextOrigin. Do not repeat a // looser tag heuristic here: unknown custom elements may be user-authored // Web Component examples and must remain eligible for explicit opt-in. - acc.prompts.push({ cwd, text: trimmed.slice(0, 2500) }); + acc.prompts.push({ + cwd, + text: trimmed.slice(0, 2500), + observedAtEpochMs: Number.isFinite(observedAtEpochMs) ? observedAtEpochMs : 0, + }); if (acc.prompts.length > 600) { acc.prompts.shift(); } @@ -404,7 +418,7 @@ export function analyzeLine(parsed: any, acc: AnalysisAcc, isSubagentFile = fals addToBucket(acc.cat, 'userPrompts', content); collectCommandUse(acc, content, sessionId, parsed.timestamp); if (allowPromptSample && origin === 'user-authored') { - collectPrompt(acc, cwd, content); + collectPrompt(acc, cwd, content, Date.parse(parsed.timestamp)); } } else if (Array.isArray(content)) { for (const block of content) { @@ -435,7 +449,7 @@ export function analyzeLine(parsed: any, acc: AnalysisAcc, isSubagentFile = fals addToBucket(acc.cat, 'userPrompts', block.text); collectCommandUse(acc, block.text, sessionId, parsed.timestamp); if (allowPromptSample && origin === 'user-authored') { - collectPrompt(acc, cwd, block.text); + collectPrompt(acc, cwd, block.text, Date.parse(parsed.timestamp)); } } } @@ -1410,13 +1424,11 @@ export class ClaudeDataLoader { } static getTodayData(records: ClaudeUsageRecord[]): UsageData { - const today = new Date(); - today.setHours(0, 0, 0, 0); - - const todayRecords = records.filter((record) => { - const recordDate = new Date(record.timestamp); - return recordDate >= today; - }); + const tz = I18n.getTimezone(); + const today = dayKeyInZone(new Date(), tz); + const todayRecords = records.filter((record) => + dayKeyInZone(new Date(record.timestamp), tz) === today + ); return this.calculateUsageData(todayRecords); } @@ -2523,13 +2535,13 @@ export class ClaudeDataLoader { static getUsageAttribution( records: ClaudeUsageRecord[], analysis: ContentAnalysis | null, - scope: AttributionScope + scope: AttributionScope, + now: Date = new Date(), ): UsageAttribution { - const now = new Date(); - const startOfDay = new Date(now.getFullYear(), now.getMonth(), now.getDate()).getTime(); + const timeZone = I18n.getTimezone(); + const configuredToday = dayKeyInZone(now, timeZone); const minTs = - scope.kind === 'day' ? startOfDay - : scope.kind === 'week' ? now.getTime() - 7 * 24 * 60 * 60 * 1000 + scope.kind === 'week' ? now.getTime() - 7 * 24 * 60 * 60 * 1000 : scope.kind === 'month' ? now.getTime() - 30 * 24 * 60 * 60 * 1000 : 0; const normScope = scope.projectPath ? this.normalizePath(scope.projectPath) : ''; @@ -2545,7 +2557,12 @@ export class ClaudeDataLoader { return this.normalizePath(r._projectPath || '').startsWith(normScope); } const t = Date.parse(r.timestamp); - return !isNaN(t) && t >= minTs; + if (isNaN(t)) { + return false; + } + return scope.kind === 'day' + ? dayKeyInZone(new Date(t), timeZone) === configuredToday + : t >= minTs; }); // Skill / plugin activation points. A skill's usage share is the weight @@ -2568,7 +2585,9 @@ export class ClaudeDataLoader { uses = uses.filter((u) => sessionIds.has(u.sessionId)); } else { // "YYYY-MM-DD" compares correctly as a string. - const minDay = localDayKey(new Date(minTs).toISOString()); + const minDay = scope.kind === 'day' + ? configuredToday + : localDayKey(new Date(minTs).toISOString()); uses = uses.filter((u) => u.day >= minDay); } // key → session → earliest invocation ts (skills and plugins separately) @@ -3047,20 +3066,21 @@ export class ClaudeDataLoader { } static getHourlyDataForToday(records: ClaudeUsageRecord[]): { hour: string; data: UsageData }[] { - const today = new Date(); - today.setHours(0, 0, 0, 0); - - const todayRecords = records.filter((record) => { - const recordDate = new Date(record.timestamp); - return recordDate >= today; - }); + const tz = I18n.getTimezone(); + const today = dayKeyInZone(new Date(), tz); + const todayRecords = records.filter((record) => + dayKeyInZone(new Date(record.timestamp), tz) === today + ); // Group records by hour const recordsByHour: Record = {}; todayRecords.forEach((record) => { const recordDate = new Date(record.timestamp); - const hourKey = `${recordDate.getHours().toString().padStart(2, '0')}:00`; // HH:00 format + const hourKey = formatHourLabel(hourKeyInZone(recordDate, tz)); + if (!hourKey) { + return; + } if (!recordsByHour[hourKey]) { recordsByHour[hourKey] = []; @@ -3080,23 +3100,20 @@ export class ClaudeDataLoader { } static getHourlyDataForDate(records: ClaudeUsageRecord[], dateString: string): { hour: string; data: UsageData }[] { - const targetDate = new Date(dateString); - targetDate.setHours(0, 0, 0, 0); - - const nextDate = new Date(targetDate); - nextDate.setDate(nextDate.getDate() + 1); - - const dateRecords = records.filter((record) => { - const recordDate = new Date(record.timestamp); - return recordDate >= targetDate && recordDate < nextDate; - }); + const tz = I18n.getTimezone(); + const dateRecords = records.filter((record) => + dayKeyInZone(new Date(record.timestamp), tz) === dateString + ); // Group records by hour const recordsByHour: Record = {}; dateRecords.forEach((record) => { const recordDate = new Date(record.timestamp); - const hourKey = `${recordDate.getHours().toString().padStart(2, '0')}:00`; // HH:00 format + const hourKey = formatHourLabel(hourKeyInZone(recordDate, tz)); + if (!hourKey) { + return; + } if (!recordsByHour[hourKey]) { recordsByHour[hourKey] = []; diff --git a/src/dateKeys.ts b/src/dateKeys.ts index 8b3ac0f5..f4d01a0f 100644 --- a/src/dateKeys.ts +++ b/src/dateKeys.ts @@ -55,6 +55,11 @@ export function hourKeyInZone(date: Date, timeZone: string): string { ?.value ?? ''; } +/** Provider-neutral display label for a validated "00" through "23" hour key. */ +export function formatHourLabel(hour: string): string { + return /^(?:[01]\d|2[0-3])$/.test(hour) ? `${hour}:00` : ''; +} + /** * Calendar-day keys ending on `now` in `timeZone`, ordered from oldest to newest. * The range walks civil dates rather than subtracting fixed-duration milliseconds. diff --git a/src/extension.ts b/src/extension.ts index 1ae1b69e..43bd15f1 100644 --- a/src/extension.ts +++ b/src/extension.ts @@ -22,12 +22,7 @@ import { fetchLatestPricing } from './pricing'; import { ClaudeApiClient } from './claudeApiClient'; import { buildOptimizerSystemPrompt, - callModel, - getUsageAdvice, - parseOptimizerOutput } from './advisor'; -import { buildAdviceSummary } from './adviceSummary'; -import { getDemoBody } from './adviceDemoSample'; import { ClaudeApiUsageResponse, ContentAnalysis, ExtensionConfig } from './types'; import { SettingsStore } from './settings'; import { normalizeQuotaWindows } from './quotaWindows'; @@ -80,14 +75,46 @@ import { adaptClaudeAdvice, adaptCodexLocalAdvice, } from './adviceEffectiveness/adapters'; -import { AdviceEffectivenessProviderStates } from './adviceEffectiveness/integration'; +import { + AdviceEffectivenessProviderStates, + selectAdvicePromptSamples, +} from './adviceEffectiveness/integration'; import { buildAdviceAggregateSnapshot } from './adviceEffectiveness/payload'; +import { + prepareStructuredAdviceInvocation, + requestStructuredAdvice, +} from './adviceEffectiveness/remoteAdvice'; +import { + prepareOptimizerInvocation, + requestPreparedOptimizer, +} from './optimizerRequest'; +import { + BackgroundWorkReason, + BackgroundWorkState, + beginBackgroundWork, + createBackgroundWorkState, + interruptBackgroundWork, + recordBackgroundWorkFailure, + recordBackgroundWorkProgress, + restoreBackgroundWorkState, +} from './backgroundWorkState'; +import { + ResourceLease, + ResourceOwnershipRegistry, + ResourceStopCondition, +} from './resourceOwnership'; interface LocalizedReleaseAnnouncement { version: string; body: () => string; } +interface ActiveNetworkOperation { + readonly lease: ResourceLease; + /** Resolves only after the request promise has reached its own terminal path. */ + readonly settled: Promise; +} + // Full-version entries only: an installed patch must never inherit stale notes // from an older major/minor release. const WHATS_NEW: ReleaseAnnouncementCatalog = { @@ -98,16 +125,23 @@ const WHATS_NEW: ReleaseAnnouncementCatalog = { }; export class ClaudeCodeUsageExtension { + private static readonly CODEX_BACKGROUND_WORK_STATE_KEY = + 'ccu.codex.backgroundWork.v1'; + private static readonly CODEX_BACKGROUND_MEASUREMENT_VERSION = 1; private statusBar: StatusBarManager; private webviewProvider: UsageWebviewProvider; private apiClient: ClaudeApiClient; private settings: SettingsStore; private refreshTimer: NodeJS.Timeout | undefined; + private refreshTimerLease: ResourceLease | undefined; private fileWatcher: fs.FSWatcher | undefined; + private fileWatcherLease: ResourceLease | undefined; private codexWatchers: fs.FSWatcher[] = []; - private readonly codexWatchDebounce = new QuietDebounce(); + private readonly codexWatcherLeases = new Map(); + private readonly debounceTimerLeases = new Map(); + private readonly codexWatchDebounce = this.createOwnedRefreshDebounce('codex'); private codexWatchedHome: string | null = null; - private readonly watchDebounce = new QuietDebounce(); + private readonly watchDebounce = this.createOwnedRefreshDebounce('claude'); private readonly refreshGate = new RefreshSingleFlight(); private readonly windowActivity = new WindowActivityGate(vscode.window.state.focused); @@ -117,7 +151,9 @@ export class ClaudeCodeUsageExtension { // Watches ~/.claude/.credentials.json so an account switch is reflected // promptly instead of after a full quota TTL (#45). private credsWatcher: fs.FSWatcher | undefined; + private credsWatcherLease: ResourceLease | undefined; private credsDebounceTimer: NodeJS.Timeout | undefined; + private credsDebounceTimerLease: ResourceLease | undefined; private cache: { records: any[]; contentAnalysis: ContentAnalysis | null; @@ -154,6 +190,8 @@ export class ClaudeCodeUsageExtension { // flaky network and the very first /usage fetch fails, try once more shortly // after so the indicator appears without waiting for the next regular tick. private quotaColdRetryDone: boolean = false; + private quotaColdRetryTimer: NodeJS.Timeout | undefined; + private quotaColdRetryTimerLease: ResourceLease | undefined; private claudeProfileGeneration: number = 0; private claudeWeeklyQuotaHistory: WeeklyQuotaObservation[] = []; private codexProvider: CodexProvider; @@ -167,6 +205,26 @@ export class ClaudeCodeUsageExtension { private codexProgressLastRenderedAt = 0; private codexCheckpointHydration: Promise | null = null; private codexCheckpointHydrationLastAttemptAt = 0; + private codexBackgroundState: BackgroundWorkState; + private readonly resourceOwnership = new ResourceOwnershipRegistry(); + private codexBackfillLease: ResourceLease | undefined; + private codexWorkerLease: ResourceLease | undefined; + private codexFirstBackfillActive = false; + private codexWorkerCancellationRequested = false; + private readonly activeAdviceNetworks = new Map(); + private readonly activeQuotaNetworks = new Map(); + private readonly codexProviderRetirements = new Set>(); + private codexProviderRetirementFailure: unknown = null; + private readonly activeCodexRefreshes = new Set>(); + private readonly pendingResourceStops = new Set>(); + private resourceStopFailure: unknown = null; + private codexBackgroundStateWrite: Promise = Promise.resolve(); + private configurationGeneration = 0; + private fileWatcherGeneration = 0; + private codexWatcherGeneration = 0; + private credentialsWatcherGeneration = 0; + private disposed = false; + private disposal: Promise | null = null; constructor(private context: vscode.ExtensionContext) { console.log('Claude Code Usage Extension: Constructor called'); @@ -174,6 +232,30 @@ export class ClaudeCodeUsageExtension { context.subscriptions.push(this.outputChannel); this.statusBar = new StatusBarManager(); this.settings = new SettingsStore(context); + const backgroundRestore = restoreBackgroundWorkState( + context.globalState.get( + ClaudeCodeUsageExtension.CODEX_BACKGROUND_WORK_STATE_KEY, + ), + { + measurementVersion: + ClaudeCodeUsageExtension.CODEX_BACKGROUND_MEASUREMENT_VERSION, + reason: 'history-backfill', + now: Date.now(), + }, + ); + this.codexBackgroundState = backgroundRestore.state; + if (this.codexBackgroundState.status === 'running') { + this.codexBackgroundState = interruptBackgroundWork( + this.codexBackgroundState, + { now: Date.now() }, + ); + } + if ( + backgroundRestore.disposition !== 'valid' || + backgroundRestore.state.status === 'running' + ) { + void this.saveCodexBackgroundState().catch(() => undefined); + } this.webviewProvider = new UsageWebviewProvider(context); this.apiClient = new ClaudeApiClient( this.outputChannel, @@ -197,7 +279,69 @@ export class ClaudeCodeUsageExtension { // Usage Optimizer (Phase 9c): the webview posts a draft prompt; we run it // through the same model backend as the advice feature and post back a // tightened prompt + a settings recommendation. Consent gate lives here. - this.webviewProvider.onOptimize = (draft, options) => this.runOptimizer(draft, options); + this.webviewProvider.onPrepareOptimizerInvocation = ( + draft, + options, + sourceRevision, + consentGeneration, + ) => this.prepareOptimizerRequest( + draft, + options, + sourceRevision, + consentGeneration, + ); + this.webviewProvider.onSendOptimizerInvocation = ( + prepared, + expectedSourceRevision, + expectedConsentGeneration, + ) => { + const config = this.getConfiguration(); + return this.runAdviceNetwork((signal) => + requestPreparedOptimizer(prepared, { + apiKey: config.adviceApiKey, + expectedSourceRevision, + expectedConsentGeneration, + signal, + }), + ); + }; + this.webviewProvider.onAiSurfaceClosed = () => { + void this.cancelAdviceNetworks('cancelled'); + }; + this.webviewProvider.onAdviceDataCleared = () => + this.cancelAdviceNetworks('cancelled'); + this.webviewProvider.onPrepareAdviceInvocation = ( + snapshot, + sourceRevision, + consentGeneration, + ) => { + const config = this.getConfiguration(); + return prepareStructuredAdviceInvocation(snapshot.prepared, { + apiFormat: config.adviceApiFormat, + apiUrl: config.adviceApiUrl, + model: config.adviceModel, + reasoningEffort: config.adviceReasoningEffort, + sourceRevision, + consentGeneration, + createdAtEpochMs: Date.now(), + }); + }; + this.webviewProvider.onSendAdviceInvocation = ( + prepared, + references, + expectedSourceRevision, + expectedConsentGeneration, + ) => { + const config = this.getConfiguration(); + return this.runAdviceNetwork((signal) => + requestStructuredAdvice(prepared, references, { + apiKey: config.adviceApiKey, + expectedSourceRevision, + expectedConsentGeneration, + signal, + }), + ); + }; // Share the settings store with the dashboard's ⚙ Settings panel, and have // it tell us when the user changes a setting there so we re-apply config // (globalState changes don't fire onDidChangeConfiguration). @@ -209,7 +353,14 @@ export class ClaudeCodeUsageExtension { this.loadPersistedQuota(); if (this.windowActivity.focused) { this.startAutoRefresh(); + const startupGeneration = this.configurationGeneration; void this.refreshData(false, 'startup').then(() => { + if ( + this.disposed || + startupGeneration !== this.configurationGeneration + ) { + return; + } void this.startFileWatching(); this.startCodexWatching(); }); @@ -549,123 +700,27 @@ export class ClaudeCodeUsageExtension { } private async getAdvice(): Promise { - const config = this.getConfiguration(); - // Runtime advice is API/BYOK-only. An empty user key opens the local demo; - // Claude Code subscription OAuth remains reserved for quota observation. - const needsKey = - config.adviceBackend === 'api' && (!config.adviceApiKey || config.adviceApiKey.trim() === ''); - if (needsKey) { - const picked = await vscode.window.showWarningMessage( - I18n.t.popup.adviceNeedsKey, + // The legacy command now enters the one evidence/preview/send surface. It + // never constructs or sends an alternate free-form summary. + this.webviewProvider.show('content'); + if (!this.settings.get('advice.effectiveness.enabled')) { + const picked = await vscode.window.showInformationMessage( + I18n.t.popup.adviceEffectiveness.description, I18n.t.popup.settings, - I18n.t.popup.adviceDemoButton ); if (picked === I18n.t.popup.settings) { vscode.commands.executeCommand('claudeCodeUsage.openSettings'); - } else if (picked === I18n.t.popup.adviceDemoButton) { - await this.openAdviceDemo(); } - return; - } - - const records = this.cache.records; - const analysis = this.cache.contentAnalysis; - if (!records || records.length === 0 || !analysis) { - vscode.window.showWarningMessage(I18n.t.popup.noDataMessage); - return; - } - - // Let the user scope the advice to everything, or to one project. - const projects = ClaudeDataLoader.getProjectBreakdown(records); - const items: (vscode.QuickPickItem & { scope: string })[] = [ - { label: I18n.t.popup.adviceScopeOverall, scope: 'overall' }, - ...projects.map((p) => ({ label: p.groupName, description: p.groupPath, scope: p.groupPath })) - ]; - const picked = await vscode.window.showQuickPick(items, { placeHolder: I18n.t.popup.adviceScopePrompt }); - if (!picked) { - return; } - - const summary = buildAdviceSummary( - records, - analysis, - picked.scope, - picked.label, - config.advicePromptWindowDays - ); - - await this.runAdviceRequest(config, picked.scope, picked.label, summary); - } - - private async openAdviceDemo(): Promise { - const now = new Date(); - const pad = (n: number): string => String(n).padStart(2, '0'); - const stamp = - `${now.getFullYear()}-${pad(now.getMonth() + 1)}-${pad(now.getDate())}` + - `_${pad(now.getHours())}${pad(now.getMinutes())}`; - const uri = vscode.Uri.parse(`untitled:claude-advice-DEMO-${stamp}.md`); - const doc = await vscode.workspace.openTextDocument(uri); - const editor = await vscode.window.showTextDocument(doc); - const lang = I18n.getCurrentLanguage(); - const banner = I18n.t.popup.adviceDemoNotice; - const body = getDemoBody(lang); - const content = `${banner}\n\n---\n\n${body}`; - await editor.edit((eb) => eb.insert(new vscode.Position(0, 0), content)); - } - - private async runAdviceRequest( - config: ExtensionConfig, - scope: string, - label: string, - summary: string - ): Promise { - await vscode.window.withProgress( - { location: vscode.ProgressLocation.Notification, title: I18n.t.popup.adviceGenerating }, - async () => { - try { - const advice = await getUsageAdvice({ - backend: config.adviceBackend, - apiFormat: config.adviceApiFormat, - apiKey: config.adviceApiKey, - apiUrl: config.adviceApiUrl, - model: config.adviceModel, - reasoningEffort: config.adviceReasoningEffort, - userContext: config.adviceUserContext, - language: I18n.getLanguageName(), - summary - }); - - // Give the document a distinguishable name like - // claude-advice--YYYY-MM-DD_HHmm.md so different runs are easy - // to tell apart in the tab strip. - const now = new Date(); - const pad = (n: number): string => String(n).padStart(2, '0'); - const stamp = - `${now.getFullYear()}-${pad(now.getMonth() + 1)}-${pad(now.getDate())}` + - `_${pad(now.getHours())}${pad(now.getMinutes())}`; - const safeScope = - scope === 'overall' - ? 'overall' - : (label || 'project').replace(/[^A-Za-z0-9_-]+/g, '_').slice(0, 30) || 'project'; - const uri = vscode.Uri.parse(`untitled:claude-advice-${safeScope}-${stamp}.md`); - const doc = await vscode.workspace.openTextDocument(uri); - const editor = await vscode.window.showTextDocument(doc); - await editor.edit((eb) => eb.insert(new vscode.Position(0, 0), advice)); - } catch (error) { - const message = error instanceof Error ? error.message : String(error); - vscode.window.showErrorMessage(`${I18n.t.popup.adviceFailed}: ${message}`); - } - } - ); } /** * Usage Optimizer round-trip (Phase 9c). Takes the user's rough draft and the * three optional lenses, asks the configured model to return a tightened * paste-ready prompt plus a settings recommendation, and parses the two - * sections out. ONLY the pasted draft is sent — no filesystem access. First - * use shows a one-time consent modal (the text is going to a model, not to - * Claude Code's terminal). + * sections out. ONLY the pasted draft enters the sealed preview — no + * filesystem access. A separate Send click is the sole authorization for the + * prepared bytes to leave the machine. */ /** Distinct models the user actually uses — Claude reduced to family names * (haiku/sonnet/opus/fable), third-party models kept as-is — so the optimizer @@ -703,30 +758,17 @@ export class ClaudeCodeUsageExtension { return out.slice(0, 8); } - private async runOptimizer( + private async prepareOptimizerRequest( draft: string, - options: { resolve: boolean; distil: boolean; aesthetic: boolean } - ): Promise<{ prompt?: string; settings?: string; error?: string }> { + options: { resolve: boolean; distil: boolean; aesthetic: boolean }, + sourceRevision: string, + consentGeneration: number, + ): Promise<{ prepared?: ReturnType; error?: string }> { const text = (draft || '').trim(); if (text === '') { return { error: I18n.t.popup.noDataMessage }; } - // One-time consent: the draft leaves the machine for whichever model the - // advice backend points at. Remember the choice in globalState. - const consentKey = 'claudeCodeUsage.optimizerConsented'; - if (!this.context.globalState.get(consentKey, false)) { - const proceed = await vscode.window.showWarningMessage( - I18n.t.popup.optimizerConsent, - { modal: true }, - I18n.t.popup.optimizerRun - ); - if (proceed !== I18n.t.popup.optimizerRun) { - return { error: '' }; - } - await this.context.globalState.update(consentKey, true); - } - const config = this.getConfiguration(); const needsKey = config.adviceBackend === 'api' && (!config.adviceApiKey || config.adviceApiKey.trim() === ''); @@ -738,18 +780,18 @@ export class ClaudeCodeUsageExtension { const systemPrompt = buildOptimizerSystemPrompt(language, options, this.usedModelNames()); try { - const raw = await callModel(systemPrompt, text, { - backend: config.adviceBackend, + return { prepared: prepareOptimizerInvocation({ apiFormat: config.adviceApiFormat, - apiKey: config.adviceApiKey, apiUrl: config.adviceApiUrl, model: config.adviceModel, reasoningEffort: config.adviceReasoningEffort, - language, - summary: '', - timeoutMs: 90_000 - }); - return parseOptimizerOutput(raw); + systemPrompt, + draft: text, + sourceRevision, + consentGeneration, + createdAtEpochMs: Date.now(), + timeoutMs: 90_000, + }) }; } catch (error) { const message = error instanceof Error ? error.message : String(error); return { error: `${I18n.t.popup.adviceFailed}: ${message}` }; @@ -876,8 +918,17 @@ export class ClaudeCodeUsageExtension { totalFiles: this.codexProgress.totalFiles, indexedBytes: this.codexProgress.indexedBytes, totalBytes: this.codexProgress.totalBytes, + reason: this.codexBackgroundState.reason, } - : null, + : this.codexRefreshing + ? { + scannedFiles: this.codexView?.coverage.indexedFiles ?? 0, + totalFiles: this.codexView?.coverage.totalFiles ?? 0, + indexedBytes: this.codexView?.coverage.indexedBytes ?? 0, + totalBytes: this.codexView?.coverage.totalBytes ?? 0, + reason: this.codexBackgroundState.reason, + } + : null, }, ); @@ -905,8 +956,8 @@ export class ClaudeCodeUsageExtension { /** * Build the default-off experimental evidence view from narrow, numeric - * inputs. Raw records are used only here to form a same-window aggregate; - * neither provider adapter accepts records, paths, session IDs, or prompts. + * inputs already materialized by each provider index. Neither provider + * adapter accepts records, paths, session IDs, or prompt bodies. */ private buildAdviceEffectivenessProviderStates( config: ExtensionConfig, @@ -919,20 +970,19 @@ export class ClaudeCodeUsageExtension { const generatedAt = new Date(now).toISOString(); const epochDay = Math.floor(now / 86_400_000); const windowDays = Math.max(1, Math.round(config.advicePromptWindowDays)); - const cutoff = now - windowDays * 86_400_000; try { - const windowRecords = this.cache.records.filter((record) => { - const timestamp = typeof record?.timestamp === 'string' ? Date.parse(record.timestamp) : NaN; - return Number.isFinite(timestamp) && timestamp >= cutoff && timestamp <= now; + const materialized = claudeUsageDashboardSnapshot(this.cache.claudeIndex, { + now: new Date(now), + adviceWindowDays: windowDays, }); - if (windowRecords.length > 0) { + const adviceWindow = materialized.adviceWindow; + if (adviceWindow && adviceWindow.aggregate.messageCount > 0) { const aggregate = buildAdviceAggregateSnapshot( - ClaudeDataLoader.getAllTimeData(windowRecords), + adviceWindow.aggregate, 'overall', windowDays, ); - const sessions = ClaudeDataLoader.getSessionBreakdown(windowRecords); const framework = this.cache.contentAnalysis?.frameworkOverhead; const adapted = adaptClaudeAdvice({ adviceId: `advice-claude-${windowDays}d-${epochDay}`, @@ -942,15 +992,9 @@ export class ClaudeCodeUsageExtension { sessionSummary: { scope: 'overall', windowDays, - totalSessions: sessions.length, - longSessionCount: sessions.filter( - (session) => - session.endTime.getTime() - session.startTime.getTime() >= - 8 * 60 * 60 * 1000, - ).length, - largeContextSessionCount: sessions.filter( - (session) => session.peakContextTokens >= 150_000, - ).length, + totalSessions: adviceWindow.totalSessions, + longSessionCount: adviceWindow.longSessionCount, + largeContextSessionCount: adviceWindow.largeContextSessionCount, }, ...(framework ? { @@ -968,9 +1012,12 @@ export class ClaudeCodeUsageExtension { contract: adapted.value.contract, remotePreviewEligible: adapted.value.remoteEvidenceEligible, aggregate: adapted.value.aggregate, - promptSamples: (this.cache.contentAnalysis?.recentPrompts ?? []).map((prompt) => ({ - text: prompt.text, - })), + userContext: config.adviceUserContext, + promptSamples: selectAdvicePromptSamples( + this.cache.contentAnalysis?.recentPrompts ?? [], + now, + windowDays, + ), }; } } @@ -1018,9 +1065,286 @@ export class ClaudeCodeUsageExtension { return states; } + private codexHistoricalWorkPending(snapshot: CodexProviderSnapshot | null): boolean { + if (!snapshot) return true; + const coverage = snapshot.coverage; + return !coverage.complete || + !coverage.identity.complete || + !coverage.period.allTime.complete || + !snapshot.hourlyCoverage?.complete; + } + + private codexBackgroundReason(snapshot: CodexProviderSnapshot | null): BackgroundWorkReason { + if (!snapshot || snapshot.coverage.totalFiles === 0) return 'first-index'; + if (!snapshot.coverage.period.allTime.complete) return 'period-migration'; + if (!snapshot.hourlyCoverage?.complete) return 'hourly-history'; + return 'history-backfill'; + } + + private codexBackgroundProgress(snapshot: CodexProviderSnapshot): BackgroundWorkState['progress'] { + const main = snapshot.coverage; + const period = main.period.allTime; + const hourly = snapshot.hourlyCoverage; + return { + completedUnits: main.indexedFiles + period.migratedFiles + (hourly?.indexedFiles ?? 0), + totalUnits: main.totalFiles + period.totalFiles + (hourly?.totalFiles ?? 0), + completedBytes: main.indexedBytes + period.migratedBytes + (hourly?.indexedBytes ?? 0), + totalBytes: main.totalBytes + period.totalBytes + (hourly?.totalBytes ?? 0), + }; + } + + private async saveCodexBackgroundState(): Promise { + const snapshot: BackgroundWorkState = { + ...this.codexBackgroundState, + progress: { ...this.codexBackgroundState.progress }, + }; + const previous = this.codexBackgroundStateWrite; + const write = previous + .catch(() => undefined) + .then(() => this.context.globalState.update( + ClaudeCodeUsageExtension.CODEX_BACKGROUND_WORK_STATE_KEY, + snapshot, + )); + this.codexBackgroundStateWrite = write; + await write; + } + + private trackResourceStop(stop: Promise): void { + let tracked!: Promise; + tracked = stop + .then(() => undefined) + .catch((error: unknown) => { + this.resourceStopFailure ??= error; + throw error; + }) + .finally(() => this.pendingResourceStops.delete(tracked)); + this.pendingResourceStops.add(tracked); + // The owning lifecycle later drains and reports the failure. Attaching a + // handler here prevents a fire-and-forget watcher close from becoming an + // unhandled rejection before disposal reaches that drain point. + void tracked.catch(() => undefined); + } + + private async drainResourceStops(): Promise { + while (this.pendingResourceStops.size > 0) { + await Promise.allSettled([...this.pendingResourceStops]); + } + if (this.resourceStopFailure) { + const failure = this.resourceStopFailure; + this.resourceStopFailure = null; + throw failure; + } + } + + private async runAdviceNetwork( + request: (signal: AbortSignal) => Promise, + ): Promise { + if (this.disposed) { + throw new Error('Extension is disposed'); + } + const controller = new AbortController(); + const lease = this.resourceOwnership.register({ + kind: 'network', + capability: 'advice-personalization', + scope: 'advice', + creator: 'advice-runtime', + stopConditions: [ + 'settled', + 'cancelled', + 'feature-disabled', + 'extension-dispose', + 'settings-change', + ], + boundedException: 'none', + }); + let markSettled!: () => void; + const settled = new Promise((resolve) => { + markSettled = resolve; + }); + this.activeAdviceNetworks.set(controller, { lease, settled }); + try { + return await request(controller.signal); + } finally { + this.activeAdviceNetworks.delete(controller); + markSettled(); + if (lease.active) { + await lease.stop(controller.signal.aborted ? 'cancelled' : 'settled', () => undefined); + } + } + } + + private async cancelAdviceNetworks( + condition: Extract< + ResourceStopCondition, + 'cancelled' | 'feature-disabled' | 'extension-dispose' | 'settings-change' + >, + ): Promise { + const active = [...this.activeAdviceNetworks.entries()]; + await Promise.all(active.map(async ([controller, operation]) => { + const { lease, settled } = operation; + if (lease.active) { + await lease.stop(condition, async () => { + controller.abort(); + await settled; + }); + } + this.activeAdviceNetworks.delete(controller); + })); + } + + private async cancelQuotaNetworks( + condition: Extract< + ResourceStopCondition, + | 'cancelled' + | 'window-blur' + | 'feature-disabled' + | 'extension-dispose' + | 'settings-change' + | 'profile-change' + >, + ): Promise { + const active = [...this.activeQuotaNetworks.entries()]; + await Promise.all(active.map(async ([controller, operation]) => { + const { lease, settled } = operation; + if (lease.active) { + await lease.stop(condition, async () => { + controller.abort(); + await settled; + }); + } + this.activeQuotaNetworks.delete(controller); + })); + } + + private createOwnedRefreshDebounce( + scope: 'claude' | 'codex', + ): QuietDebounce { + return new QuietDebounce( + (callback, ms) => { + if (this.disposed) { + return undefined as unknown as NodeJS.Timeout; + } + let timer!: NodeJS.Timeout; + timer = setTimeout(() => { + const lease = this.debounceTimerLeases.get(timer); + this.debounceTimerLeases.delete(timer); + if (lease?.active) { + this.trackResourceStop(lease.stop('settled', () => undefined)); + } + if (!this.disposed) callback(); + }, ms); + const lease = this.resourceOwnership.register({ + kind: 'timer', + capability: 'refresh', + scope, + creator: 'refresh-coordinator', + stopConditions: [ + 'settled', + 'cancelled', + 'window-blur', + 'feature-disabled', + 'extension-dispose', + 'settings-change', + ], + boundedException: 'none', + }); + this.debounceTimerLeases.set(timer, lease); + return timer; + }, + (timer) => { + if (!timer) return; + const lease = this.debounceTimerLeases.get(timer); + this.debounceTimerLeases.delete(timer); + if (lease?.active) { + this.trackResourceStop(lease.stop('cancelled', () => clearTimeout(timer))); + } + else clearTimeout(timer); + }, + ); + } + + private scheduleQuotaColdRetry(): void { + if ( + this.disposed || + this.quotaColdRetryTimer || + !this.windowActivity.focused || + !this.getConfiguration().usageLimitTracking + ) { + return; + } + this.quotaColdRetryTimer = setTimeout(() => { + const lease = this.quotaColdRetryTimerLease; + this.quotaColdRetryTimer = undefined; + this.quotaColdRetryTimerLease = undefined; + if (lease?.active) { + this.trackResourceStop(lease.stop('settled', () => undefined)); + } + if ( + this.disposed || + !this.windowActivity.focused || + !this.getConfiguration().usageLimitTracking + ) { + return; + } + void this.maybeFetchUsageLimits(this.getConfiguration()).then((retry) => { + if (retry) { + this.statusBar.updateQuota(retry); + this.webviewProvider.updateQuota(retry); + } + }); + }, 8_000); + this.quotaColdRetryTimerLease = this.resourceOwnership.register({ + kind: 'timer', + capability: 'quota', + scope: 'claude', + creator: 'refresh-coordinator', + stopConditions: [ + 'settled', + 'window-blur', + 'feature-disabled', + 'extension-dispose', + 'settings-change', + 'profile-change', + ], + boundedException: 'none', + }); + } + + private stopQuotaColdRetry( + condition: Extract< + ResourceStopCondition, + | 'window-blur' + | 'feature-disabled' + | 'extension-dispose' + | 'settings-change' + | 'profile-change' + >, + ): void { + const timer = this.quotaColdRetryTimer; + const lease = this.quotaColdRetryTimerLease; + this.quotaColdRetryTimer = undefined; + this.quotaColdRetryTimerLease = undefined; + if (timer) { + if (lease?.active) { + this.trackResourceStop(lease.stop(condition, () => clearTimeout(timer))); + } + else clearTimeout(timer); + } + } + private async refreshCodexData(trigger: RefreshTrigger): Promise { + if (this.disposed) return; + const generation = this.configurationGeneration; try { - await this.runCodexRefresh(trigger); + await this.waitForCodexProviderRetirements(); + } catch { + return; + } + if (this.disposed || generation !== this.configurationGeneration) return; + const operation = this.runCodexRefresh(trigger); + this.activeCodexRefreshes.add(operation); + try { + await operation; } catch { this.codexView = null; this.codexInsights = emptyCodexScopedInsights(); @@ -1028,6 +1352,7 @@ export class ClaudeCodeUsageExtension { this.codexRefreshing = false; this.codexProgress = null; this.codexProgressLastRenderedAt = 0; + if (this.disposed) return; this.outputChannel.appendLine( formatCodexIndexDiagnostic({ outcome: 'error', @@ -1046,12 +1371,41 @@ export class ClaudeCodeUsageExtension { }), ); this.syncProviderUi(); + } finally { + this.activeCodexRefreshes.delete(operation); + } + } + + private async cancelCodexProviderAndWait(provider = this.codexProvider): Promise { + const lifecycleProvider = provider as CodexProvider & { + cancelAndWait?: () => Promise; + }; + if (typeof lifecycleProvider.cancelAndWait === 'function') { + await lifecycleProvider.cancelAndWait(); + return; } + // Retains compatibility with narrow provider doubles while production + // always uses the awaitable Codex provider lifecycle. + provider.cancel(); } private async runCodexRefresh(trigger: RefreshTrigger): Promise { + if (this.disposed) return; + let ownedBackfillLease: ResourceLease | undefined; + let ownedWorkerLease: ResourceLease | undefined; + let workerStoppedSafely = false; + let continueHistoricalWork = false; const config = this.getConfiguration(); if (!config.codexEnabled) { + await this.cancelCodexProviderAndWait(); + if (this.codexBackfillLease?.active) { + await this.codexBackfillLease.stop('feature-disabled', () => undefined); + } + this.codexBackfillLease = undefined; + if (this.codexWorkerLease?.active) { + await this.codexWorkerLease.stop('feature-disabled', () => undefined); + } + this.codexWorkerLease = undefined; this.codexView = null; this.codexInsights = emptyCodexScopedInsights(); this.codexAvailable = false; @@ -1063,6 +1417,7 @@ export class ClaudeCodeUsageExtension { return; } this.codexAvailable = await this.codexProvider.isAvailable(); + if (this.disposed) return; if (!this.codexAvailable) { this.codexView = null; this.codexInsights = emptyCodexScopedInsights(); @@ -1080,18 +1435,120 @@ export class ClaudeCodeUsageExtension { const provider = this.codexProvider; this.codexCheckpointHydrationLastAttemptAt = Date.now(); const persisted = await provider.loadPersistedSnapshot(); - if (provider !== this.codexProvider) { + if (this.disposed || provider !== this.codexProvider) { return; } if (persisted) { this.applyCodexSnapshot(persisted); this.syncProviderUi(); } + if (!persisted && this.codexBackgroundState.status === 'complete') { + this.codexBackgroundState = createBackgroundWorkState({ + measurementVersion: + ClaudeCodeUsageExtension.CODEX_BACKGROUND_MEASUREMENT_VERSION, + reason: 'first-index', + now: Date.now(), + }); + await this.saveCodexBackgroundState(); + } + const historicalPending = this.codexHistoricalWorkPending(persisted); + let historicalAttempt = false; + if (historicalPending) { + const started = beginBackgroundWork(this.codexBackgroundState, { + trigger: trigger === 'manual' ? 'manual' : 'automatic', + now: Date.now(), + reason: this.codexBackgroundReason(persisted), + }); + this.codexBackgroundState = started.state; + historicalAttempt = started.started; + try { + await this.saveCodexBackgroundState(); + } catch (error) { + if (historicalAttempt && this.codexBackgroundState.status === 'running') { + this.codexBackgroundState = interruptBackgroundWork( + this.codexBackgroundState, + { now: Date.now() }, + ); + await this.saveCodexBackgroundState().catch(() => undefined); + } + throw error; + } + if (historicalAttempt) { + ownedBackfillLease = this.resourceOwnership.register({ + kind: 'backfill', + capability: 'codex-history', + scope: 'codex', + creator: 'refresh-coordinator', + stopConditions: [ + 'settled', + 'completed', + 'cancelled', + 'feature-disabled', + 'extension-dispose', + 'user-pause', + 'settings-change', + ], + boundedException: persisted ? 'none' : 'first-codex-history', + }); + this.codexBackfillLease = ownedBackfillLease; + this.codexFirstBackfillActive = !persisted; + } + } else if ( + this.codexBackgroundState.status !== 'complete' && + this.codexBackgroundState.pausedReason !== 'corrupt-state' && + persisted + ) { + const fresh = createBackgroundWorkState({ + measurementVersion: + ClaudeCodeUsageExtension.CODEX_BACKGROUND_MEASUREMENT_VERSION, + reason: this.codexBackgroundReason(persisted), + now: Date.now(), + progress: this.codexBackgroundProgress(persisted), + }); + const running = beginBackgroundWork(fresh, { + trigger: 'automatic', + now: Date.now(), + }); + if (running.started) { + this.codexBackgroundState = recordBackgroundWorkProgress(running.state, { + now: Date.now(), + complete: true, + progress: this.codexBackgroundProgress(persisted), + }); + await this.saveCodexBackgroundState(); + } + } + // The trigger reason is visible while metadata discovery and the first + // worker progress event are still pending. + this.syncProviderUi(); try { + if (!this.codexWorkerLease?.active) { + ownedWorkerLease = this.resourceOwnership.register({ + kind: 'worker', + capability: 'codex-index', + scope: 'codex', + creator: 'codex-index-client', + stopConditions: [ + 'settled', + 'cancelled', + 'feature-disabled', + 'extension-dispose', + 'settings-change', + ], + boundedException: 'none', + }); + this.codexWorkerLease = ownedWorkerLease; + this.codexWorkerCancellationRequested = false; + } const result = await provider.refresh( codexRefreshProfileForTrigger(trigger), (progress) => this.onCodexIndexProgress(progress), + historicalAttempt, ); + workerStoppedSafely = true; + if (this.disposed || provider !== this.codexProvider) { + return; + } if (result.outcome === 'unavailable') { this.codexView = null; this.codexInsights = emptyCodexScopedInsights(); @@ -1101,7 +1558,48 @@ export class ClaudeCodeUsageExtension { } this.applyCodexSnapshot(result.snapshot); + if (historicalAttempt && this.codexBackgroundState.status === 'running') { + const complete = !this.codexHistoricalWorkPending(result.snapshot); + if (this.codexWorkerCancellationRequested || provider !== this.codexProvider) { + this.codexBackgroundState = interruptBackgroundWork( + this.codexBackgroundState, + { now: Date.now() }, + ); + } else if (result.outcome === 'error' || (result.diagnostic?.failedFiles ?? 0) > 0) { + this.codexBackgroundState = recordBackgroundWorkFailure( + this.codexBackgroundState, + { now: Date.now() }, + ); + } else { + this.codexBackgroundState = recordBackgroundWorkProgress( + this.codexBackgroundState, + { + now: Date.now(), + complete, + progress: this.codexBackgroundProgress(result.snapshot), + }, + ); + continueHistoricalWork = this.codexBackgroundState.status === 'eligible'; + } + await this.saveCodexBackgroundState(); + } const diagnostic = result.diagnostic; + if ( + !historicalAttempt && + diagnostic?.indexRecovery && + this.codexBackgroundState.status === 'complete' && + this.codexHistoricalWorkPending(result.snapshot) + ) { + this.codexBackgroundState = createBackgroundWorkState({ + measurementVersion: + ClaudeCodeUsageExtension.CODEX_BACKGROUND_MEASUREMENT_VERSION, + reason: this.codexBackgroundReason(result.snapshot), + now: Date.now(), + progress: this.codexBackgroundProgress(result.snapshot), + }); + await this.saveCodexBackgroundState(); + continueHistoricalWork = true; + } this.outputChannel.appendLine( formatCodexIndexDiagnostic({ outcome: result.outcome, @@ -1122,10 +1620,55 @@ export class ClaudeCodeUsageExtension { }), ); } finally { + if (historicalAttempt && this.codexBackgroundState.status === 'running') { + this.codexBackgroundState = + this.codexWorkerCancellationRequested || provider !== this.codexProvider + ? interruptBackgroundWork(this.codexBackgroundState, { now: Date.now() }) + : recordBackgroundWorkFailure( + this.codexBackgroundState, + { now: Date.now() }, + ); + await this.saveCodexBackgroundState(); + } + if (workerStoppedSafely && ownedBackfillLease?.active) { + const condition = this.codexBackgroundState.status === 'complete' + ? 'completed' + : 'settled'; + await ownedBackfillLease.stop(condition, () => undefined); + } + if ( + workerStoppedSafely && + this.codexBackfillLease === ownedBackfillLease + ) { + this.codexBackfillLease = undefined; + } + if (workerStoppedSafely && ownedWorkerLease?.active) { + await ownedWorkerLease.stop( + this.codexWorkerCancellationRequested || provider !== this.codexProvider + ? 'cancelled' + : 'settled', + () => undefined, + ); + } + if (workerStoppedSafely && this.codexWorkerLease === ownedWorkerLease) { + this.codexWorkerLease = undefined; + } + if (workerStoppedSafely && ownedBackfillLease) { + this.codexFirstBackfillActive = false; + } this.codexRefreshing = false; this.codexProgress = null; this.codexProgressLastRenderedAt = 0; - this.syncProviderUi(); + if (!this.disposed) this.syncProviderUi(); + if ( + !this.disposed && + continueHistoricalWork && + provider === this.codexProvider && + this.getConfiguration().codexEnabled && + this.windowActivity.focused + ) { + queueMicrotask(() => void this.refreshCodexData(trigger)); + } } } @@ -1143,6 +1686,7 @@ export class ClaudeCodeUsageExtension { totalFiles: progress.totalFiles, indexedBytes: progress.indexedBytes, totalBytes: progress.totalBytes, + reason: this.codexBackgroundState.reason, }); } } @@ -1212,6 +1756,7 @@ export class ClaudeCodeUsageExtension { /** Dashboard Settings change — status-bar-only toggles apply in place, others reload. */ private onSettingsChangedFromPanel(key?: string): void { + if (this.disposed) return; if (key && ClaudeCodeUsageExtension.DASHBOARD_ONLY_SETTINGS.has(key)) { this.syncProviderUi(); return; @@ -1226,7 +1771,54 @@ export class ClaudeCodeUsageExtension { this.onConfigurationChanged(); } + private trackCodexProviderRetirement(provider: CodexProvider): Promise { + let tracked!: Promise; + tracked = Promise.resolve() + .then(() => provider.dispose()) + .catch((error: unknown) => { + this.codexProviderRetirementFailure ??= error; + throw error; + }) + .finally(() => this.codexProviderRetirements.delete(tracked)); + this.codexProviderRetirements.add(tracked); + void tracked.catch(() => undefined); + return tracked; + } + + private async waitForCodexProviderRetirements(): Promise { + while (this.codexProviderRetirements.size > 0) { + await Promise.all([...this.codexProviderRetirements]); + } + if (this.codexProviderRetirementFailure) { + throw this.codexProviderRetirementFailure; + } + } + + private async releaseCodexOwnership( + condition: Extract< + ResourceStopCondition, + 'cancelled' | 'feature-disabled' | 'extension-dispose' | 'settings-change' + >, + ): Promise { + if (this.codexWorkerLease?.active) { + await this.codexWorkerLease.stop(condition, () => undefined); + } + if (this.codexBackfillLease?.active) { + await this.codexBackfillLease.stop(condition, () => undefined); + } + this.codexWorkerLease = undefined; + this.codexBackfillLease = undefined; + } + private onConfigurationChanged(): void { + if (this.disposed) return; + const generation = ++this.configurationGeneration; + // Any endpoint/model/key/consent-affecting configuration change invalidates + // a visible preview before a later click could send it. + this.webviewProvider.invalidatePreparedAiRequests(); + void this.cancelAdviceNetworks('settings-change'); + this.stopQuotaColdRetry('settings-change'); + void this.cancelQuotaNetworks('settings-change'); const config = this.getConfiguration(); I18n.setLanguage(config.language as any); I18n.setDecimalPlaces(config.decimalPlaces); @@ -1243,16 +1835,30 @@ export class ClaudeCodeUsageExtension { this.stopCodexWatching(); this.stopCredentialsWatching(); this.selectClaudeProfile(config.dataDirectory); - this.codexProvider.dispose(); + const retiringCodexProvider = this.codexProvider; + this.codexWorkerCancellationRequested = true; + retiringCodexProvider.cancel(); + this.trackCodexProviderRetirement(retiringCodexProvider); this.codexProvider = this.createCodexProvider(config); if (!this.windowActivity.focused) { return; } - void this.refreshData(true, 'settings').then(() => { - void this.startFileWatching(); - this.startCodexWatching(); - this.startCredentialsWatching(); - }); + void (async () => { + try { + await this.waitForCodexProviderRetirements(); + if (this.disposed || generation !== this.configurationGeneration) return; + await this.releaseCodexOwnership('settings-change'); + await this.refreshData(true, 'settings'); + if (this.disposed || generation !== this.configurationGeneration) return; + await this.startFileWatching(); + if (this.disposed || generation !== this.configurationGeneration) return; + this.startCodexWatching(); + this.startCredentialsWatching(); + } catch { + // A provider that cannot be terminated stays fail-closed. Disposal will + // surface the retained failure instead of starting an unowned replacement. + } + })(); } /** Keep quota credentials on the same Claude profile as this window's logs. @@ -1263,6 +1869,8 @@ export class ClaudeCodeUsageExtension { if (nextClient.getCredentialsPath() === this.apiClient.getCredentialsPath()) { return; } + this.stopQuotaColdRetry('profile-change'); + void this.cancelQuotaNetworks('profile-change'); this.apiClient = nextClient; this.claudeProfileGeneration += 1; this.cache.usageLimits = null; @@ -1284,18 +1892,33 @@ export class ClaudeCodeUsageExtension { * filesystems do not support recursive watching). */ private async startFileWatching(): Promise { + if (this.disposed) return; + const requestGeneration = ++this.fileWatcherGeneration; if (!this.windowActivity.focused) { - this.stopFileWatching(); + this.stopFileWatching('window-blur'); return; } const config = this.getConfiguration(); if (!(config.fileWatchSeconds > 0)) { - this.stopFileWatching(); // "Off" + this.stopFileWatching('feature-disabled'); // "Off" return; } const dataDirectory = await ClaudeDataLoader.findClaudeDataDirectory(config.dataDirectory || undefined); + if ( + this.disposed || + requestGeneration !== this.fileWatcherGeneration + ) { + return; + } if (!this.windowActivity.focused) { - this.stopFileWatching(); + this.stopFileWatching('window-blur'); + return; + } + const currentConfig = this.getConfiguration(); + if ( + !(currentConfig.fileWatchSeconds > 0) || + currentConfig.dataDirectory !== config.dataDirectory + ) { return; } if (!dataDirectory) { @@ -1306,8 +1929,10 @@ export class ClaudeCodeUsageExtension { return; } this.stopFileWatching(); + const activeGeneration = this.fileWatcherGeneration; try { this.fileWatcher = fs.watch(projectsDir, { recursive: true }, (_event, filename) => { + if (this.disposed || activeGeneration !== this.fileWatcherGeneration) return; if (!filename || !String(filename).endsWith('.jsonl')) { return; } @@ -1317,7 +1942,7 @@ export class ClaudeCodeUsageExtension { this.watcherEventsSinceRefresh += 1; const delaySeconds = this.getConfiguration().fileWatchSeconds; if (!(delaySeconds > 0)) { - this.stopFileWatching(); + this.stopFileWatching('feature-disabled'); return; } this.watchDebounce.push(delaySeconds * 1000, () => { @@ -1325,32 +1950,60 @@ export class ClaudeCodeUsageExtension { }); }); this.watchedDir = projectsDir; + this.fileWatcherLease = this.resourceOwnership.register({ + kind: 'watcher', + capability: 'refresh', + scope: 'claude', + creator: 'extension', + stopConditions: [ + 'window-blur', + 'feature-disabled', + 'extension-dispose', + 'settings-change', + 'profile-change', + ], + boundedException: 'none', + }); } catch { // Recursive watching unsupported — the polling timer is enough. } } - private stopFileWatching(): void { + private stopFileWatching( + condition: Extract< + ResourceStopCondition, + 'window-blur' | 'feature-disabled' | 'extension-dispose' | 'settings-change' | 'profile-change' + > = 'settings-change', + ): void { + this.fileWatcherGeneration += 1; this.watchDebounce.clear(); - if (this.fileWatcher) { - try { - this.fileWatcher.close(); - } catch { - // Already closed. - } - this.fileWatcher = undefined; + const watcher = this.fileWatcher; + const lease = this.fileWatcherLease; + this.fileWatcher = undefined; + this.fileWatcherLease = undefined; + if (watcher) { + const close = (): void => { + try { + watcher.close(); + } catch { + // Already closed. + } + }; + if (lease?.active) this.trackResourceStop(lease.stop(condition, close)); + else close(); } this.watchedDir = null; } private startCodexWatching(): void { + if (this.disposed) return; if (!this.windowActivity.focused) { - this.stopCodexWatching(); + this.stopCodexWatching('window-blur'); return; } const config = this.getConfiguration(); if (!config.codexEnabled || !(config.codexFileWatchSeconds > 0)) { - this.stopCodexWatching(); + this.stopCodexWatching('feature-disabled'); return; } const codexHome = this.codexHome(config); @@ -1358,6 +2011,7 @@ export class ClaudeCodeUsageExtension { return; } this.stopCodexWatching(); + const activeGeneration = this.codexWatcherGeneration; for (const child of ['sessions', 'archived_sessions']) { const directory = path.join(codexHome, child); if (!fs.existsSync(directory)) { @@ -1368,12 +2022,13 @@ export class ClaudeCodeUsageExtension { directory, { recursive: true }, (_event, filename) => { + if (this.disposed || activeGeneration !== this.codexWatcherGeneration) return; if (!filename || !String(filename).endsWith('.jsonl')) { return; } const delaySeconds = this.getConfiguration().codexFileWatchSeconds; if (!(delaySeconds > 0)) { - this.stopCodexWatching(); + this.stopCodexWatching('feature-disabled'); return; } this.codexWatchDebounce.push(delaySeconds * 1000, () => { @@ -1382,6 +2037,19 @@ export class ClaudeCodeUsageExtension { }, ); this.codexWatchers.push(watcher); + this.codexWatcherLeases.set(watcher, this.resourceOwnership.register({ + kind: 'watcher', + capability: 'codex-index', + scope: 'codex', + creator: 'extension', + stopConditions: [ + 'window-blur', + 'feature-disabled', + 'extension-dispose', + 'settings-change', + ], + boundedException: 'none', + })); } catch { // Polling remains available when recursive watches are unsupported. } @@ -1390,16 +2058,28 @@ export class ClaudeCodeUsageExtension { this.codexWatchers.length > 0 ? codexHome : null; } - private stopCodexWatching(): void { + private stopCodexWatching( + condition: Extract< + ResourceStopCondition, + 'window-blur' | 'feature-disabled' | 'extension-dispose' | 'settings-change' + > = 'settings-change', + ): void { + this.codexWatcherGeneration += 1; this.codexWatchDebounce.clear(); for (const watcher of this.codexWatchers) { - try { - watcher.close(); - } catch { - // Already closed. - } + const close = (): void => { + try { + watcher.close(); + } catch { + // Already closed. + } + }; + const lease = this.codexWatcherLeases.get(watcher); + if (lease?.active) this.trackResourceStop(lease.stop(condition, close)); + else close(); } this.codexWatchers = []; + this.codexWatcherLeases.clear(); this.codexWatchedHome = null; } @@ -1414,8 +2094,9 @@ export class ClaudeCodeUsageExtension { * watch — those still self-correct on the next refresh tick. */ private startCredentialsWatching(): void { + if (this.disposed) return; if (!this.windowActivity.focused) { - this.stopCredentialsWatching(); + this.stopCredentialsWatching('window-blur'); return; } this.stopCredentialsWatching(); @@ -1426,16 +2107,64 @@ export class ClaudeCodeUsageExtension { return; } try { + const activeGeneration = this.credentialsWatcherGeneration; this.credsWatcher = fs.watch(dir, (_event, filename) => { + if (this.disposed || activeGeneration !== this.credentialsWatcherGeneration) return; if (filename && String(filename) !== name) { return; } if (this.credsDebounceTimer) { - clearTimeout(this.credsDebounceTimer); + const timer = this.credsDebounceTimer; + const timerLease = this.credsDebounceTimerLease; + this.credsDebounceTimer = undefined; + this.credsDebounceTimerLease = undefined; + if (timerLease?.active) { + this.trackResourceStop( + timerLease.stop('cancelled', () => clearTimeout(timer)), + ); + } else { + clearTimeout(timer); + } } this.credsDebounceTimer = setTimeout(() => { - this.handleCredentialsChange(); + const timerLease = this.credsDebounceTimerLease; + this.credsDebounceTimer = undefined; + this.credsDebounceTimerLease = undefined; + if (timerLease?.active) { + this.trackResourceStop(timerLease.stop('settled', () => undefined)); + } + if (!this.disposed && activeGeneration === this.credentialsWatcherGeneration) { + this.handleCredentialsChange(); + } }, 800); + this.credsDebounceTimerLease = this.resourceOwnership.register({ + kind: 'timer', + capability: 'quota', + scope: 'claude', + creator: 'extension', + stopConditions: [ + 'settled', + 'cancelled', + 'window-blur', + 'extension-dispose', + 'settings-change', + 'profile-change', + ], + boundedException: 'none', + }); + }); + this.credsWatcherLease = this.resourceOwnership.register({ + kind: 'watcher', + capability: 'quota', + scope: 'claude', + creator: 'extension', + stopConditions: [ + 'window-blur', + 'extension-dispose', + 'settings-change', + 'profile-change', + ], + boundedException: 'none', }); } catch { // Watching unsupported on this platform/filesystem — the refresh tick @@ -1452,18 +2181,37 @@ export class ClaudeCodeUsageExtension { void this.refreshData(false, 'credentials'); } - private stopCredentialsWatching(): void { + private stopCredentialsWatching( + condition: Extract< + ResourceStopCondition, + 'window-blur' | 'extension-dispose' | 'settings-change' | 'profile-change' + > = 'settings-change', + ): void { + this.credentialsWatcherGeneration += 1; if (this.credsDebounceTimer) { - clearTimeout(this.credsDebounceTimer); + const timer = this.credsDebounceTimer; + const lease = this.credsDebounceTimerLease; this.credsDebounceTimer = undefined; - } - if (this.credsWatcher) { - try { - this.credsWatcher.close(); - } catch { - // Already closed. + this.credsDebounceTimerLease = undefined; + if (lease?.active) { + this.trackResourceStop(lease.stop(condition, () => clearTimeout(timer))); } - this.credsWatcher = undefined; + else clearTimeout(timer); + } + const watcher = this.credsWatcher; + const lease = this.credsWatcherLease; + this.credsWatcher = undefined; + this.credsWatcherLease = undefined; + if (watcher) { + const close = (): void => { + try { + watcher.close(); + } catch { + // Already closed. + } + }; + if (lease?.active) this.trackResourceStop(lease.stop(condition, close)); + else close(); } } @@ -1473,13 +2221,20 @@ export class ClaudeCodeUsageExtension { } private suspendRecurringWork(): void { - this.stopAutoRefresh(); - this.stopFileWatching(); - this.stopCodexWatching(); - this.stopCredentialsWatching(); + this.stopAutoRefresh('window-blur'); + this.stopQuotaColdRetry('window-blur'); + void this.cancelQuotaNetworks('window-blur'); + this.stopFileWatching('window-blur'); + this.stopCodexWatching('window-blur'); + this.stopCredentialsWatching('window-blur'); + if (this.codexRefreshing && !this.codexFirstBackfillActive) { + this.codexWorkerCancellationRequested = true; + void this.cancelCodexProviderAndWait(); + } } private resumeRecurringWork(): void { + if (this.disposed) return; this.startAutoRefresh(); void this.startFileWatching(); this.startCodexWatching(); @@ -1488,6 +2243,7 @@ export class ClaudeCodeUsageExtension { } private handleWindowFocusChange(focused: boolean): void { + if (this.disposed) return; const transition = this.windowActivity.update(focused); if (transition === 'suspend') { this.suspendRecurringWork(); @@ -1503,37 +2259,70 @@ export class ClaudeCodeUsageExtension { private startWindowFocusRefresh(): void { this.context.subscriptions.push( vscode.window.onDidChangeWindowState((state) => { + if (this.disposed) return; this.handleWindowFocusChange(state.focused); }) ); } - private stopAutoRefresh(): void { + private stopAutoRefresh( + condition: Extract< + ResourceStopCondition, + 'window-blur' | 'extension-dispose' | 'settings-change' + > = 'settings-change', + ): void { this.refreshGen += 1; - if (this.refreshTimer) { - clearTimeout(this.refreshTimer); - this.refreshTimer = undefined; + const timer = this.refreshTimer; + const lease = this.refreshTimerLease; + this.refreshTimer = undefined; + this.refreshTimerLease = undefined; + if (timer) { + if (lease?.active) { + this.trackResourceStop(lease.stop(condition, () => clearTimeout(timer))); + } + else clearTimeout(timer); } } private startAutoRefresh(): void { + if (this.disposed) return; this.stopAutoRefresh(); if (!this.windowActivity.focused) { return; } const gen = this.refreshGen; const tick = (): void => { - if (gen !== this.refreshGen) { + if (this.disposed || gen !== this.refreshGen) { return; // superseded by a newer startAutoRefresh — stop this chain } const intervalMs = pollIntervalMs(this.getConfiguration().refreshInterval); this.refreshTimer = setTimeout(() => { + const timerLease = this.refreshTimerLease; + this.refreshTimer = undefined; + this.refreshTimerLease = undefined; + if (timerLease?.active) { + this.trackResourceStop(timerLease.stop('settled', () => undefined)); + } + if (this.disposed || gen !== this.refreshGen) return; this.refreshData(false, 'poll').finally(() => { - if (gen === this.refreshGen) { + if (!this.disposed && gen === this.refreshGen) { tick(); } }); }, intervalMs); + this.refreshTimerLease = this.resourceOwnership.register({ + kind: 'timer', + capability: 'refresh', + scope: 'extension', + creator: 'refresh-coordinator', + stopConditions: [ + 'settled', + 'window-blur', + 'extension-dispose', + 'settings-change', + ], + boundedException: 'none', + }); }; tick(); } @@ -1616,7 +2405,7 @@ export class ClaudeCodeUsageExtension { } private async maybeFetchUsageLimits(config: ExtensionConfig): Promise { - if (!config.usageLimitTracking) { + if (this.disposed || !config.usageLimitTracking) { return null; } const now = Date.now(); @@ -1642,7 +2431,47 @@ export class ClaudeCodeUsageExtension { } const profileGeneration = this.claudeProfileGeneration; const apiClient = this.apiClient; - const fetched = await apiClient.fetchUsageLimits(); + const controller = new AbortController(); + const quotaNetworkLease = this.resourceOwnership.register({ + kind: 'network', + capability: 'quota', + scope: 'claude', + creator: 'claude-api-client', + stopConditions: [ + 'settled', + 'cancelled', + 'window-blur', + 'feature-disabled', + 'extension-dispose', + 'settings-change', + 'profile-change', + ], + boundedException: 'none', + }); + let markSettled!: () => void; + const settled = new Promise((resolve) => { + markSettled = resolve; + }); + this.activeQuotaNetworks.set(controller, { + lease: quotaNetworkLease, + settled, + }); + let fetched: ClaudeApiUsageResponse | null; + try { + fetched = await apiClient.fetchUsageLimits(controller.signal); + } finally { + this.activeQuotaNetworks.delete(controller); + markSettled(); + if (quotaNetworkLease.active) { + await quotaNetworkLease.stop( + controller.signal.aborted ? 'cancelled' : 'settled', + () => undefined, + ); + } + } + if (controller.signal.aborted) { + return this.cache.usageLimits; + } if ( profileGeneration !== this.claudeProfileGeneration || apiClient !== this.apiClient @@ -1692,6 +2521,7 @@ export class ClaudeCodeUsageExtension { forceReload: boolean = false, trigger: RefreshTrigger = 'poll' ): Promise { + if (this.disposed) return; // `watch` reaches this shared path only from the Claude projects watcher. // Codex has its own watcher and quiet-delay setting, so refreshing it here // would bypass codex.fileWatchSeconds whenever Claude writes a JSONL line. @@ -1729,30 +2559,26 @@ export class ClaudeCodeUsageExtension { this.coalescedTriggersSinceRefresh = 0; let updateWebview = request.trigger === 'manual'; try { + if (this.disposed) return; const config = this.getConfiguration(); updateWebview = updateWebview || config.dashboardAutoRefresh; // Account quota is independent from local JSONL. Do not let a slow OAuth // request delay the local usage refresh. - this.maybeFetchUsageLimits(config).then((limits) => { + void this.maybeFetchUsageLimits(config).then((limits) => { + if (this.disposed) return; this.statusBar.updateQuota(limits); this.webviewProvider.updateQuota(limits); if (!limits && !this.cache.usageLimits && !this.quotaColdRetryDone) { this.quotaColdRetryDone = true; - setTimeout(() => { - this.maybeFetchUsageLimits(this.getConfiguration()).then((retry) => { - if (retry) { - this.statusBar.updateQuota(retry); - this.webviewProvider.updateQuota(retry); - } - }); - }, 8000); + this.scheduleQuotaColdRetry(); } - }); + }).catch(() => undefined); const dataDirectory = await ClaudeDataLoader.findClaudeDataDirectory( config.dataDirectory || undefined ); + if (this.disposed) return; if (!dataDirectory) { const error = 'Claude data directory not found. Please check your configuration.'; this.statusBar.updateUsageData(null, null, error); @@ -1954,34 +2780,82 @@ export class ClaudeCodeUsageExtension { totalMs: performance.now() - totalStarted, })); } finally { - this.syncProviderUi(); + if (!this.disposed) this.syncProviderUi(); const next = this.refreshGate.complete(); - if (next !== null) { - setTimeout(() => void this.runRefresh(next), 0); + if (!this.disposed && next !== null) { + queueMicrotask(() => void this.runRefresh(next)); } } } - dispose(): void { - this.stopAutoRefresh(); - this.stopFileWatching(); - this.stopCodexWatching(); - this.stopCredentialsWatching(); - this.codexProvider.dispose(); - this.statusBar.dispose(); - this.webviewProvider.dispose(); + dispose(): Promise { + if (this.disposal) return this.disposal; + this.disposed = true; + this.configurationGeneration += 1; + this.disposal = (async () => { + const failures: unknown[] = []; + const capture = async (operation: Promise): Promise => { + try { + await operation; + return true; + } catch (error) { + failures.push(error); + return false; + } + }; + this.stopQuotaColdRetry('extension-dispose'); + this.stopAutoRefresh('extension-dispose'); + this.stopFileWatching('extension-dispose'); + this.stopCodexWatching('extension-dispose'); + this.stopCredentialsWatching('extension-dispose'); + await capture(Promise.all([ + this.cancelAdviceNetworks('extension-dispose'), + this.cancelQuotaNetworks('extension-dispose'), + ])); + this.codexWorkerCancellationRequested = true; + this.codexProvider.cancel(); + const providerStopped = await capture(Promise.all([ + this.waitForCodexProviderRetirements(), + this.codexProvider.dispose(), + ])); + while (this.activeCodexRefreshes.size > 0) { + await Promise.allSettled([...this.activeCodexRefreshes]); + } + if (providerStopped) { + await capture(this.releaseCodexOwnership('extension-dispose')); + } + await capture(this.codexBackgroundStateWrite.catch((error) => { + throw error; + })); + await capture(this.drainResourceStops()); + this.statusBar.dispose(); + this.webviewProvider.dispose(); + if (failures.length > 0) { + const first = failures[0]; + throw first instanceof Error + ? first + : new Error('Extension resources could not be stopped safely'); + } + })(); + return this.disposal; } } +let activeExtension: ClaudeCodeUsageExtension | null = null; + export function activate(context: vscode.ExtensionContext) { console.log('Claude Code Usage extension is now active'); const extension = new ClaudeCodeUsageExtension(context); + activeExtension = extension; context.subscriptions.push({ - dispose: () => extension.dispose() + dispose: () => { void extension.dispose(); } }); } -export function deactivate() { +export async function deactivate(): Promise { console.log('Claude Code Usage extension is now deactivated'); + const extension = activeExtension; + activeExtension = null; + await extension?.dispose(); } diff --git a/src/httpClient.ts b/src/httpClient.ts index 8d7d4168..051a4588 100644 --- a/src/httpClient.ts +++ b/src/httpClient.ts @@ -17,7 +17,8 @@ export interface HttpResponse { export interface HttpRequestOptions { method?: string; headers?: Record; - body?: string; + body?: string | Uint8Array; + signal?: AbortSignal; } /** Run an HTTP request via Node fetch. Optional timeoutMs aborts the request @@ -30,22 +31,31 @@ export async function requestViaFetch( throw new Error('fetch unavailable in this VS Code version'); } let signal: AbortSignal | undefined; + let controller: AbortController | undefined; let timer: ReturnType | undefined; - if (opts.timeoutMs && opts.timeoutMs > 0) { - const controller = new AbortController(); + const abortFromCaller = (): void => controller?.abort(); + if ((opts.timeoutMs && opts.timeoutMs > 0) || opts.signal) { + controller = new AbortController(); + if (opts.signal?.aborted) controller.abort(); + else opts.signal?.addEventListener('abort', abortFromCaller, { once: true }); + } + if (opts.timeoutMs && opts.timeoutMs > 0 && controller) { timer = setTimeout(() => controller.abort(), opts.timeoutMs); - signal = controller.signal; } + signal = controller?.signal ?? opts.signal; try { const res = await fetch(url, { method: opts.method || 'GET', headers: opts.headers, - body: opts.body, + body: typeof opts.body === 'string' || opts.body === undefined + ? opts.body + : Buffer.from(opts.body), signal }); return { status: res.status, body: await res.text() }; } catch (e) { if ((e as Error).name === 'AbortError') { + if (opts.signal?.aborted) throw new Error('Request cancelled'); throw new Error(`Request timed out after ${Math.round((opts.timeoutMs || 0) / 1000)}s`); } throw e; @@ -53,6 +63,7 @@ export async function requestViaFetch( if (timer !== undefined) { clearTimeout(timer); } + opts.signal?.removeEventListener('abort', abortFromCaller); } } @@ -63,6 +74,10 @@ export function requestViaCurl( log?: (line: string) => void ): Promise { return new Promise((resolve, reject) => { + if (opts.signal?.aborted) { + reject(new Error('Request cancelled')); + return; + } const args: string[] = ['-sS', '-w', '\n__CCU_STATUS__%{http_code}', '--max-time', String(opts.timeoutSec ?? 15)]; if (opts.method && opts.method !== 'GET') { args.push('-X', opts.method); @@ -84,6 +99,23 @@ export function requestViaCurl( // quota silently stops loading (while a fresh window, with a valid cwd, works // fine). curl needs no particular cwd, so anchor it somewhere always valid. const child = spawn(cmd, args, { shell: false, windowsHide: true, cwd: os.homedir() }); + let settled = false; + let aborted = false; + const finishReject = (error: Error): void => { + if (settled) return; + settled = true; + reject(error); + }; + const finishResolve = (response: HttpResponse): void => { + if (settled) return; + settled = true; + resolve(response); + }; + const abort = (): void => { + aborted = true; + child.kill(); + }; + opts.signal?.addEventListener('abort', abort, { once: true }); let stdout = ''; let stderr = ''; child.stdout.on('data', (c: Buffer) => (stdout += c.toString('utf-8'))); @@ -92,17 +124,22 @@ export function requestViaCurl( if (log) { log(`curl: spawn error ${(e as Error).message} (is curl on PATH?)`); } - reject(e); + finishReject(e); }); child.on('close', (code) => { + opts.signal?.removeEventListener('abort', abort); + if (aborted) { + return finishReject(new Error('Request cancelled')); + } + if (settled) return; if (code !== 0) { - return reject(new Error(`curl exit ${code}: ${stderr.trim().slice(0, 200)}`)); + return finishReject(new Error(`curl exit ${code}: ${stderr.trim().slice(0, 200)}`)); } const m = stdout.match(/^([\s\S]*)\n__CCU_STATUS__(\d{3})$/); if (!m) { - return reject(new Error(`Could not parse curl output: ${stdout.slice(0, 200)}`)); + return finishReject(new Error(`Could not parse curl output: ${stdout.slice(0, 200)}`)); } - resolve({ status: parseInt(m[2], 10), body: m[1] }); + finishResolve({ status: parseInt(m[2], 10), body: m[1] }); }); if (opts.body !== undefined) { child.stdin.end(opts.body); diff --git a/src/i18n.ts b/src/i18n.ts index d99ab04c..a24d5f3d 100644 --- a/src/i18n.ts +++ b/src/i18n.ts @@ -62,15 +62,20 @@ export interface AdviceEffectivenessTranslations { payloadTitle: string; payloadDescription: string; aggregatesOnly: string; + aggregatesWithPersonalization: string; aggregatesWithPromptSamples: string; payloadBytes: string; promptSamplesIncluded: string; previewPayload: string; noNetworkTransport: string; + sendPreparedRequest: string; + sendingPreparedRequest: string; + sentPreparedRequest: string; aggregateConsentLabel: string; aggregateConsentHelp: string; promptConsentLabel: string; promptConsentHelp: string; + promptWindowHelp: string; feedbackTitle: string; helpful: string; notHelpful: string; @@ -84,6 +89,8 @@ export interface AdviceEffectivenessTranslations { qualityGuardrailFailed: string; improved: string; noDemonstratedImprovement: string; + clearLocalData: string; + clearLocalDataConfirm: string; strictOutputRejected: string; } @@ -301,13 +308,15 @@ type CodexCopyMapKey = | 'insightTitles' | 'insightObservations' | 'insightTips' - | 'insightEvidenceLabels'; + | 'insightEvidenceLabels' + | 'indexingReasons'; type CodexCopyOverrides = Partial> & { insightTitles?: Partial; insightObservations?: Partial; insightTips?: Partial; insightEvidenceLabels?: Partial; + indexingReasons?: Partial; }; const TASK5_CODEX_COPY: Record, CodexCopyOverrides> = { @@ -546,6 +555,7 @@ type CodexTask8CopyKey = | 'indexedAllTime' | 'indexedSubtotal' | 'indexingInProgress' + | 'indexingReasons' | 'updatedAt' | 'claudeTokenAccounting' | 'codexTokenAccounting' @@ -565,6 +575,15 @@ const TASK8_CODEX_COPY: Record< indexedAllTime: 'Indexierter Gesamtzeitraum', indexedSubtotal: 'Indizierte Zwischensumme', indexingInProgress: 'Die Indexierung läuft noch; nicht verifizierte Altsummen werden ausgeschlossen.', + indexingReasons: { + 'first-index': 'Erste lokale Verlaufseinrichtung', + 'parser-migration': 'Lokalen Parser-Index aktualisieren', + 'period-migration': 'Datumsverlauf aktualisieren', + 'hourly-history': 'Stündlichen Verlauf der letzten Zeit erstellen', + 'history-backfill': 'Lokalen Verlauf vervollständigen', + 'rule-migration': 'Messregeln aktualisieren', + resume: 'Lokalen Verlauf fortsetzen', + }, updatedAt: 'Aktualisiert um', claudeTokenAccounting: 'Claude-Token-Zählung', codexTokenAccounting: 'Codex-Token-Zählung', @@ -599,6 +618,15 @@ const TASK8_CODEX_COPY: Record< indexedAllTime: '已索引的全部時間', indexedSubtotal: '已索引小計', indexingInProgress: '索引仍在進行;未驗證的舊版總量不會計入。', + indexingReasons: { + 'first-index': '首次設定本機歷史', + 'parser-migration': '更新本機解析器索引', + 'period-migration': '更新日期歷史', + 'hourly-history': '建立近期每小時歷史', + 'history-backfill': '補齊本機歷史', + 'rule-migration': '更新量測規則', + resume: '繼續整理本機歷史', + }, updatedAt: '更新於', claudeTokenAccounting: 'Claude Token 口徑', codexTokenAccounting: 'Codex Token 口徑', @@ -633,6 +661,15 @@ const TASK8_CODEX_COPY: Record< indexedAllTime: '已索引的全部时间', indexedSubtotal: '已索引小计', indexingInProgress: '索引仍在进行;未验证的旧版总量不会计入。', + indexingReasons: { + 'first-index': '首次整理本地历史', + 'parser-migration': '更新本地解析器索引', + 'period-migration': '更新日期历史', + 'hourly-history': '整理近期小时历史', + 'history-backfill': '补齐本地历史', + 'rule-migration': '更新测量规则', + resume: '继续整理本地历史', + }, updatedAt: '更新时间', claudeTokenAccounting: 'Claude Token 口径', codexTokenAccounting: 'Codex Token 口径', @@ -667,6 +704,15 @@ const TASK8_CODEX_COPY: Record< indexedAllTime: '索引済みの全期間', indexedSubtotal: 'インデックス済み小計', indexingInProgress: 'インデックス作成中です。未検証の旧集計値は除外されています。', + indexingReasons: { + 'first-index': '初回のローカル履歴設定', + 'parser-migration': 'ローカル解析インデックスを更新中', + 'period-migration': '日付履歴を更新中', + 'hourly-history': '最近の時間別履歴を作成中', + 'history-backfill': 'ローカル履歴を補完中', + 'rule-migration': '測定ルールを更新中', + resume: 'ローカル履歴を再開中', + }, updatedAt: '更新日時', claudeTokenAccounting: 'Claude トークン集計', codexTokenAccounting: 'Codex トークン集計', @@ -701,6 +747,15 @@ const TASK8_CODEX_COPY: Record< indexedAllTime: '인덱싱된 전체 기간', indexedSubtotal: '인덱싱된 소계', indexingInProgress: '인덱싱이 진행 중이며 검증되지 않은 이전 합계는 제외됩니다.', + indexingReasons: { + 'first-index': '첫 로컬 기록 설정', + 'parser-migration': '로컬 파서 인덱스 업데이트', + 'period-migration': '날짜 기록 업데이트', + 'hourly-history': '최근 시간별 기록 생성', + 'history-backfill': '로컬 기록 완성', + 'rule-migration': '측정 규칙 업데이트', + resume: '로컬 기록 재개', + }, updatedAt: '업데이트 시각', claudeTokenAccounting: 'Claude 토큰 집계', codexTokenAccounting: 'Codex 토큰 집계', @@ -735,6 +790,15 @@ const TASK8_CODEX_COPY: Record< indexedAllTime: 'Todo o período indexado', indexedSubtotal: 'Subtotal indexado', indexingInProgress: 'A indexação ainda está em andamento; totais legados não verificados são excluídos.', + indexingReasons: { + 'first-index': 'Primeira configuração do histórico local', + 'parser-migration': 'Atualizando o índice do analisador local', + 'period-migration': 'Atualizando o histórico por data', + 'hourly-history': 'Criando o histórico horário recente', + 'history-backfill': 'Concluindo o histórico local', + 'rule-migration': 'Atualizando as regras de medição', + resume: 'Retomando o histórico local', + }, updatedAt: 'Atualizado em', claudeTokenAccounting: 'Contagem de tokens do Claude', codexTokenAccounting: 'Contagem de tokens do Codex', @@ -769,6 +833,15 @@ const TASK8_CODEX_COPY: Record< indexedAllTime: 'Seluruh waktu terindeks', indexedSubtotal: 'Subtotal terindeks', indexingInProgress: 'Pengindeksan masih berlangsung; total lama yang belum diverifikasi tidak disertakan.', + indexingReasons: { + 'first-index': 'Penyiapan awal riwayat lokal', + 'parser-migration': 'Memperbarui indeks parser lokal', + 'period-migration': 'Memperbarui riwayat tanggal', + 'hourly-history': 'Membuat riwayat per jam terbaru', + 'history-backfill': 'Melengkapi riwayat lokal', + 'rule-migration': 'Memperbarui aturan pengukuran', + resume: 'Melanjutkan riwayat lokal', + }, updatedAt: 'Diperbarui pada', claudeTokenAccounting: 'Penghitungan token Claude', codexTokenAccounting: 'Penghitungan token Codex', @@ -832,6 +905,11 @@ function providerTranslations( ...overrides.insightEvidenceLabels, ...task5Overrides.insightEvidenceLabels, }, + indexingReasons: { + ...CODEX_COPY_EN.indexingReasons, + ...overrides.indexingReasons, + ...task5Overrides.indexingReasons, + }, }, }; } @@ -1116,7 +1194,7 @@ const translations: Record = { adviceEffectiveness: { title: 'AI advice effectiveness', description: 'Review how each suggestion moves from a local observation to evidence, an action, and a guarded result.', - candidateNotice: 'v2.3.1 candidate · off by default · no network transport connected.', + candidateNotice: 'v2.3.1 candidate · off by default · no default or background AI requests.', spineLabel: 'Advice evidence path', observation: 'Observation', evidence: 'Evidence', @@ -1137,17 +1215,22 @@ const translations: Record = { elapsedTimeProxy: 'Elapsed session span is a proxy, not measured active work time.', qualityGuardrailPending: 'Quality evidence is pending, so no effectiveness conclusion is available.', payloadTitle: 'Sealed payload snapshot', - payloadDescription: 'This sealed snapshot is the exact UTF-8 JSON body a future transport would receive.', + payloadDescription: 'This sealed snapshot is the exact UTF-8 JSON body the configured BYOK endpoint receives only after the separate Send click.', aggregatesOnly: 'Aggregates only', + aggregatesWithPersonalization: 'Aggregates + explicitly allowed personal context', aggregatesWithPromptSamples: 'Aggregates + explicitly allowed prompt samples', payloadBytes: '{bytes} UTF-8 bytes', promptSamplesIncluded: '{count} prompt samples included', previewPayload: 'Preview sealed snapshot', - noNetworkTransport: 'Preview only — this candidate has no network transport.', + noNetworkTransport: 'Previewing sends nothing. Sending requires a separate click.', + sendPreparedRequest: 'Send this exact request', + sendingPreparedRequest: 'Sending…', + sentPreparedRequest: 'Structured advice received.', aggregateConsentLabel: 'Allow aggregate usage data', aggregateConsentHelp: 'Includes only allowlisted numeric aggregates and structural signals — no prompts, paths, session IDs, or individual records.', - promptConsentLabel: 'Also include my prompt samples', - promptConsentHelp: 'Optional and off by default. Bounded prompt text is added only after this separate explicit consent.', + promptConsentLabel: 'Also include prompt samples and configured personal context', + promptConsentHelp: 'Optional and off by default. Bounded prompt text and configured personal context are added only after this separate explicit consent.', + promptWindowHelp: 'Prompt samples older than the configured {days}-day evidence window are excluded.', feedbackTitle: 'Local feedback', helpful: 'Helpful', notHelpful: 'Not helpful', @@ -1161,6 +1244,8 @@ const translations: Record = { qualityGuardrailFailed: 'The quality guardrail did not hold, so no effectiveness claim is made.', improved: 'Improvement met the pre-declared threshold while the quality guardrail held.', noDemonstratedImprovement: 'No improvement was demonstrated; this does not establish harm or causation.', + clearLocalData: 'Clear local advice data', + clearLocalDataConfirm: 'Clear advice consent, feedback, comparable pairs, and comparison results stored on this device?', strictOutputRejected: 'The structured model output was rejected. No advice was created.', }, totalTokens: 'Total Tokens', @@ -1305,14 +1390,14 @@ const translations: Record = { getAdvice: 'Get AI Advice', adviceCardTitle: 'AI advice', adviceCardDesc: - 'Send your usage digest + a sample of your own prompts to your model and get concrete tips on writing clearer instructions and cutting waste.', + 'Review local evidence first. The exact request defaults to aggregates only; prompt samples and personal context require separate consent, and nothing is sent until you click Send.', optimizerTitle: 'Usage optimizer', optimizerDesc: 'Turn a rough, half-formed request into a clean prompt you can paste straight into Claude Code — plus a suggested effort / thinking / model for the task.', optimizerHowto: - 'Type or paste your draft below, tick any of the optional tweaks, then click Optimise. Only the text you paste is sent to your model — never to Claude Code or your terminal.', + 'Type or paste your draft below, choose any optional tweaks, then build the exact request preview. Nothing is sent until you click Send; only the text you pasted can be included.', optimizerConsent: - 'The Usage Optimizer sends the text you paste to your configured API model. Nothing is sent to Claude Code and nothing is typed into a terminal. Continue?', + 'Build the exact request preview? This does not send anything; sending to your configured API model requires a separate click.', optimizerEnableBtn: 'Enable in settings', optimizerPlaceholder: 'Paste a rough prompt to optimise…', optimizerRun: 'Optimise', @@ -1401,7 +1486,7 @@ const translations: Record = { adviceEffectiveness: { title: 'Wirksamkeit von KI-Empfehlungen', description: 'Prüfe, wie jede Empfehlung von einer lokalen Beobachtung über Evidenz und eine Maßnahme zu einem abgesicherten Ergebnis gelangt.', - candidateNotice: 'v2.3.1-Kandidat · standardmäßig deaktiviert · keine Netzwerkübertragung verbunden.', + candidateNotice: 'v2.3.1-Kandidat · standardmäßig deaktiviert · keine standardmäßigen oder Hintergrund-KI-Anfragen.', spineLabel: 'Evidenzpfad der Empfehlung', observation: 'Beobachtung', evidence: 'Evidenz', @@ -1422,17 +1507,22 @@ const translations: Record = { elapsedTimeProxy: 'Die verstrichene Sitzungsspanne ist ein Proxy und keine gemessene aktive Arbeitszeit.', qualityGuardrailPending: 'Qualitätsevidenz steht noch aus; daher ist keine Aussage zur Wirksamkeit möglich.', payloadTitle: 'Versiegelte Payload-Momentaufnahme', - payloadDescription: 'Diese versiegelte Momentaufnahme ist exakt der UTF-8-JSON-Body, den eine künftige Übertragung erhalten würde.', + payloadDescription: 'Diese versiegelte Momentaufnahme ist exakt der UTF-8-JSON-Body, den der konfigurierte BYOK-Endpunkt erst nach einem separaten Klick auf Senden erhält.', aggregatesOnly: 'Nur Aggregate', + aggregatesWithPersonalization: 'Aggregate + ausdrücklich erlaubter persönlicher Kontext', aggregatesWithPromptSamples: 'Aggregate + ausdrücklich erlaubte Prompt-Beispiele', payloadBytes: '{bytes} UTF-8-Bytes', promptSamplesIncluded: '{count} Prompt-Beispiele enthalten', previewPayload: 'Versiegelte Momentaufnahme anzeigen', - noNetworkTransport: 'Nur Vorschau — dieser Kandidat hat keine Netzwerkübertragung.', + noNetworkTransport: 'Die Vorschau sendet nichts. Das Senden erfordert einen separaten Klick.', + sendPreparedRequest: 'Genau diese Anfrage senden', + sendingPreparedRequest: 'Wird gesendet…', + sentPreparedRequest: 'Strukturierter Rat empfangen.', aggregateConsentLabel: 'Aggregierte Nutzungsdaten erlauben', aggregateConsentHelp: 'Enthält nur freigegebene numerische Aggregate und strukturelle Signale — keine Prompts, Pfade, Sitzungs-IDs oder Einzelaufzeichnungen.', - promptConsentLabel: 'Auch meine Prompt-Beispiele einschließen', - promptConsentHelp: 'Optional und standardmäßig deaktiviert. Begrenzter Prompt-Text wird nur nach dieser separaten ausdrücklichen Einwilligung hinzugefügt.', + promptConsentLabel: 'Auch Prompt-Beispiele und konfigurierten persönlichen Kontext einschließen', + promptConsentHelp: 'Optional und standardmäßig deaktiviert. Begrenzter Prompt-Text und der konfigurierte persönliche Kontext werden nur nach dieser separaten ausdrücklichen Einwilligung hinzugefügt.', + promptWindowHelp: 'Prompt-Beispiele außerhalb des konfigurierten Evidenzfensters von {days} Tagen werden ausgeschlossen.', feedbackTitle: 'Lokales Feedback', helpful: 'Hilfreich', notHelpful: 'Nicht hilfreich', @@ -1446,6 +1536,8 @@ const translations: Record = { qualityGuardrailFailed: 'Die Qualitätsleitplanke hielt nicht; deshalb wird keine Wirksamkeit behauptet.', improved: 'Die Verbesserung erreichte den vorab festgelegten Schwellenwert, während die Qualitätsleitplanke hielt.', noDemonstratedImprovement: 'Es wurde keine Verbesserung nachgewiesen; daraus folgen weder Schaden noch Kausalität.', + clearLocalData: 'Lokale Empfehlungsdaten löschen', + clearLocalDataConfirm: 'Einwilligungen, Feedback, vergleichbare Paare und Vergleichsergebnisse auf diesem Gerät löschen?', strictOutputRejected: 'Die strukturierte Modellausgabe wurde abgelehnt. Es wurde keine Empfehlung erstellt.', }, totalTokens: "Gesamte Token", @@ -1590,14 +1682,14 @@ const translations: Record = { getAdvice: "KI-Rat holen", adviceCardTitle: "KI-Rat", adviceCardDesc: - "Sende deine Nutzungsübersicht + eine Auswahl deiner eigenen Prompts an dein Modell und erhalte konkrete Tipps für klarere Anweisungen und weniger Verschwendung.", + "Prüfe zuerst die lokalen Belege. Die exakte Anfrage enthält standardmäßig nur Aggregate; Prompt-Beispiele und persönlicher Kontext brauchen eine separate Einwilligung, und erst ein Klick auf Senden überträgt etwas.", optimizerTitle: "Nutzungs-Optimierer", optimizerDesc: "Mach aus einer groben, halbfertigen Anfrage einen sauberen Prompt, den du direkt in Claude Code einfügen kannst — plus empfohlenes Effort / Thinking / Modell für die Aufgabe.", optimizerHowto: - "Tippe oder füge deinen Entwurf unten ein, aktiviere optionale Feineinstellungen und klicke Optimieren. Nur der eingefügte Text wird an dein Modell gesendet — nie an Claude Code oder dein Terminal.", + "Tippe oder füge deinen Entwurf unten ein, wähle optionale Anpassungen und erstelle dann die exakte Anfragevorschau. Erst ein separater Klick auf Senden überträgt etwas; enthalten sein kann nur dein eingefügter Text.", optimizerConsent: - "Der Nutzungs-Optimierer sendet den eingefügten Text an dein konfiguriertes API-Modell. Nichts geht an Claude Code, nichts wird ins Terminal getippt. Fortfahren?", + "Exakte Anfragevorschau erstellen? Dabei wird nichts gesendet; die Übertragung an dein konfiguriertes API-Modell erfordert einen separaten Klick.", optimizerEnableBtn: "In Einstellungen aktivieren", optimizerPlaceholder: "Groben Prompt zum Optimieren einfügen…", optimizerRun: "Optimieren", @@ -1689,7 +1781,7 @@ const translations: Record = { adviceEffectiveness: { title: 'AI 建議有效性', description: '檢視每項建議如何從本機觀察,經由證據與行動,走到有品質護欄的結果。', - candidateNotice: 'v2.3.1 候選功能 · 預設關閉 · 尚未連接網路傳輸。', + candidateNotice: 'v2.3.1 候選功能 · 預設關閉 · 沒有預設或背景 AI 請求。', spineLabel: '建議證據路徑', observation: '觀察', evidence: '證據', @@ -1710,17 +1802,22 @@ const translations: Record = { elapsedTimeProxy: '工作階段首尾跨度是代理指標,不是實測的活躍工作時間。', qualityGuardrailPending: '品質證據仍待補充,因此目前無法判定建議有效性。', payloadTitle: '密封 payload 快照', - payloadDescription: '此密封快照就是未來傳輸會收到的確切 UTF-8 JSON body。', + payloadDescription: '此密封快照就是設定的 BYOK 端點在另行點擊「傳送」後才會收到的確切 UTF-8 JSON body。', aggregatesOnly: '僅彙總資料', + aggregatesWithPersonalization: '彙總資料 + 明確允許的個人上下文', aggregatesWithPromptSamples: '彙總資料 + 明確允許的 prompt 樣本', payloadBytes: '{bytes} 個 UTF-8 位元組', promptSamplesIncluded: '包含 {count} 個 prompt 樣本', previewPayload: '預覽密封快照', - noNetworkTransport: '僅供預覽——此候選功能尚無網路傳輸。', + noNetworkTransport: '預覽不會傳送任何內容;傳送需要另行點擊。', + sendPreparedRequest: '傳送這個完全相同的請求', + sendingPreparedRequest: '正在傳送…', + sentPreparedRequest: '已收到結構化建議。', aggregateConsentLabel: '允許使用彙總用量資料', aggregateConsentHelp: '只包含允許清單內的數值彙總與結構訊號——不含 prompt、路徑、工作階段 ID 或逐筆記錄。', - promptConsentLabel: '同時包含我的 prompt 樣本', - promptConsentHelp: '選用且預設關閉。只有在另行明確同意後,才會加入有數量與長度限制的 prompt 文字。', + promptConsentLabel: '同時包含 prompt 樣本與設定的個人上下文', + promptConsentHelp: '選用且預設關閉。只有在另行明確同意後,才會加入有數量與長度限制的 prompt 文字及設定的個人上下文。', + promptWindowHelp: '早於設定之 {days} 天證據視窗的 prompt 樣本不會包含。', feedbackTitle: '本機意見回饋', helpful: '有用', notHelpful: '無用', @@ -1734,6 +1831,8 @@ const translations: Record = { qualityGuardrailFailed: '品質護欄未通過,因此不聲稱建議有效。', improved: '改善達到預先聲明的門檻,且品質護欄維持通過。', noDemonstratedImprovement: '尚未證明有改善;這不代表造成傷害,也不建立因果關係。', + clearLocalData: '清除本機建議資料', + clearLocalDataConfirm: '要清除此裝置上的建議同意、回饋、可比配對與比較結果嗎?', strictOutputRejected: '結構化模型輸出遭拒絕,因此未建立任何建議。', }, totalTokens: '總 Token 數', @@ -1874,14 +1973,14 @@ const translations: Record = { getAdvice: '取得 AI 建議', adviceCardTitle: 'AI 建議', adviceCardDesc: - '將你的用量摘要 + 你自己的 prompt 樣本送給模型,取得寫出更清楚指令、減少浪費的具體建議。', + '先查看本機證據。精確請求預設只含彙總資料;prompt 樣本與個人上下文需另行同意,而且只有按下「傳送」才會送出。', optimizerTitle: '用量優化器', optimizerDesc: '把粗略、半成形的需求,變成可直接貼進 Claude Code 的乾淨 prompt,並附上這個任務建議的 effort / thinking / 模型。', optimizerHowto: - '在下方輸入或貼上你的草稿,按需勾選可選的微調項,再按「優化」。只有你貼上的文字會送給模型——不會送給 Claude Code 或終端。', + '在下方輸入或貼上草稿,選取需要的調整,再建立精確請求預覽。此步驟不會傳送;只有另按「傳送」才會送出,且只會包含你貼上的文字。', optimizerConsent: - '用量優化器會把你貼上的文字送給你配置的 API 模型。不會送給 Claude Code,也不會注入終端。要繼續嗎?', + '要建立精確請求預覽嗎?此步驟不會傳送;送往你配置的 API 模型仍需另按一次「傳送」。', optimizerEnableBtn: '在設定中啟用', optimizerPlaceholder: '貼上要優化的粗略 prompt…', optimizerRun: '優化', @@ -1970,7 +2069,7 @@ const translations: Record = { adviceEffectiveness: { title: 'AI 建议有效性', description: '查看每条建议如何从本地观察,经由证据与行动,走到有质量护栏的结果。', - candidateNotice: 'v2.3.1 候选功能 · 默认关闭 · 尚未连接网络传输。', + candidateNotice: 'v2.3.1 候选功能 · 默认关闭 · 不存在默认或后台 AI 请求。', spineLabel: '建议证据路径', observation: '观察', evidence: '证据', @@ -1991,17 +2090,22 @@ const translations: Record = { elapsedTimeProxy: '会话首尾跨度是代理指标,不是实测的活跃工作时间。', qualityGuardrailPending: '质量证据仍待补充,因此目前无法判断建议有效性。', payloadTitle: '密封 payload 快照', - payloadDescription: '此密封快照就是未来传输会收到的确切 UTF-8 JSON body。', + payloadDescription: '此密封快照就是配置的 BYOK 端点仅在另行点击“发送”后才会收到的确切 UTF-8 JSON body。', aggregatesOnly: '仅汇总数据', + aggregatesWithPersonalization: '汇总数据 + 明确允许的个人上下文', aggregatesWithPromptSamples: '汇总数据 + 明确允许的 prompt 样本', payloadBytes: '{bytes} 个 UTF-8 字节', promptSamplesIncluded: '包含 {count} 个 prompt 样本', previewPayload: '预览密封快照', - noNetworkTransport: '仅供预览——此候选功能尚无网络传输。', + noNetworkTransport: '预览不会发送任何内容;发送需要另行点击。', + sendPreparedRequest: '发送这份完全相同的请求', + sendingPreparedRequest: '正在发送…', + sentPreparedRequest: '已收到结构化建议。', aggregateConsentLabel: '允许使用汇总用量数据', aggregateConsentHelp: '只包含允许列表内的数值汇总与结构信号——不含 prompt、路径、会话 ID 或逐条记录。', - promptConsentLabel: '同时包含我的 prompt 样本', - promptConsentHelp: '可选且默认关闭。只有另行明确同意后,才会加入有数量和长度限制的 prompt 文本。', + promptConsentLabel: '同时包含 prompt 样本与配置的个人上下文', + promptConsentHelp: '可选且默认关闭。只有另行明确同意后,才会加入有数量和长度限制的 prompt 文本及配置的个人上下文。', + promptWindowHelp: '早于所配置 {days} 天证据窗口的 prompt 样本不会包含。', feedbackTitle: '本地反馈', helpful: '有用', notHelpful: '无用', @@ -2015,6 +2119,8 @@ const translations: Record = { qualityGuardrailFailed: '质量护栏未通过,因此不声称建议有效。', improved: '改善达到预先声明的门槛,且质量护栏保持通过。', noDemonstratedImprovement: '尚未证明有改善;这不代表造成伤害,也不建立因果关系。', + clearLocalData: '清除本地建议数据', + clearLocalDataConfirm: '要清除此设备上保存的建议同意、反馈、可比配对和比较结果吗?', strictOutputRejected: '结构化模型输出被拒绝,因此未创建任何建议。', }, totalTokens: '总 Token 数', @@ -2155,14 +2261,14 @@ const translations: Record = { getAdvice: '获取 AI 建议', adviceCardTitle: 'AI 建议', adviceCardDesc: - '将你的用量摘要 + 你自己的 prompt 样本送给模型,获得写出更清晰指令、减少浪费的具体建议。', + '先查看本地证据。精确请求默认只含汇总数据;prompt 样本与个人上下文需另行同意,而且只有点击“发送”才会发出。', optimizerTitle: '用量优化器', optimizerDesc: '把粗略、没成形的需求,变成可以直接粘进 Claude Code 的干净 prompt,并附上这个任务建议的 effort / thinking / 模型。', optimizerHowto: - '在下方输入或粘贴你的草稿,按需勾选下面的可选微调项,再点「优化」。只有你粘贴的文字会发给模型——不会发给 Claude Code,也不会注入终端。', + '在下方输入或粘贴草稿,选择需要的调整,再生成精确请求预览。此步骤不会发送;只有另点“发送”才会发出,且只会包含你粘贴的文字。', optimizerConsent: - '用量优化器会把你粘贴的文字发送给你配置的 API 模型。不会发送给 Claude Code,也不会注入终端。要继续吗?', + '要生成精确请求预览吗?此步骤不会发送;发送到你配置的 API 模型仍需另点一次“发送”。', optimizerEnableBtn: '在设置中启用', optimizerPlaceholder: '粘贴要优化的粗略 prompt…', optimizerRun: '优化', @@ -2251,7 +2357,7 @@ const translations: Record = { adviceEffectiveness: { title: 'AI アドバイスの有効性', description: '各提案がローカルの観察から根拠と行動を経て、品質ガードレール付きの結果に至る流れを確認します。', - candidateNotice: 'v2.3.1 候補機能 · デフォルトは無効 · ネットワーク転送は未接続です。', + candidateNotice: 'v2.3.1 候補機能 · デフォルトは無効 · 既定またはバックグラウンドの AI リクエストはありません。', spineLabel: 'アドバイスの根拠経路', observation: '観察', evidence: '根拠', @@ -2272,17 +2378,22 @@ const translations: Record = { elapsedTimeProxy: 'セッションの経過範囲は代理指標であり、実測した作業時間ではありません。', qualityGuardrailPending: '品質の根拠が未確定のため、有効性について結論を出せません。', payloadTitle: '封印済み payload スナップショット', - payloadDescription: 'この封印済みスナップショットは、将来の転送が受け取る UTF-8 JSON body と完全に同一です。', + payloadDescription: 'この封印済みスナップショットは、別の送信クリック後にだけ設定済み BYOK エンドポイントが受け取る UTF-8 JSON body と完全に同一です。', aggregatesOnly: '集計データのみ', + aggregatesWithPersonalization: '集計データ + 明示的に許可した個人コンテキスト', aggregatesWithPromptSamples: '集計データ + 明示的に許可したプロンプト例', payloadBytes: '{bytes} UTF-8 バイト', promptSamplesIncluded: '{count} 件のプロンプト例を含む', previewPayload: '封印済みスナップショットをプレビュー', - noNetworkTransport: 'プレビューのみ — この候補機能にはネットワーク転送がありません。', + noNetworkTransport: 'プレビューでは何も送信されません。送信には別のクリックが必要です。', + sendPreparedRequest: 'この同一リクエストを送信', + sendingPreparedRequest: '送信中…', + sentPreparedRequest: '構造化された助言を受信しました。', aggregateConsentLabel: '集計使用データを許可する', aggregateConsentHelp: '許可リスト内の数値集計と構造シグナルだけを含み、プロンプト、パス、セッション ID、個別レコードは含みません。', - promptConsentLabel: '自分のプロンプト例も含める', - promptConsentHelp: '任意で、デフォルトは無効です。この独立した明示的同意の後にだけ、件数と長さを制限したプロンプト本文を追加します。', + promptConsentLabel: 'プロンプト例と設定済み個人コンテキストも含める', + promptConsentHelp: '任意で、デフォルトは無効です。この独立した明示的同意の後にだけ、件数と長さを制限したプロンプト本文と設定済み個人コンテキストを追加します。', + promptWindowHelp: '設定した {days} 日間の根拠期間より古いプロンプト例は除外します。', feedbackTitle: 'ローカルフィードバック', helpful: '役に立った', notHelpful: '役に立たなかった', @@ -2296,6 +2407,8 @@ const translations: Record = { qualityGuardrailFailed: '品質ガードレールを満たさなかったため、有効性を主張しません。', improved: '品質ガードレールを保ったまま、事前に定めた改善基準に達しました。', noDemonstratedImprovement: '改善は実証されていません。これは害や因果関係を示すものではありません。', + clearLocalData: 'ローカルのアドバイスデータを消去', + clearLocalDataConfirm: 'この端末に保存された同意、フィードバック、比較可能ペア、比較結果を消去しますか?', strictOutputRejected: '構造化されたモデル出力を拒否しました。アドバイスは作成されていません。', }, totalTokens: '総トークン数', @@ -2440,14 +2553,14 @@ const translations: Record = { getAdvice: 'AI アドバイスを取得', adviceCardTitle: 'AI アドバイス', adviceCardDesc: - '使用量サマリー + あなた自身のプロンプトのサンプルをモデルに送り、より明確な指示と無駄削減の具体的なヒントを得ます。', + 'まずローカルの根拠を確認します。正確なリクエストは既定で集計のみです。プロンプト例と個人コンテキストには別の同意が必要で、「送信」を押すまで何も送られません。', optimizerTitle: '使用量オプティマイザー', optimizerDesc: '雑で半端な依頼を、Claude Code にそのまま貼り付けられる整ったプロンプトに変換し、そのタスクに推奨の effort / thinking / モデルも返します。', optimizerHowto: - '下に下書きを入力または貼り付け、必要に応じて任意の調整オプションを選び「最適化」を押します。送信されるのは貼り付けたテキストのみ——Claude Code やターミナルには送られません。', + '下に下書きを入力または貼り付け、必要な調整を選んで正確なリクエストをプレビューします。この段階では送信されません。別途「送信」を押した場合のみ、貼り付けたテキストが含まれます。', optimizerConsent: - '使用量オプティマイザーは貼り付けたテキストを設定した API モデルに送信します。Claude Code には送られず、ターミナルにも入力されません。続行しますか?', + '正確なリクエストをプレビューしますか?この操作では送信されません。設定した API モデルへの送信には別のクリックが必要です。', optimizerEnableBtn: '設定で有効化', optimizerPlaceholder: '最適化する雑なプロンプトを貼り付け…', optimizerRun: '最適化', @@ -2537,7 +2650,7 @@ const translations: Record = { adviceEffectiveness: { title: 'AI 조언 효과성', description: '각 제안이 로컬 관찰에서 근거와 행동을 거쳐 품질 가드레일이 있는 결과로 이어지는 과정을 확인합니다.', - candidateNotice: 'v2.3.1 후보 기능 · 기본적으로 꺼짐 · 네트워크 전송이 연결되지 않음.', + candidateNotice: 'v2.3.1 후보 기능 · 기본적으로 꺼짐 · 기본 또는 백그라운드 AI 요청 없음.', spineLabel: '조언 근거 경로', observation: '관찰', evidence: '근거', @@ -2558,17 +2671,22 @@ const translations: Record = { elapsedTimeProxy: '세션의 시작과 끝 사이 시간은 프록시이며, 실제로 측정한 활성 작업 시간이 아닙니다.', qualityGuardrailPending: '품질 근거가 아직 준비되지 않아 효과성 결론을 내릴 수 없습니다.', payloadTitle: '봉인된 payload 스냅샷', - payloadDescription: '이 봉인된 스냅샷은 향후 전송이 받게 될 UTF-8 JSON body와 정확히 같습니다.', + payloadDescription: '이 봉인된 스냅샷은 별도의 보내기 클릭 후에만 설정된 BYOK 엔드포인트가 받는 UTF-8 JSON body와 정확히 같습니다.', aggregatesOnly: '집계 데이터만', + aggregatesWithPersonalization: '집계 데이터 + 명시적으로 허용한 개인 컨텍스트', aggregatesWithPromptSamples: '집계 데이터 + 명시적으로 허용한 프롬프트 샘플', payloadBytes: '{bytes} UTF-8바이트', promptSamplesIncluded: '프롬프트 샘플 {count}개 포함', previewPayload: '봉인된 스냅샷 미리보기', - noNetworkTransport: '미리보기 전용 — 이 후보 기능에는 네트워크 전송이 없습니다.', + noNetworkTransport: '미리보기는 아무것도 보내지 않습니다. 전송하려면 별도로 클릭해야 합니다.', + sendPreparedRequest: '이 동일한 요청 보내기', + sendingPreparedRequest: '전송 중…', + sentPreparedRequest: '구조화된 조언을 받았습니다.', aggregateConsentLabel: '집계 사용량 데이터 허용', aggregateConsentHelp: '허용 목록의 수치 집계와 구조 신호만 포함하며 프롬프트, 경로, 세션 ID, 개별 레코드는 포함하지 않습니다.', - promptConsentLabel: '내 프롬프트 샘플도 포함', - promptConsentHelp: '선택 사항이며 기본적으로 꺼져 있습니다. 이 별도의 명시적 동의 후에만 개수와 길이가 제한된 프롬프트 텍스트를 추가합니다.', + promptConsentLabel: '프롬프트 샘플과 설정된 개인 컨텍스트도 포함', + promptConsentHelp: '선택 사항이며 기본적으로 꺼져 있습니다. 이 별도의 명시적 동의 후에만 제한된 프롬프트 텍스트와 설정된 개인 컨텍스트를 추가합니다.', + promptWindowHelp: '설정한 {days}일 근거 기간보다 오래된 프롬프트 샘플은 제외합니다.', feedbackTitle: '로컬 피드백', helpful: '유용함', notHelpful: '유용하지 않음', @@ -2582,6 +2700,8 @@ const translations: Record = { qualityGuardrailFailed: '품질 가드레일을 충족하지 못했으므로 효과성을 주장하지 않습니다.', improved: '품질 가드레일을 유지하면서 사전에 정한 개선 기준을 충족했습니다.', noDemonstratedImprovement: '개선이 입증되지 않았습니다. 이는 피해나 인과관계를 뜻하지 않습니다.', + clearLocalData: '로컬 조언 데이터 지우기', + clearLocalDataConfirm: '이 기기에 저장된 조언 동의, 피드백, 비교 쌍 및 비교 결과를 지우시겠습니까?', strictOutputRejected: '구조화된 모델 출력을 거부했습니다. 조언이 생성되지 않았습니다.', }, totalTokens: '총 토큰 수', @@ -2726,14 +2846,14 @@ const translations: Record = { getAdvice: 'AI 조언 받기', adviceCardTitle: 'AI 조언', adviceCardDesc: - '사용량 요약 + 본인 프롬프트 샘플을 모델에 보내 더 명확한 지시와 낭비 줄이기에 대한 구체적 팁을 받습니다.', + '먼저 로컬 근거를 확인합니다. 정확한 요청은 기본적으로 집계만 포함하며, 프롬프트 샘플과 개인 컨텍스트는 별도 동의가 필요하고 「전송」을 누르기 전에는 아무것도 보내지 않습니다.', optimizerTitle: '사용량 옵티마이저', optimizerDesc: '대략적이고 정리되지 않은 요청을 Claude Code에 바로 붙여넣을 수 있는 깔끔한 프롬프트로 바꾸고, 그 작업에 추천하는 effort / thinking / 모델도 함께 제공합니다.', optimizerHowto: - '아래에 초안을 입력하거나 붙여넣고, 필요하면 선택 옵션을 고른 뒤 「최적화」를 누르세요. 붙여넣은 텍스트만 모델로 전송됩니다 — Claude Code나 터미널로는 가지 않습니다.', + '아래에 초안을 입력하거나 붙여넣고 필요한 옵션을 고른 뒤 정확한 요청을 미리 봅니다. 이 단계에서는 전송되지 않으며, 별도로 「전송」을 눌렀을 때만 붙여넣은 텍스트가 포함됩니다.', optimizerConsent: - '사용량 옵티마이저는 붙여넣은 텍스트를 설정한 API 모델로 보냅니다. Claude Code로는 전송되지 않고 터미널에도 입력되지 않습니다. 계속할까요?', + '정확한 요청을 미리 볼까요? 이 단계에서는 아무것도 보내지 않으며, 설정한 API 모델로 보내려면 별도의 클릭이 필요합니다.', optimizerEnableBtn: '설정에서 사용', optimizerPlaceholder: '최적화할 대략적인 프롬프트 붙여넣기…', optimizerRun: '최적화', @@ -2823,7 +2943,7 @@ const translations: Record = { adviceEffectiveness: { title: 'Eficácia dos conselhos de IA', description: 'Revise como cada sugestão passa de uma observação local para evidências, uma ação e um resultado com proteção de qualidade.', - candidateNotice: 'Candidato v2.3.1 · desativado por padrão · sem transporte de rede conectado.', + candidateNotice: 'Candidato v2.3.1 · desativado por padrão · sem solicitações de IA padrão ou em segundo plano.', spineLabel: 'Trilha de evidências do conselho', observation: 'Observação', evidence: 'Evidência', @@ -2844,17 +2964,22 @@ const translations: Record = { elapsedTimeProxy: 'O intervalo decorrido da sessão é um proxy, não tempo de trabalho ativo medido.', qualityGuardrailPending: 'As evidências de qualidade estão pendentes; portanto, ainda não há conclusão sobre eficácia.', payloadTitle: 'Snapshot de payload selado', - payloadDescription: 'Este snapshot selado é exatamente o body JSON em UTF-8 que um transporte futuro receberia.', + payloadDescription: 'Este snapshot selado é exatamente o body JSON em UTF-8 que o endpoint BYOK configurado recebe somente após um clique separado em Enviar.', aggregatesOnly: 'Somente agregados', + aggregatesWithPersonalization: 'Agregados + contexto pessoal permitido explicitamente', aggregatesWithPromptSamples: 'Agregados + amostras de prompt permitidas explicitamente', payloadBytes: '{bytes} bytes UTF-8', promptSamplesIncluded: '{count} amostras de prompt incluídas', previewPayload: 'Visualizar snapshot selado', - noNetworkTransport: 'Somente prévia — este candidato não tem transporte de rede.', + noNetworkTransport: 'A prévia não envia nada. O envio exige um clique separado.', + sendPreparedRequest: 'Enviar esta solicitação exata', + sendingPreparedRequest: 'Enviando…', + sentPreparedRequest: 'Conselho estruturado recebido.', aggregateConsentLabel: 'Permitir dados agregados de uso', aggregateConsentHelp: 'Inclui somente agregados numéricos e sinais estruturais da lista permitida — sem prompts, caminhos, IDs de sessão ou registros individuais.', - promptConsentLabel: 'Incluir também minhas amostras de prompt', - promptConsentHelp: 'Opcional e desativado por padrão. Texto de prompt com quantidade e tamanho limitados só é adicionado após este consentimento explícito separado.', + promptConsentLabel: 'Incluir amostras de prompt e contexto pessoal configurado', + promptConsentHelp: 'Opcional e desativado por padrão. Texto de prompt limitado e contexto pessoal configurado só são adicionados após este consentimento explícito separado.', + promptWindowHelp: 'Amostras de prompt anteriores à janela de evidências configurada de {days} dias são excluídas.', feedbackTitle: 'Feedback local', helpful: 'Útil', notHelpful: 'Não útil', @@ -2868,6 +2993,8 @@ const translations: Record = { qualityGuardrailFailed: 'A proteção de qualidade não foi mantida, portanto nenhuma eficácia é afirmada.', improved: 'A melhoria atingiu o limite pré-declarado enquanto a proteção de qualidade foi mantida.', noDemonstratedImprovement: 'Nenhuma melhoria foi demonstrada; isso não estabelece dano nem causalidade.', + clearLocalData: 'Limpar dados locais de conselho', + clearLocalDataConfirm: 'Limpar consentimentos, feedback, pares comparáveis e resultados de comparação armazenados neste dispositivo?', strictOutputRejected: 'A saída estruturada do modelo foi rejeitada. Nenhum conselho foi criado.', }, totalTokens: 'Total de tokens', @@ -3012,14 +3139,14 @@ const translations: Record = { getAdvice: 'Obter conselho de IA', adviceCardTitle: 'Conselho de IA', adviceCardDesc: - 'Envie o resumo do seu uso + uma amostra dos seus próprios prompts ao seu modelo e receba dicas concretas para escrever instruções mais claras e reduzir desperdício.', + 'Revise primeiro as evidências locais. A solicitação exata usa apenas agregados por padrão; amostras de prompts e contexto pessoal exigem consentimento separado, e nada é enviado até você clicar em Enviar.', optimizerTitle: 'Otimizador de uso', optimizerDesc: 'Transforme um pedido rascunhado e malformado em um prompt limpo para colar direto no Claude Code — além de um esforço / raciocínio / modelo sugeridos para a tarefa.', optimizerHowto: - 'Digite ou cole seu rascunho abaixo, marque quaisquer ajustes opcionais e clique em Otimizar. Apenas o texto que você cola é enviado ao seu modelo — nunca ao Claude Code nem ao seu terminal.', + 'Digite ou cole seu rascunho abaixo, escolha os ajustes opcionais e gere a visualização da solicitação exata. Nada é enviado até um clique separado em Enviar; somente o texto colado pode ser incluído.', optimizerConsent: - 'O Otimizador de Uso envia o texto que você cola ao modelo de API configurado. Nada é enviado ao Claude Code e nada é digitado em um terminal. Continuar?', + 'Gerar a visualização da solicitação exata? Isso não envia nada; o envio ao modelo de API configurado exige outro clique.', optimizerEnableBtn: 'Ativar nas configurações', optimizerPlaceholder: 'Cole um prompt rascunhado para otimizar…', optimizerRun: 'Otimizar', @@ -3108,7 +3235,7 @@ const translations: Record = { adviceEffectiveness: { title: 'Efektivitas saran AI', description: 'Tinjau bagaimana setiap saran bergerak dari pengamatan lokal menuju bukti, tindakan, dan hasil dengan pagar pengaman kualitas.', - candidateNotice: 'Kandidat v2.3.1 · nonaktif secara default · tidak ada transport jaringan yang terhubung.', + candidateNotice: 'Kandidat v2.3.1 · nonaktif secara default · tanpa permintaan AI default atau latar belakang.', spineLabel: 'Alur bukti saran', observation: 'Pengamatan', evidence: 'Bukti', @@ -3129,17 +3256,22 @@ const translations: Record = { elapsedTimeProxy: 'Rentang waktu sesi adalah proksi, bukan waktu kerja aktif yang diukur.', qualityGuardrailPending: 'Bukti kualitas masih tertunda, sehingga belum ada kesimpulan efektivitas.', payloadTitle: 'Snapshot payload tersegel', - payloadDescription: 'Snapshot tersegel ini persis sama dengan body JSON UTF-8 yang akan diterima transport mendatang.', + payloadDescription: 'Snapshot tersegel ini persis sama dengan body JSON UTF-8 yang diterima endpoint BYOK terkonfigurasi hanya setelah klik Kirim yang terpisah.', aggregatesOnly: 'Hanya agregat', + aggregatesWithPersonalization: 'Agregat + konteks pribadi yang diizinkan secara eksplisit', aggregatesWithPromptSamples: 'Agregat + sampel prompt yang diizinkan secara eksplisit', payloadBytes: '{bytes} byte UTF-8', promptSamplesIncluded: '{count} sampel prompt disertakan', previewPayload: 'Pratinjau snapshot tersegel', - noNetworkTransport: 'Hanya pratinjau — kandidat ini tidak memiliki transport jaringan.', + noNetworkTransport: 'Pratinjau tidak mengirim apa pun. Pengiriman memerlukan klik terpisah.', + sendPreparedRequest: 'Kirim permintaan persis ini', + sendingPreparedRequest: 'Mengirim…', + sentPreparedRequest: 'Saran terstruktur diterima.', aggregateConsentLabel: 'Izinkan data penggunaan agregat', aggregateConsentHelp: 'Hanya mencakup agregat numerik dan sinyal struktural dalam daftar izin — tanpa prompt, path, ID sesi, atau catatan individual.', - promptConsentLabel: 'Sertakan juga sampel prompt saya', - promptConsentHelp: 'Opsional dan nonaktif secara default. Teks prompt yang dibatasi jumlah dan panjangnya hanya ditambahkan setelah persetujuan eksplisit terpisah ini.', + promptConsentLabel: 'Sertakan sampel prompt dan konteks pribadi terkonfigurasi', + promptConsentHelp: 'Opsional dan nonaktif secara default. Teks prompt terbatas dan konteks pribadi terkonfigurasi hanya ditambahkan setelah persetujuan eksplisit terpisah ini.', + promptWindowHelp: 'Sampel prompt yang lebih lama dari jendela bukti {days} hari yang dikonfigurasi akan dikecualikan.', feedbackTitle: 'Umpan balik lokal', helpful: 'Bermanfaat', notHelpful: 'Tidak bermanfaat', @@ -3153,6 +3285,8 @@ const translations: Record = { qualityGuardrailFailed: 'Pagar pengaman kualitas tidak terpenuhi, sehingga tidak ada klaim efektivitas.', improved: 'Peningkatan mencapai ambang yang ditetapkan sebelumnya sambil mempertahankan pagar pengaman kualitas.', noDemonstratedImprovement: 'Belum ada peningkatan yang terbukti; ini tidak membuktikan kerugian atau hubungan sebab-akibat.', + clearLocalData: 'Hapus data saran lokal', + clearLocalDataConfirm: 'Hapus persetujuan saran, umpan balik, pasangan sebanding, dan hasil perbandingan yang tersimpan di perangkat ini?', strictOutputRejected: 'Output model terstruktur ditolak. Tidak ada saran yang dibuat.', }, totalTokens: 'Total Token', @@ -3298,14 +3432,14 @@ const translations: Record = { getAdvice: 'Dapatkan Saran AI', adviceCardTitle: 'Saran AI', adviceCardDesc: - 'Kirim ringkasan penggunaan Anda + sampel prompt Anda sendiri ke model Anda dan dapatkan tips konkret untuk menulis instruksi yang lebih jelas dan mengurangi pemborosan.', + 'Tinjau bukti lokal terlebih dahulu. Permintaan persis hanya berisi agregat secara default; sampel prompt dan konteks pribadi memerlukan persetujuan terpisah, dan tidak ada yang dikirim sampai Anda mengeklik Kirim.', optimizerTitle: 'Pengoptimal Penggunaan', optimizerDesc: 'Ubah permintaan yang masih kasar dan belum rapi menjadi prompt bersih yang bisa langsung ditempel ke Claude Code — plus saran effort / thinking / model untuk tugas tersebut.', optimizerHowto: - 'Ketik atau tempel draf Anda di bawah, centang penyesuaian opsional yang diinginkan, lalu klik Optimalkan. Hanya teks yang Anda tempel yang dikirim ke model Anda — tidak pernah ke Claude Code atau terminal Anda.', + 'Ketik atau tempel draf Anda di bawah, pilih penyesuaian opsional, lalu buat pratinjau permintaan persis. Tidak ada yang dikirim sampai Anda mengeklik Kirim secara terpisah; hanya teks yang ditempel yang dapat disertakan.', optimizerConsent: - 'Usage Optimizer mengirim teks yang Anda tempel ke model API yang telah dikonfigurasi. Tidak ada yang dikirim ke Claude Code dan tidak ada yang diketik ke terminal. Lanjutkan?', + 'Buat pratinjau permintaan persis? Ini tidak mengirim apa pun; pengiriman ke model API yang dikonfigurasi memerlukan klik terpisah.', optimizerEnableBtn: 'Aktifkan di pengaturan', optimizerPlaceholder: 'Tempel draf prompt untuk dioptimalkan…', optimizerRun: 'Optimalkan', @@ -3403,14 +3537,14 @@ const SETTINGS_I18N: Partial Promise; +} + +export type PreparedOptimizerResult = + | { ok: true; value: { prompt: string; settings: string } } + | { + ok: false; + code: 'invalid-prepared-request' | 'invalid-output' | 'transport-error'; + issues: string[]; + }; + +export function prepareOptimizerInvocation( + input: PrepareOptimizerInvocationInput, +): PreparedAiInvocation { + return prepareAiInvocation({ + kind: 'optimizer', + apiFormat: input.apiFormat, + apiUrl: input.apiUrl, + model: input.model, + ...(input.reasoningEffort ? { reasoningEffort: input.reasoningEffort } : {}), + systemPrompt: input.systemPrompt, + userContent: input.draft, + dataMode: 'user-draft-only', + sourceRevision: input.sourceRevision, + consentGeneration: input.consentGeneration, + createdAtEpochMs: input.createdAtEpochMs, + ...(input.timeoutMs === undefined ? {} : { timeoutMs: input.timeoutMs }), + }); +} + +/** Strict marker and three-line settings parser; never surfaces malformed text. */ +export function parseStrictOptimizerOutput( + raw: string, +): { prompt: string; settings: string } | undefined { + const text = raw.trim(); + const promptMarker = '===PROMPT==='; + const settingsMarker = '===SETTINGS==='; + if (!text.startsWith(`${promptMarker}\n`)) return undefined; + const settingsIndex = text.indexOf(`\n${settingsMarker}\n`); + if (settingsIndex <= promptMarker.length) return undefined; + if ( + text.indexOf(promptMarker, promptMarker.length) !== -1 || + text.indexOf(settingsMarker, settingsIndex + settingsMarker.length + 2) !== -1 + ) { + return undefined; + } + const prompt = text.slice(promptMarker.length, settingsIndex).trim(); + const settings = text + .slice(settingsIndex + settingsMarker.length + 2) + .trim(); + if (!prompt || prompt.length > 100_000 || !settings || settings.length > 10_000) { + return undefined; + } + const lines = settings.split(/\r?\n/); + if (lines.length !== 3) return undefined; + const reason = String.raw`\s+—\s+\S.{0,1000}`; + if (!new RegExp(`^Effort: (low|medium|high|max|ultracode)${reason}$`).test(lines[0])) { + return undefined; + } + if (!new RegExp(`^Thinking: (on|off)${reason}$`).test(lines[1])) { + return undefined; + } + if (!new RegExp(`^Model: [A-Za-z0-9._/-]{1,128}${reason}$`).test(lines[2])) { + return undefined; + } + return { prompt, settings }; +} + +export async function requestPreparedOptimizer( + prepared: PreparedAiInvocation, + options: PreparedOptimizerSendOptions, +): Promise { + if (prepared.kind !== 'optimizer' || prepared.dataMode !== 'user-draft-only') { + return { + ok: false, + code: 'invalid-prepared-request', + issues: ['prepared request is not an optimizer request'], + }; + } + let raw: string; + try { + raw = await sendPreparedModelRequest(prepared, options.apiKey, { + ...(options.expectedSourceRevision === undefined + ? {} + : { expectedSourceRevision: options.expectedSourceRevision }), + ...(options.expectedConsentGeneration === undefined + ? {} + : { expectedConsentGeneration: options.expectedConsentGeneration }), + ...(options.signal ? { signal: options.signal } : {}), + ...(options.transport ? { transport: options.transport } : {}), + }); + } catch { + return { + ok: false, + code: 'transport-error', + issues: ['configured BYOK transport failed'], + }; + } + const parsed = parseStrictOptimizerOutput(raw); + return parsed + ? { ok: true, value: parsed } + : { + ok: false, + code: 'invalid-output', + issues: ['optimizer output did not match the strict response contract'], + }; +} diff --git a/src/providers/codex/codexFilePassPool.ts b/src/providers/codex/codexFilePassPool.ts index f5d7a62a..5560a7fa 100644 --- a/src/providers/codex/codexFilePassPool.ts +++ b/src/providers/codex/codexFilePassPool.ts @@ -67,6 +67,9 @@ export class CodexFilePassPool { private workers: WorkerSlot[] = []; private active: ActiveBatch | null = null; private disposed = false; + private readonly pendingTerminations = new Set>(); + private terminationFailure: unknown = null; + private disposal: Promise | null = null; readonly run: CodexFilePassBatchRunner = async ( tasks, @@ -121,19 +124,19 @@ export class CodexFilePassPool { this.workerFactory = workerFactory; } - async dispose(): Promise { + dispose(): Promise { + if (this.disposal) return this.disposal; this.disposed = true; - if (this.active) { - this.failActive(new CodexIndexCancelledError()); - } - const workers = this.workers.splice(0); - await Promise.all(workers.map(async ({ worker }) => { - try { - await worker.terminate(); - } catch { - // Best-effort local worker cleanup. + this.disposal = (async () => { + if (this.active) { + this.failActive(new CodexIndexCancelledError()); } - })); + const workers = this.workers.splice(0); + for (const { worker } of workers) this.trackTermination(worker); + await Promise.all([...this.pendingTerminations]); + if (this.terminationFailure) throw this.terminationFailure; + })(); + return this.disposal; } private ensureWorkers(count: number): void { @@ -249,8 +252,25 @@ export class CodexFilePassPool { this.active = null; const workers = this.workers.splice(0); for (const { worker } of workers) { - void worker.terminate(); + this.trackTermination(worker); } active.reject(error); } + + private trackTermination(worker: CodexFilePassWorkerLike): void { + let pending!: Promise; + pending = Promise.resolve() + .then(() => worker.terminate()) + .then(() => undefined) + .catch((error: unknown) => { + this.terminationFailure ??= error; + throw error; + }) + .finally(() => this.pendingTerminations.delete(pending)); + this.pendingTerminations.add(pending); + // Disposal consumes and propagates this failure. Attach a side handler now + // so a fast rejection cannot become unhandled before the pool reaches its + // finally/dispose path. + void pending.catch(() => undefined); + } } diff --git a/src/providers/codex/codexIndex.ts b/src/providers/codex/codexIndex.ts index 93cfc4dd..462b1967 100644 --- a/src/providers/codex/codexIndex.ts +++ b/src/providers/codex/codexIndex.ts @@ -26,12 +26,15 @@ import { parseCodexLine, } from './codexParser'; import { + CODEX_ROLLING_HOURLY_DAYS, CodexFilePeriodIndex, CodexFileTodayIndex, + CodexHourlyDays, CodexPeriodMigrationState, CodexStructuralSummary, CodexTodayMigrationState, - reduceCodexHourlySlice, + pruneCodexHourlyDays, + reduceCodexRollingHourlySlice, reduceCodexStructuralSlice, reduceCodexUsageSlice, } from './codexPeriodIndex'; @@ -177,6 +180,28 @@ export interface CodexTodayCoverage { complete: boolean; } +export interface CodexHourlyDayCoverage { + day: string; + indexedFiles: number; + totalFiles: number; + indexedBytes: number; + totalBytes: number; + complete: boolean; +} + +export interface CodexHourlyCoverage { + timeZone: string; + asOfDay: string; + windowDays: number; + indexedFiles: number; + totalFiles: number; + indexedBytes: number; + totalBytes: number; + complete: boolean; + /** Coverage for each civil day that has a verified period contribution. */ + days: Record; +} + export interface CodexIndexCoverage { indexedFiles: number; totalFiles: number; @@ -185,6 +210,9 @@ export interface CodexIndexCoverage { complete: boolean; identity: CodexIdentityCoverage; period: CodexPeriodCoverage; + /** Missing only on pre-v2.3.1 schema-3 snapshots. */ + hourly?: CodexHourlyCoverage; + /** Current-day compatibility projection of `hourly`. */ today: CodexTodayCoverage; } @@ -219,6 +247,8 @@ export interface CodexIndexProgress { indexedBytes: number; totalBytes: number; period: CodexPeriodCoverage; + /** Present once rolling-hour coverage has been normalized or recomputed. */ + hourly?: CodexHourlyCoverage; } export interface CodexIndexSchedulingPolicy { @@ -292,6 +322,9 @@ export interface CodexIndexUpdateOptions { scheduling?: Partial; /** Present only for accelerated cold/incomplete backfills. */ filePassBatch?: CodexFilePassBatchRunner; + /** False skips unchanged historical lineage/period/hour/identity passes while + * still allowing the main changed-file and append-tail stage to run. */ + allowHistoricalBackfill?: boolean; /** Internal resolved civil day shared with local file-pass workers. */ asOfDay?: string; } @@ -462,6 +495,17 @@ export function createEmptyCodexIndex(timeZone = 'UTC'): CodexIndexV3 { last30Days: emptyRange(), allTime: emptyRange(), }, + hourly: { + timeZone: resolvedTimeZone, + asOfDay, + windowDays: CODEX_ROLLING_HOURLY_DAYS, + indexedFiles: 0, + totalFiles: 0, + indexedBytes: 0, + totalBytes: 0, + complete: true, + days: {}, + }, today: { timeZone: resolvedTimeZone, day: asOfDay, @@ -612,7 +656,58 @@ function emptyTodayIndex( timeZone: string, indexedThrough = 0, ): CodexFileTodayIndex { - return { day, timeZone, indexedThrough, hours: {} }; + return { + day, + timeZone, + indexedThrough, + hours: {}, + windowDays: CODEX_ROLLING_HOURLY_DAYS, + days: {}, + }; +} + +function rollingHourlyDaySet(day: string): Set { + return new Set( + rollingDayKeysFromDayKey(day, CODEX_ROLLING_HOURLY_DAYS), + ); +} + +function isRollingTodayIndex( + value: CodexFileTodayIndex | undefined, + timeZone: string, +): value is CodexFileTodayIndex & { days: CodexHourlyDays } { + return Boolean( + value && + value.timeZone === timeZone && + value.windowDays === CODEX_ROLLING_HOURLY_DAYS && + value.days, + ); +} + +function syncTodayProjection( + value: CodexFileTodayIndex & { days: CodexHourlyDays }, +): void { + pruneCodexHourlyDays(value.days, rollingHourlyDaySet(value.day)); + value.hours = value.days[value.day] ?? {}; +} + +/** + * A forward civil-day rollover needs only pruning: unchanged files cannot have + * events in the newly entered days. Moving the clock backwards can require + * days that were previously evicted, so it deliberately falls back to the + * resumable migration pass. + */ +function reanchorRollingToday( + value: CodexFileTodayIndex | undefined, + day: string, + timeZone: string, +): boolean { + if (!isRollingTodayIndex(value, timeZone) || day < value.day) { + return false; + } + value.day = day; + syncTodayProjection(value); + return true; } function promoteCaughtUpToday( @@ -625,6 +720,8 @@ function promoteCaughtUpToday( if ( migration?.day !== day || migration.timeZone !== timeZone || + migration.windowDays !== CODEX_ROLLING_HOURLY_DAYS || + !migration.days || migration.prefixEvents !== prefixEvents || migration.offset < contribution.offset || migration.discardingOversizedLine @@ -635,8 +732,15 @@ function promoteCaughtUpToday( day, timeZone, indexedThrough: contribution.offset, - hours: migration.hours, + hours: migration.days[day] ?? {}, + windowDays: CODEX_ROLLING_HOURLY_DAYS, + days: migration.days, }; + syncTodayProjection( + contribution.aggregate.today as CodexFileTodayIndex & { + days: CodexHourlyDays; + }, + ); contribution.qualityFlags = uniqueFlags( contribution.qualityFlags, migration.qualityFlags, @@ -726,28 +830,29 @@ async function updateContribution( new Date((options.now ?? Date.now)()), timeZone, ); + const rollingDays = rollingHourlyDaySet(asOfDay); const advancePeriod = aggregate.period?.timeZone === timeZone && aggregate.period.indexedThrough === contribution.offset; - if ( - aggregate.today && - (aggregate.today.day !== asOfDay || aggregate.today.timeZone !== timeZone) - ) { + if (aggregate.today && !reanchorRollingToday(aggregate.today, asOfDay, timeZone)) { delete aggregate.today; } if (!aggregate.today && contribution.offset === 0) { aggregate.today = emptyTodayIndex(asOfDay, timeZone); } - const advanceToday = - aggregate.today?.day === asOfDay && - aggregate.today.timeZone === timeZone && - aggregate.today.indexedThrough === contribution.offset; + const activeToday = + isRollingTodayIndex(aggregate.today, timeZone) && + aggregate.today.day === asOfDay && + aggregate.today.indexedThrough === contribution.offset + ? aggregate.today + : undefined; let invalidEventTimestamp = false; const snapshot = (cursor: CodexJsonlCursor): CodexFileContribution => { syncSession(aggregate, parserState); - if (advanceToday && aggregate.today) { - aggregate.today.indexedThrough = cursor.offset; + if (activeToday) { + activeToday.indexedThrough = cursor.offset; + syncTodayProjection(activeToday); } return { ...contribution, @@ -795,11 +900,11 @@ async function updateContribution( } else if (advancePeriod && aggregate.period) { reduceCodexUsageSlice(aggregate.period.days, event, timeZone); } - if (advanceToday && aggregate.today) { - reduceCodexHourlySlice( - aggregate.today.hours, + if (activeToday) { + reduceCodexRollingHourlySlice( + activeToday.days, event, - asOfDay, + rollingDays, timeZone, ); } @@ -847,8 +952,9 @@ async function updateContribution( if (advancePeriod && aggregate.period) { aggregate.period.indexedThrough = scan.cursor.offset; } - if (advanceToday && aggregate.today) { - aggregate.today.indexedThrough = scan.cursor.offset; + if (activeToday) { + activeToday.indexedThrough = scan.cursor.offset; + syncTodayProjection(activeToday); } if ( scan.cursor.offset >= entry.size && @@ -901,10 +1007,14 @@ async function reconcileContributionLineage( new Date((options.now ?? Date.now)()), timeZone, ); + const rollingDays = rollingHourlyDaySet(asOfDay); const initial = contribution.lineageReconciliation?.prefixEvents === prefixEvents && - contribution.lineageReconciliation.aggregate.today?.day === asOfDay && - contribution.lineageReconciliation.aggregate.today.timeZone === timeZone + isRollingTodayIndex( + contribution.lineageReconciliation.aggregate.today, + timeZone, + ) && + contribution.lineageReconciliation.aggregate.today.day === asOfDay ? contribution.lineageReconciliation : { prefixEvents, @@ -932,6 +1042,9 @@ async function reconcileContributionLineage( } if (aggregate.today) { aggregate.today.indexedThrough = cursor.offset; + if (isRollingTodayIndex(aggregate.today, timeZone)) { + syncTodayProjection(aggregate.today); + } } return { ...contribution, @@ -977,11 +1090,11 @@ async function reconcileContributionLineage( } else if (aggregate.period) { reduceCodexUsageSlice(aggregate.period.days, event, timeZone); } - if (aggregate.today) { - reduceCodexHourlySlice( - aggregate.today.hours, + if (isRollingTodayIndex(aggregate.today, timeZone)) { + reduceCodexRollingHourlySlice( + aggregate.today.days, event, - asOfDay, + rollingDays, timeZone, ); } @@ -1143,10 +1256,14 @@ async function migrateContributionToday( new Date((options.now ?? Date.now)()), timeZone, ); + const rollingDays = rollingHourlyDaySet(day); const prefixEvents = contribution.lineage?.desiredPrefixEvents ?? 0; const initial = contribution.todayMigration?.day === day && contribution.todayMigration.timeZone === timeZone && + contribution.todayMigration.windowDays === + CODEX_ROLLING_HOURLY_DAYS && + contribution.todayMigration.days && contribution.todayMigration.prefixEvents === prefixEvents ? contribution.todayMigration : { @@ -1158,11 +1275,14 @@ async function migrateContributionToday( discardingOversizedLine: false, parserState: createCodexParserState(entry.fileKey), hours: {}, + windowDays: CODEX_ROLLING_HOURLY_DAYS, + days: {}, qualityFlags: [], }; let parserState = initial.parserState; let tokenEventsSeen = initial.tokenEventsSeen; - const hours = initial.hours; + const days = initial.days as CodexHourlyDays; + pruneCodexHourlyDays(days, rollingDays); let invalidEventTimestamp = false; const pseudonymize = pseudonymizer(options.salt); @@ -1180,7 +1300,9 @@ async function migrateContributionToday( offset: cursor.offset, discardingOversizedLine: cursor.discardingOversizedLine, parserState, - hours, + hours: days[day] ?? {}, + windowDays: CODEX_ROLLING_HOURLY_DAYS, + days, qualityFlags: uniqueFlags( initial.qualityFlags, parserState.qualityFlags, @@ -1214,7 +1336,7 @@ async function migrateContributionToday( invalidEventTimestamp = true; continue; } - reduceCodexHourlySlice(hours, event, day, timeZone); + reduceCodexRollingHourlySlice(days, event, rollingDays, timeZone); } }, async (progress) => { @@ -1222,7 +1344,7 @@ async function migrateContributionToday( }, ); if (!scan.reachedEnd) { - throw new Error('Codex log changed during current-day migration'); + throw new Error('Codex log changed during rolling-hour migration'); } const updated = snapshot(scan.cursor); if (updated.todayMigration && scan.oversizedLines > 0) { @@ -1735,14 +1857,20 @@ function coverageFor( }; }; const last7Start = rollingDayKeysFromDayKey(asOfDay, 7)[0]; - const last30Start = rollingDayKeysFromDayKey(asOfDay, 30)[0]; + const last30DayKeys = rollingDayKeysFromDayKey( + asOfDay, + CODEX_ROLLING_HOURLY_DAYS, + ); + const last30Start = last30DayKeys[0]; const last7Days = rangeCoverage(last7Start); const last30Days = rangeCoverage(last30Start); const allTime = rangeCoverage(); - let todayIndexedFiles = 0; - let todayTotalFiles = 0; - let todayIndexedBytes = 0; - let todayTotalBytes = 0; + const rollingDays = new Set(last30DayKeys); + const hourlyDays: Record = {}; + let hourlyIndexedFiles = 0; + let hourlyTotalFiles = 0; + let hourlyIndexedBytes = 0; + let hourlyTotalBytes = 0; for (const entry of manifest.files) { if (!deduplication.canonicalFileKeys.has(entry.fileKey)) { continue; @@ -1754,15 +1882,20 @@ function coverageFor( contribution.lineageReconciliation || contribution.lineage?.appliedPrefixEvents !== contribution.lineage?.desiredPrefixEvents || - !contribution.aggregate.period?.days[asOfDay] + contribution.aggregate.period?.timeZone !== timeZone ) { continue; } - todayTotalFiles += 1; - todayTotalBytes += entry.size; + const eligibleDays = Object.keys(contribution.aggregate.period.days) + .filter((day) => rollingDays.has(day)); + if (eligibleDays.length === 0) { + continue; + } + hourlyTotalFiles += 1; + hourlyTotalBytes += entry.size; const promoted = - contribution.aggregate.today?.day === asOfDay && - contribution.aggregate.today.timeZone === timeZone + isRollingTodayIndex(contribution.aggregate.today, timeZone) && + contribution.aggregate.today.day === asOfDay ? Math.min( contribution.aggregate.today.indexedThrough, contribution.offset, @@ -1772,6 +1905,9 @@ function coverageFor( const draft = contribution.todayMigration?.day === asOfDay && contribution.todayMigration.timeZone === timeZone && + contribution.todayMigration.windowDays === + CODEX_ROLLING_HOURLY_DAYS && + contribution.todayMigration.days && contribution.todayMigration.prefixEvents === (contribution.lineage?.desiredPrefixEvents ?? 0) ? Math.min( @@ -1780,15 +1916,47 @@ function coverageFor( entry.size, ) : 0; - todayIndexedBytes += Math.max(0, Math.max(promoted, draft)); - if ( - contribution.aggregate.today?.day === asOfDay && - contribution.aggregate.today.timeZone === timeZone && - contribution.aggregate.today.indexedThrough >= contribution.offset - ) { - todayIndexedFiles += 1; + const migratedBytes = Math.max(0, Math.max(promoted, draft)); + hourlyIndexedBytes += migratedBytes; + const completeFile = Boolean( + isRollingTodayIndex(contribution.aggregate.today, timeZone) && + contribution.aggregate.today.day === asOfDay && + contribution.aggregate.today.indexedThrough >= contribution.offset, + ); + if (completeFile) { + hourlyIndexedFiles += 1; + } + for (const day of eligibleDays) { + const current = (hourlyDays[day] ??= { + day, + indexedFiles: 0, + totalFiles: 0, + indexedBytes: 0, + totalBytes: 0, + complete: false, + }); + current.totalFiles += 1; + current.totalBytes += entry.size; + current.indexedBytes += migratedBytes; + if (completeFile) { + current.indexedFiles += 1; + } } } + const hourlyComplete = + last30Days.complete && hourlyIndexedFiles === hourlyTotalFiles; + for (const coverage of Object.values(hourlyDays)) { + coverage.complete = + last30Days.complete && coverage.indexedFiles === coverage.totalFiles; + } + const todayDayCoverage = hourlyDays[asOfDay] ?? { + day: asOfDay, + indexedFiles: 0, + totalFiles: 0, + indexedBytes: 0, + totalBytes: 0, + complete: last30Days.complete, + }; return { indexedFiles, totalFiles, @@ -1807,14 +1975,25 @@ function coverageFor( last30Days, allTime, }, + hourly: { + timeZone, + asOfDay, + windowDays: CODEX_ROLLING_HOURLY_DAYS, + indexedFiles: hourlyIndexedFiles, + totalFiles: hourlyTotalFiles, + indexedBytes: hourlyIndexedBytes, + totalBytes: hourlyTotalBytes, + complete: hourlyComplete, + days: hourlyDays, + }, today: { timeZone, day: asOfDay, - indexedFiles: todayIndexedFiles, - totalFiles: todayTotalFiles, - indexedBytes: todayIndexedBytes, - totalBytes: todayTotalBytes, - complete: last7Days.complete && todayIndexedFiles === todayTotalFiles, + indexedFiles: todayDayCoverage.indexedFiles, + totalFiles: todayDayCoverage.totalFiles, + indexedBytes: todayDayCoverage.indexedBytes, + totalBytes: todayDayCoverage.totalBytes, + complete: todayDayCoverage.complete, }, }; } @@ -1851,6 +2030,7 @@ function progressFor(index: CodexIndexV3, scannedFiles: number): CodexIndexProgr indexedBytes: index.coverage.indexedBytes, totalBytes: index.coverage.totalBytes, period: index.coverage.period, + hourly: index.coverage.hourly, }; } @@ -1862,6 +2042,7 @@ function isWarmNoOp( asOfDay: string, ): boolean { const totalBytes = manifest.files.reduce((sum, entry) => sum + entry.size, 0); + const hourlyCoverage = previous.coverage.hourly; if ( diff.appended.length > 0 || diff.truncated.length > 0 || @@ -1877,6 +2058,10 @@ function isWarmNoOp( !previous.coverage.period.allTime.complete || previous.coverage.period.timeZone !== timeZone || previous.coverage.period.asOfDay !== asOfDay || + !hourlyCoverage?.complete || + hourlyCoverage.timeZone !== timeZone || + hourlyCoverage.asOfDay !== asOfDay || + hourlyCoverage.windowDays !== CODEX_ROLLING_HOURLY_DAYS || !previous.coverage.today?.complete || previous.coverage.today.timeZone !== timeZone || previous.coverage.today.day !== asOfDay @@ -1885,6 +2070,7 @@ function isWarmNoOp( } const canonical = classifyCodexSessionDuplicates(previous.files) .canonicalFileKeys; + const rollingDays = rollingHourlyDaySet(asOfDay); return manifest.files.every((entry) => { const contribution = previous.files[entry.fileKey]; const baseIsCurrent = Boolean( @@ -1903,15 +2089,15 @@ function isWarmNoOp( if (!baseIsCurrent || !contribution) { return false; } - if ( - !canonical.has(entry.fileKey) || - !contribution.aggregate.period?.days[asOfDay] - ) { + const hasRollingDay = Object.keys( + contribution.aggregate.period?.days ?? {}, + ).some((day) => rollingDays.has(day)); + if (!canonical.has(entry.fileKey) || !hasRollingDay) { return true; } return Boolean( - contribution.aggregate.today?.day === asOfDay && - contribution.aggregate.today.timeZone === timeZone && + isRollingTodayIndex(contribution.aggregate.today, timeZone) && + contribution.aggregate.today.day === asOfDay && contribution.aggregate.today.indexedThrough >= contribution.offset && !contribution.todayMigration ); @@ -2034,20 +2220,34 @@ export async function updateCodexIndex( delete contribution.periodMigration; } promoteCaughtUpPeriod(contribution, timeZone); - if ( - contribution.aggregate.today && - ( - contribution.aggregate.today.day !== asOfDay || - contribution.aggregate.today.timeZone !== timeZone - ) - ) { - delete contribution.aggregate.today; + const today = contribution.aggregate.today; + if (today) { + if (isRollingTodayIndex(today, timeZone)) { + const previousDay = today.day; + const nextWindow = rollingHourlyDaySet(asOfDay); + const hasPreviouslyFutureUsage = + asOfDay > previousDay && + Object.keys(contribution.aggregate.period?.days ?? {}).some( + (day) => day > previousDay && nextWindow.has(day), + ); + if ( + hasPreviouslyFutureUsage || + !reanchorRollingToday(today, asOfDay, timeZone) + ) { + delete contribution.aggregate.today; + } + } else if (today.day !== asOfDay || today.timeZone !== timeZone) { + delete contribution.aggregate.today; + } } if ( contribution.todayMigration && ( contribution.todayMigration.day !== asOfDay || contribution.todayMigration.timeZone !== timeZone || + contribution.todayMigration.windowDays !== + CODEX_ROLLING_HOURLY_DAYS || + !contribution.todayMigration.days || contribution.todayMigration.prefixEvents !== (contribution.lineage?.desiredPrefixEvents ?? 0) ) @@ -2388,7 +2588,8 @@ export async function updateCodexIndex( } await finishStage(filePasses > mainPassStart, true); - const lineageWork = manifest.files + const allowHistoricalBackfill = options.allowHistoricalBackfill !== false; + const lineageWork = (allowHistoricalBackfill ? manifest.files : []) .filter((entry) => { const contribution = index.files[entry.fileKey]; return Boolean( @@ -2510,7 +2711,7 @@ export async function updateCodexIndex( await finishStage(filePasses > lineagePassStart); const canonical = classifyCodexSessionDuplicates(index.files).canonicalFileKeys; - const periodWork = manifest.files + const periodWork = (allowHistoricalBackfill ? manifest.files : []) .filter((entry) => { if (!canonical.has(entry.fileKey)) { return false; @@ -2633,7 +2834,8 @@ export async function updateCodexIndex( const todayCanonical = classifyCodexSessionDuplicates(index.files) .canonicalFileKeys; - const todayWork = manifest.files + const hourlyWindowDays = rollingHourlyDaySet(asOfDay); + const todayWork = (allowHistoricalBackfill ? manifest.files : []) .filter((entry) => { if (!todayCanonical.has(entry.fileKey)) { return false; @@ -2648,10 +2850,11 @@ export async function updateCodexIndex( contribution.offset > 0 && contribution.offset >= entry.size && !contribution.discardingOversizedLine && - contribution.aggregate.period?.days[asOfDay] && + Object.keys(contribution.aggregate.period?.days ?? {}) + .some((day) => hourlyWindowDays.has(day)) && ( - contribution.aggregate.today?.day !== asOfDay || - contribution.aggregate.today.timeZone !== timeZone || + !isRollingTodayIndex(contribution.aggregate.today, timeZone) || + contribution.aggregate.today.day !== asOfDay || contribution.aggregate.today.indexedThrough !== contribution.offset ) ); @@ -2680,6 +2883,8 @@ export async function updateCodexIndex( const start = migration?.day === asOfDay && migration.timeZone === timeZone && + migration.windowDays === CODEX_ROLLING_HOURLY_DAYS && + migration.days && migration.prefixEvents === (base.lineage?.desiredPrefixEvents ?? 0) ? migration.offset @@ -2727,6 +2932,8 @@ export async function updateCodexIndex( const start = migration?.day === asOfDay && migration.timeZone === timeZone && + migration.windowDays === CODEX_ROLLING_HOURLY_DAYS && + migration.days && migration.prefixEvents === (base.lineage?.desiredPrefixEvents ?? 0) ? migration.offset @@ -2773,7 +2980,9 @@ export async function updateCodexIndex( reconcileLineageForCheckpoint = true; const identityPassStart = filePasses; - const identityWork = manifest.files.sort(recentFirst); + const identityWork = allowHistoricalBackfill + ? manifest.files.sort(recentFirst) + : []; const pendingIdentity = identityWork.filter( (entry) => index.files[entry.fileKey]?.identityChecked !== true, ); @@ -2869,7 +3078,7 @@ export async function updateCodexIndex( pending: !index.coverage.complete || !index.coverage.period.allTime.complete || - !index.coverage.today.complete, + !index.coverage.hourly?.complete, }, }; } @@ -3217,6 +3426,21 @@ function sanitizeHourlySlices( ); } +function sanitizeHourlyDays(value: unknown): CodexHourlyDays { + if (!isRecord(value)) { + return {}; + } + return Object.fromEntries( + Object.entries(value).flatMap(([day, rawHours]) => { + if (rollingDayKeysFromDayKey(day, 1)[0] !== day) { + return []; + } + const hours = sanitizeHourlySlices(rawHours); + return Object.keys(hours).length > 0 ? [[day, hours]] : []; + }), + ); +} + function sanitizeToday(value: unknown): CodexFileTodayIndex | undefined { if ( !isRecord(value) || @@ -3226,11 +3450,24 @@ function sanitizeToday(value: unknown): CodexFileTodayIndex | undefined { ) { return undefined; } + const timeZone = resolveTimeZone(value.timeZone); + const modern = value.windowDays === CODEX_ROLLING_HOURLY_DAYS && + isRecord(value.days); + const days = modern ? sanitizeHourlyDays(value.days) : undefined; + if (days) { + pruneCodexHourlyDays(days, rollingHourlyDaySet(value.day)); + } return { day: value.day, - timeZone: resolveTimeZone(value.timeZone), + timeZone, indexedThrough: Math.max(0, finiteNumber(value.indexedThrough)), - hours: sanitizeHourlySlices(value.hours), + hours: days ? days[value.day] ?? {} : sanitizeHourlySlices(value.hours), + ...(days + ? { + windowDays: CODEX_ROLLING_HOURLY_DAYS, + days, + } + : {}), }; } @@ -3328,6 +3565,8 @@ function sanitizeTodayMigration( timeZone: value.timeZone, indexedThrough: value.offset, hours: value.hours, + windowDays: value.windowDays, + days: value.days, }); if (!today) { return undefined; @@ -3344,6 +3583,12 @@ function sanitizeTodayMigration( discardingOversizedLine: value.discardingOversizedLine === true, parserState: sanitizeParserState(value.parserState, fileKey), hours: today.hours, + ...(today.windowDays === CODEX_ROLLING_HOURLY_DAYS && today.days + ? { + windowDays: CODEX_ROLLING_HOURLY_DAYS, + days: today.days, + } + : {}), qualityFlags: sanitizeQualityFlags(value.qualityFlags), }; } @@ -3518,6 +3763,39 @@ function sanitizeCoverage(value: unknown): CodexIndexCoverage { : dayKeyInZone(new Date(Date.now()), timeZone); const last7Days = sanitizeRange(period.last7Days); const last30Days = sanitizeRange(period.last30Days); + const rawHourly = isRecord(coverage.hourly) ? coverage.hourly : {}; + const hourlyTimeZone = resolveTimeZone( + typeof rawHourly.timeZone === 'string' ? rawHourly.timeZone : timeZone, + ); + const candidateHourlyDay = typeof rawHourly.asOfDay === 'string' + ? rawHourly.asOfDay + : ''; + const hasValidHourly = + typeof rawHourly.timeZone === 'string' && + rawHourly.windowDays === CODEX_ROLLING_HOURLY_DAYS && + rollingDayKeysFromDayKey(candidateHourlyDay, 1)[0] === candidateHourlyDay; + const hourlyAsOfDay = hasValidHourly ? candidateHourlyDay : asOfDay; + const allowedHourlyDays = rollingHourlyDaySet(hourlyAsOfDay); + const rawHourlyDays = isRecord(rawHourly.days) ? rawHourly.days : {}; + const hourlyDays = Object.fromEntries( + Object.entries(rawHourlyDays).flatMap(([day, raw]) => { + if (!allowedHourlyDays.has(day) || !isRecord(raw)) { + return []; + } + const indexedFiles = Math.max(0, finiteNumber(raw.indexedFiles)); + const totalFiles = Math.max(0, finiteNumber(raw.totalFiles)); + return [[day, { + day, + indexedFiles, + totalFiles, + indexedBytes: Math.max(0, finiteNumber(raw.indexedBytes)), + totalBytes: Math.max(0, finiteNumber(raw.totalBytes)), + complete: hasValidHourly && raw.complete === true, + }]]; + }), + ); + const hourlyIndexedFiles = Math.max(0, finiteNumber(rawHourly.indexedFiles)); + const hourlyTotalFiles = Math.max(0, finiteNumber(rawHourly.totalFiles)); const rawToday = isRecord(coverage.today) ? coverage.today : {}; const todayTimeZone = resolveTimeZone( typeof rawToday.timeZone === 'string' ? rawToday.timeZone : timeZone, @@ -3562,6 +3840,17 @@ function sanitizeCoverage(value: unknown): CodexIndexCoverage { last30Days, allTime: sanitizeRange(period.allTime), }, + hourly: { + timeZone: hourlyTimeZone, + asOfDay: hourlyAsOfDay, + windowDays: CODEX_ROLLING_HOURLY_DAYS, + indexedFiles: hourlyIndexedFiles, + totalFiles: hourlyTotalFiles, + indexedBytes: Math.max(0, finiteNumber(rawHourly.indexedBytes)), + totalBytes: Math.max(0, finiteNumber(rawHourly.totalBytes)), + complete: hasValidHourly && rawHourly.complete === true, + days: hourlyDays, + }, today: { timeZone: todayTimeZone, day: todayDay, @@ -3605,6 +3894,12 @@ function markLineageRescanRequired(index: CodexIndexV3): CodexIndexV3 { index.coverage.today.indexedFiles = 0; index.coverage.today.indexedBytes = 0; index.coverage.today.complete = false; + if (index.coverage.hourly) { + index.coverage.hourly.indexedFiles = 0; + index.coverage.hourly.indexedBytes = 0; + index.coverage.hourly.complete = false; + index.coverage.hourly.days = {}; + } return index; } diff --git a/src/providers/codex/codexIndexClient.ts b/src/providers/codex/codexIndexClient.ts index 96ceeb0b..bded606b 100644 --- a/src/providers/codex/codexIndexClient.ts +++ b/src/providers/codex/codexIndexClient.ts @@ -33,6 +33,7 @@ interface ActiveRefresh { reject: (error: CodexWorkerError) => void; progressListeners: Array<(progress: CodexIndexProgress) => void>; cancelSent: boolean; + finishing: boolean; profile: 'background' | 'foreground'; } @@ -48,6 +49,9 @@ export class CodexIndexClient { private worker: CodexWorkerLike | null = null; private active: ActiveRefresh | null = null; private disposed = false; + private termination: Promise | null = null; + private terminationFailure: unknown = null; + private disposal: Promise | null = null; constructor(private readonly workerFactory: CodexWorkerFactory = defaultWorkerFactory) {} @@ -91,6 +95,7 @@ export class CodexIndexClient { reject, progressListeners: onProgress ? [onProgress] : [], cancelSent: false, + finishing: false, profile: input.profile ?? 'background', }; worker.postMessage({ type: 'refresh', requestId, ...input }); @@ -108,19 +113,51 @@ export class CodexIndexClient { }); } - dispose(): void { - if (this.disposed) { - return; + async whenIdle(): Promise { + const active = this.active; + if (active) { + try { + await active.promise; + } catch { + // Idle is a lifecycle condition, independent of the refresh outcome. + } + } + if (this.termination) { + try { + await this.termination; + } catch { + // A terminal refresh already surfaces termination failure to callers. + } } + } + + dispose(): Promise { + if (this.disposal) return this.disposal; this.disposed = true; - if (this.active) { - this.active.reject( + const active = this.active; + if (active) { + active.reject( new CodexWorkerError('disposed', 'Codex index client is disposed'), ); this.active = null; } - void this.worker?.terminate(); + const worker = this.worker; this.worker = null; + const terminating = worker + ? Promise.resolve(worker.terminate()) + .then(() => undefined) + .catch((error: unknown) => { + this.terminationFailure ??= error; + throw error; + }) + : Promise.resolve(); + this.disposal = Promise.all([ + terminating, + this.termination ?? Promise.resolve(), + ]).then(() => { + if (this.terminationFailure) throw this.terminationFailure; + }); + return this.disposal; } private ensureWorker(): CodexWorkerLike { @@ -145,24 +182,58 @@ export class CodexIndexClient { } return; } - this.active = null; - if (message.type === 'result') { - active.resolve(message.result); - } else { - active.reject(new CodexWorkerError(message.error.code, message.error.message)); + if (active.finishing) return; + active.finishing = true; + const terminal = message; + void this.terminateSettledWorker() + .then(() => { + if (this.active === active) this.active = null; + if (terminal.type === 'result') { + active.resolve(terminal.result); + } else { + active.reject(new CodexWorkerError(terminal.error.code, terminal.error.message)); + } + }) + .catch(() => { + if (this.active === active) this.active = null; + active.reject(new CodexWorkerError( + 'worker-failed', + 'Codex index worker could not be stopped safely', + )); + }); + } + + /** A worker owns no steady-state responsibility after one terminal reply. */ + private async terminateSettledWorker(): Promise { + const worker = this.worker; + this.worker = null; + const termination = worker + ? Promise.resolve(worker.terminate()).then(() => undefined) + : Promise.resolve(); + this.termination = termination; + try { + await termination; + } catch (error) { + this.terminationFailure ??= error; + throw error; + } finally { + if (this.termination === termination) this.termination = null; } } private handleWorkerFailure(): void { - if (this.active) { - this.active.reject( + const active = this.active; + if (!active || active.finishing) return; + active.finishing = true; + const settle = (): void => { + if (this.active === active) this.active = null; + active.reject( new CodexWorkerError( 'worker-failed', 'Codex index worker stopped unexpectedly', ), ); - this.active = null; - } - this.worker = null; + }; + void this.terminateSettledWorker().then(settle, settle); } } diff --git a/src/providers/codex/codexIndexWorker.ts b/src/providers/codex/codexIndexWorker.ts index 8d10bb9f..b125c217 100644 --- a/src/providers/codex/codexIndexWorker.ts +++ b/src/providers/codex/codexIndexWorker.ts @@ -112,7 +112,8 @@ export async function runCodexWorkerRefresh( const metadataMs = now() - metadataStarted; const parseStarted = now(); const foreground = request.profile === 'foreground'; - const expeditedBackfill = needsExpeditedBackfill( + const allowHistoricalBackfill = request.allowHistoricalBackfill !== false; + const expeditedBackfill = allowHistoricalBackfill && needsExpeditedBackfill( previous, manifest.files.length, ); @@ -149,6 +150,7 @@ export async function runCodexWorkerRefresh( requestId: request.requestId, progress, }), + allowHistoricalBackfill, ...(filePassPool ? { filePassBatch: filePassPool.run } : {}), }); } finally { diff --git a/src/providers/codex/codexPeriodIndex.ts b/src/providers/codex/codexPeriodIndex.ts index 0fce1bdd..9553dc34 100644 --- a/src/providers/codex/codexPeriodIndex.ts +++ b/src/providers/codex/codexPeriodIndex.ts @@ -34,15 +34,28 @@ export interface CodexHourlySlice { byModel: Record; } +export const CODEX_ROLLING_HOURLY_DAYS = 30; + +/** Sparse local civil-day -> hour buckets. Empty dates and hours are omitted. */ +export type CodexHourlyDays = Record< + string, + Record +>; + /** - * Deliberately ephemeral across civil days: only one target day is persisted, - * and empty hours do not consume index space. + * The historic `day` / `hours` fields remain as the current-day compatibility + * projection. New indexes also persist `days`, a sparse rolling window whose + * semantic width is identified by `windowDays`. */ export interface CodexFileTodayIndex { day: string; timeZone: string; indexedThrough: number; hours: Record; + /** Missing on v2.3.0 one-day sidecars and therefore migration-required. */ + windowDays?: number; + /** Missing on v2.3.0 one-day sidecars and therefore migration-required. */ + days?: CodexHourlyDays; } export interface CodexTodayMigrationState extends CodexJsonlCursor { @@ -52,6 +65,10 @@ export interface CodexTodayMigrationState extends CodexJsonlCursor { tokenEventsSeen: number; parserState: CodexParserState; hours: Record; + /** Missing on v2.3.0 drafts; a missing value restarts the bounded pass. */ + windowDays?: number; + /** Sparse rolling buckets checkpointed with the existing file cursor. */ + days?: CodexHourlyDays; qualityFlags: string[]; } @@ -173,6 +190,43 @@ export function reduceCodexHourlySlice( addTokens(tokenBucket(slice.byModel, event.model ?? 'unknown'), event.tokens); } +/** + * Add an event to a precomputed rolling civil-day window. Callers construct + * `allowedDays` once per file pass so multi-gigabyte backfills do not rebuild a + * 30-day set for every JSONL event. + */ +export function reduceCodexRollingHourlySlice( + days: CodexHourlyDays, + event: NormalizedUsageEvent, + allowedDays: ReadonlySet, + timeZone: string, +): void { + if (!Number.isFinite(event.timestamp) || event.timestamp <= 0) { + return; + } + const day = dayKeyInZone(new Date(event.timestamp), timeZone); + if (!allowedDays.has(day)) { + return; + } + const hours = (days[day] ??= {}); + reduceCodexHourlySlice(hours, event, day, timeZone); + if (Object.keys(hours).length === 0) { + delete days[day]; + } +} + +/** Mutate a sparse rolling map in place, discarding every out-of-window day. */ +export function pruneCodexHourlyDays( + days: CodexHourlyDays, + allowedDays: ReadonlySet, +): void { + for (const day of Object.keys(days)) { + if (!allowedDays.has(day)) { + delete days[day]; + } + } +} + export function reduceCodexStructuralSlice( days: Record, event: CodexStructuralEvent, diff --git a/src/providers/codex/codexProvider.ts b/src/providers/codex/codexProvider.ts index b75ee92f..d7d00779 100644 --- a/src/providers/codex/codexProvider.ts +++ b/src/providers/codex/codexProvider.ts @@ -12,12 +12,14 @@ import { CodexIndexCoverage, CodexIndexProgress, CodexIndexRecovery, + CodexHourlyCoverage, CodexTodayCoverage, CodexIndexV1, createEmptyCodexIndex, isCodexUsageContributionCurrent, loadCodexIndex, } from './codexIndex'; +import { CODEX_ROLLING_HOURLY_DAYS } from './codexPeriodIndex'; import { CodexIndexClient } from './codexIndexClient'; import { CodexWorkerRefreshInput, @@ -40,7 +42,9 @@ export interface CodexIndexClientLike { input: CodexWorkerRefreshInput, onProgress?: (progress: CodexIndexProgress) => void, ): Promise; - dispose(): void; + cancel?(): void; + whenIdle?(): Promise; + dispose(): void | Promise; } export interface CodexProviderSnapshot { @@ -53,6 +57,8 @@ export interface CodexProviderSnapshot { limit: ProviderLimitSnapshot | null; /** Aggregate-only inputs for reset-aligned API-equivalent value estimates. */ weeklyValueInputs?: WeeklyValueInputs; + /** Exact sparse configured-zone hour coverage for the rolling 30-day view. */ + hourlyCoverage?: CodexHourlyCoverage; /** Independent exact-hour backfill state for the current civil day. */ todayCoverage: CodexTodayCoverage; todayPartial: boolean; @@ -246,6 +252,17 @@ function snapshotFromIndex( (right.session.startedAt ?? 0) - (left.session.startedAt ?? 0), ); const limits = latestLimits(contributions); + const hourlyCoverage = index.coverage.hourly ?? { + timeZone: index.coverage.period.timeZone, + asOfDay: index.coverage.period.asOfDay, + windowDays: CODEX_ROLLING_HOURLY_DAYS, + indexedFiles: 0, + totalFiles: 0, + indexedBytes: 0, + totalBytes: 0, + complete: false, + days: {}, + }; return { provider: 'codex', total: aggregateTotal(usageContributions), @@ -255,6 +272,7 @@ function snapshotFromIndex( limits, limit: limits[0] ?? null, weeklyValueInputs: codexWeeklyValueInputs(usageContributions), + hourlyCoverage, todayCoverage: index.coverage.today, todayPartial: !index.coverage.today.complete, }; @@ -275,6 +293,8 @@ export class CodexProvider { private lastProgress: CodexIndexProgress | undefined; private snapshotGeneration = 0; private persistedSnapshotLoad: Promise | null = null; + private disposed = false; + private disposal: Promise | null = null; constructor( private readonly options: CodexProviderOptions, @@ -283,7 +303,7 @@ export class CodexProvider { ) {} async isAvailable(): Promise { - if (!this.options.enabled) { + if (this.disposed || !this.options.enabled) { return false; } const [sessions, archive] = await Promise.all([ @@ -301,6 +321,7 @@ export class CodexProvider { * lineage reconciliation continues in the background. */ async loadPersistedSnapshot(): Promise { + if (this.disposed) return null; if (this.currentSnapshot) { return this.currentSnapshot; } @@ -346,7 +367,14 @@ export class CodexProvider { async refresh( profile: 'background' | 'foreground' = 'background', onProgress?: (progress: CodexIndexProgress) => void, + allowHistoricalBackfill: boolean = true, ): Promise { + if (this.disposed) { + return { + outcome: 'unavailable', + snapshot: this.currentSnapshot ?? emptySnapshot(this.options.timeZone), + }; + } if (!(await this.isAvailable())) { return { outcome: 'unavailable', @@ -354,15 +382,17 @@ export class CodexProvider { }; } this.client ??= this.clientFactory(); + const client = this.client; this.lastProgress = undefined; try { - const result = await this.client.refresh( + const result = await client.refresh( { codexHome: this.options.codexHome, indexPath: this.options.indexPath, salt: this.options.salt, timeZone: this.options.timeZone, profile, + allowHistoricalBackfill, }, (progress) => { this.lastProgress = progress; @@ -381,7 +411,9 @@ export class CodexProvider { !result.index.coverage.identity.complete || !result.index.coverage.period.last7Days.complete || !result.index.coverage.period.last30Days.complete || - !result.index.coverage.period.allTime.complete + !result.index.coverage.period.allTime.complete || + (result.index.coverage.hourly !== undefined && + !result.index.coverage.hourly.complete) ? 'partial' : 'success'; return { @@ -400,6 +432,11 @@ export class CodexProvider { }, }; } catch { + try { + await client.dispose(); + } finally { + if (this.client === client) this.client = null; + } return { outcome: 'error', snapshot: this.currentSnapshot ?? emptySnapshot(this.options.timeZone), @@ -412,9 +449,25 @@ export class CodexProvider { return this.currentSnapshot; } - dispose(): void { + cancel(): void { + this.client?.cancel?.(); + } + + async cancelAndWait(): Promise { + const client = this.client; + client?.cancel?.(); + await client?.whenIdle?.(); + } + + async dispose(): Promise { + if (this.disposal) return this.disposal; + this.disposed = true; this.snapshotGeneration += 1; - this.client?.dispose(); + const client = this.client; this.client = null; + this.disposal = Promise.resolve() + .then(() => client?.dispose()) + .then(() => undefined); + await this.disposal; } } diff --git a/src/providers/codex/codexUsage.ts b/src/providers/codex/codexUsage.ts index b2b6e26d..d84c1a96 100644 --- a/src/providers/codex/codexUsage.ts +++ b/src/providers/codex/codexUsage.ts @@ -1,13 +1,15 @@ import { CodexProviderSnapshot } from './codexProvider'; import { CodexFileAggregate, + CodexHourlyCoverage, CodexIndexCoverage, CodexPeriodCoverage, CodexRangeCoverage, CodexStructuralSummary, CodexTodayCoverage, } from './codexIndex'; -import { rollingDayKeysFromDayKey } from '../../dateKeys'; +import { formatHourLabel, rollingDayKeysFromDayKey } from '../../dateKeys'; +import { CODEX_ROLLING_HOURLY_DAYS } from './codexPeriodIndex'; import { freshInputPlusOutput, processedTokens, @@ -132,6 +134,7 @@ export interface CodexPeriodUsageView { export interface CodexHourlyUsageView { hour: string; + label: string; total: CodexMetricTotals; apiEquivalent: EquivalentCostBreakdown; threads: number; @@ -170,6 +173,9 @@ export interface CodexUsageView { /** Exact, sparse current-day hours from the independently resumable sidecar. */ todayHourly: CodexHourlyUsageView[]; todayCoverage: CodexTodayCoverage; + /** Sparse data-day -> exact hour rows for the configured rolling window. */ + last30DaysHourlyByDay: Record; + hourlyCoverage: CodexHourlyCoverage; lastTask: CodexUsageScopeView | null; lastTaskIdentity: CodexTaskIdentityView | null; last7Days: CodexUsageScopeView; @@ -591,7 +597,7 @@ function monthlyRows( }); } -function todayHourlyRows( +function hourlyRows( files: CodexFileAggregate[], day: string, timeZone: string, @@ -605,10 +611,16 @@ function todayHourlyRows( } >(); for (const file of files) { - if (file.today?.day !== day || file.today.timeZone !== timeZone) { + if (!file.today || file.today.timeZone !== timeZone) { continue; } - for (const [hour, slice] of Object.entries(file.today.hours)) { + const slices = + file.today.windowDays === CODEX_ROLLING_HOURLY_DAYS && file.today.days + ? file.today.days[day] ?? {} + : file.today.day === day + ? file.today.hours + : {}; + for (const [hour, slice] of Object.entries(slices)) { if (!/^(?:[01]\d|2[0-3])$/.test(hour)) { continue; } @@ -629,6 +641,7 @@ function todayHourlyRows( const total = metrics(row.tokens); return { hour, + label: formatHourLabel(hour), total, apiEquivalent: apiEquivalentForBuckets(row.models, total.processed), threads: row.threads.size, @@ -933,6 +946,18 @@ export function buildCodexUsageView( const periodCoverage = snapshot.coverage.period; const last7DayKeys = rollingDayKeysFromDayKey(periodCoverage.asOfDay, 7); const last30DayKeys = rollingDayKeysFromDayKey(periodCoverage.asOfDay, 30); + const hourlyCoverage = snapshot.hourlyCoverage ?? + snapshot.coverage.hourly ?? { + timeZone: periodCoverage.timeZone, + asOfDay: periodCoverage.asOfDay, + windowDays: CODEX_ROLLING_HOURLY_DAYS, + indexedFiles: 0, + totalFiles: 0, + indexedBytes: 0, + totalBytes: 0, + complete: false, + days: {}, + }; const periodContext: CodexThreadPeriodContext = { recentSessionKeys: new Set(recent.map(sessionIdentityKey)), last7DayKeys: new Set(last7DayKeys), @@ -982,6 +1007,14 @@ export function buildCodexUsageView( ); const allTime = scope(snapshot.files, aggregateIndexIncomplete); const daily = dailyRows(snapshot.files, periodCoverage.timeZone); + const last30DaysHourlyByDay = Object.fromEntries( + last30DayKeys.flatMap((day) => { + const rows = hourlyRows(snapshot.files, day, periodCoverage.timeZone); + return rows.length > 0 || (hourlyCoverage.days[day]?.totalFiles ?? 0) > 0 + ? [[day, rows]] + : []; + }), + ); const taskRoot = taskRootFile(recent); const taskIdentityFile = taskRoot ?? fallbackTaskIdentityFile(recent); const sourceLimits = snapshot.limits.length > 0 @@ -1020,12 +1053,14 @@ export function buildCodexUsageView( return { today: todayScope, - todayHourly: todayHourlyRows( + todayHourly: hourlyRows( snapshot.files, periodCoverage.asOfDay, periodCoverage.timeZone, ), todayCoverage: snapshot.coverage.today, + last30DaysHourlyByDay, + hourlyCoverage, lastTask: recentScope, lastTaskIdentity: recent.length > 0 ? { diff --git a/src/providers/codex/codexWorkerProtocol.ts b/src/providers/codex/codexWorkerProtocol.ts index 0a74aae4..b24a4233 100644 --- a/src/providers/codex/codexWorkerProtocol.ts +++ b/src/providers/codex/codexWorkerProtocol.ts @@ -12,6 +12,9 @@ export interface CodexWorkerRefreshInput { timeZone: string; /** Controls the steady-state budget after any first-time/migration backfill. */ profile?: 'background' | 'foreground'; + /** False keeps changed-file/tail maintenance active but suppresses unchanged + * historical reconciliation until the persisted eligibility time. */ + allowHistoricalBackfill?: boolean; } export type CodexWorkerRequest = diff --git a/src/resourceOwnership.ts b/src/resourceOwnership.ts new file mode 100644 index 00000000..6a07755d --- /dev/null +++ b/src/resourceOwnership.ts @@ -0,0 +1,204 @@ +export const RESOURCE_KINDS = ['timer', 'watcher', 'worker', 'network', 'backfill'] as const; + +export const RESOURCE_CAPABILITIES = [ + 'refresh', + 'quota', + 'codex-index', + 'codex-history', + 'hourly-history', + 'advice-personalization', +] as const; + +export const RESOURCE_SCOPES = ['extension', 'claude', 'codex', 'advice'] as const; + +export const RESOURCE_CREATORS = [ + 'extension', + 'refresh-coordinator', + 'claude-api-client', + 'codex-index-client', + 'codex-index-worker', + 'codex-file-pass-pool', + 'advice-runtime', +] as const; + +export const RESOURCE_STOP_CONDITIONS = [ + 'settled', + 'completed', + 'cancelled', + 'window-blur', + 'feature-disabled', + 'extension-dispose', + 'user-pause', + 'settings-change', + 'profile-change', +] as const; + +export const RESOURCE_BOUNDED_EXCEPTIONS = ['none', 'first-codex-history'] as const; + +export type ResourceKind = typeof RESOURCE_KINDS[number]; +export type ResourceCapability = typeof RESOURCE_CAPABILITIES[number]; +export type ResourceScope = typeof RESOURCE_SCOPES[number]; +export type ResourceCreator = typeof RESOURCE_CREATORS[number]; +export type ResourceStopCondition = typeof RESOURCE_STOP_CONDITIONS[number]; +export type ResourceBoundedException = typeof RESOURCE_BOUNDED_EXCEPTIONS[number]; + +export interface ResourceDescriptor { + readonly kind: ResourceKind; + readonly capability: ResourceCapability; + readonly scope: ResourceScope; + readonly creator: ResourceCreator; + readonly stopConditions: readonly ResourceStopCondition[]; + readonly boundedException: ResourceBoundedException; +} + +export interface ResourceLease { + readonly descriptor: ResourceDescriptor; + readonly active: boolean; + + /** Runs the real close/abort/terminate/settle action. Ownership is released + * only after that action resolves; a rejected action leaves the lease active. */ + stop( + condition: ResourceStopCondition, + actualStopOrSettle: () => void | Promise, + ): Promise; +} + +export interface ResourceOwnershipSnapshot { + readonly activeCount: number; + readonly byKind: Readonly>; + readonly active: readonly ResourceDescriptor[]; +} + +const DESCRIPTOR_FIELDS = [ + 'kind', + 'capability', + 'scope', + 'creator', + 'stopConditions', + 'boundedException', +] as const; + +const kindSet = new Set(RESOURCE_KINDS); +const capabilitySet = new Set(RESOURCE_CAPABILITIES); +const scopeSet = new Set(RESOURCE_SCOPES); +const creatorSet = new Set(RESOURCE_CREATORS); +const stopConditionSet = new Set(RESOURCE_STOP_CONDITIONS); +const boundedExceptionSet = new Set(RESOURCE_BOUNDED_EXCEPTIONS); + +function isRecord(value: unknown): value is Record { + return typeof value === 'object' && value !== null && !Array.isArray(value); +} + +function assertExactDescriptorFields(value: Record): void { + const keys = Object.keys(value); + if (keys.length !== DESCRIPTOR_FIELDS.length || + !DESCRIPTOR_FIELDS.every((field) => Object.prototype.hasOwnProperty.call(value, field))) { + throw new TypeError('resource ownership metadata contains an unknown or missing field'); + } +} + +function checkedDescriptor(value: ResourceDescriptor): ResourceDescriptor { + if (!isRecord(value)) throw new TypeError('resource descriptor must be an object'); + assertExactDescriptorFields(value); + if (!kindSet.has(value.kind)) throw new TypeError('resource kind must be fixed'); + if (!capabilitySet.has(value.capability)) throw new TypeError('resource capability must be fixed'); + if (!scopeSet.has(value.scope)) throw new TypeError('resource scope must be fixed'); + if (!creatorSet.has(value.creator)) throw new TypeError('resource creator must be fixed'); + if (!boundedExceptionSet.has(value.boundedException)) { + throw new TypeError('resource bounded exception must be fixed'); + } + if (!Array.isArray(value.stopConditions) || value.stopConditions.length === 0) { + throw new TypeError('resource stop conditions must be a non-empty fixed list'); + } + const seen = new Set(); + for (const condition of value.stopConditions) { + if (!stopConditionSet.has(condition)) throw new TypeError('resource stop condition must be fixed'); + if (seen.has(condition)) throw new TypeError('resource stop conditions contain a duplicate'); + seen.add(condition); + } + if (value.boundedException === 'first-codex-history' && + (value.kind !== 'backfill' || value.capability !== 'codex-history' || value.scope !== 'codex')) { + throw new TypeError('first Codex history bounded exception is only valid for a Codex history backfill'); + } + + return Object.freeze({ + kind: value.kind, + capability: value.capability, + scope: value.scope, + creator: value.creator, + stopConditions: Object.freeze([...value.stopConditions]), + boundedException: value.boundedException, + }); +} + +function emptyKindCounts(): Record { + return { + timer: 0, + watcher: 0, + worker: 0, + network: 0, + backfill: 0, + }; +} + +/** Test-observable ownership bookkeeping only. It neither schedules resources + * nor retains callbacks, paths, request bodies, or other user content. */ +export class ResourceOwnershipRegistry { + private nextLeaseId = 1; + private readonly activeDescriptors = new Map(); + + register(input: ResourceDescriptor): ResourceLease { + const descriptor = checkedDescriptor(input); + const leaseId = this.nextLeaseId; + this.nextLeaseId += 1; + this.activeDescriptors.set(leaseId, descriptor); + + let active = true; + let stopping: Promise | undefined; + const registry = this; + return { + descriptor, + get active(): boolean { + return active; + }, + stop( + condition: ResourceStopCondition, + actualStopOrSettle: () => void | Promise, + ): Promise { + if (!stopConditionSet.has(condition) || !descriptor.stopConditions.includes(condition)) { + return Promise.reject(new TypeError('stop condition is not declared by this resource')); + } + if (typeof actualStopOrSettle !== 'function') { + return Promise.reject(new TypeError('actual stop or settle action must be a function')); + } + if (!active) return Promise.resolve(false); + if (stopping !== undefined) return stopping; + + stopping = Promise.resolve() + .then(actualStopOrSettle) + .then(() => { + if (!active) return false; + active = false; + registry.activeDescriptors.delete(leaseId); + return true; + }) + .catch((error: unknown) => { + stopping = undefined; + throw error; + }); + return stopping; + }, + }; + } + + snapshotForTests(): ResourceOwnershipSnapshot { + const byKind = emptyKindCounts(); + const active = [...this.activeDescriptors.values()]; + for (const descriptor of active) byKind[descriptor.kind] += 1; + return Object.freeze({ + activeCount: active.length, + byKind: Object.freeze(byKind), + active: Object.freeze(active), + }); + } +} diff --git a/src/settings.ts b/src/settings.ts index 5308a2f5..e306d3f1 100644 --- a/src/settings.ts +++ b/src/settings.ts @@ -563,15 +563,12 @@ export const SETTINGS: SettingDef[] = [ storage: 'state', group: 'advice', label: 'Enable AI advice effectiveness preview', - help: 'Off by default. Shows the local-first v2.3.1 candidate evidence, payload preview, feedback, and comparison interface without enabling a network transport.', + help: 'Off by default. Shows local evidence, an optional exact BYOK request preview/send flow, local feedback, and comparable-task results. Previewing never sends; sending requires a separate click.', providers: ['claude', 'codex'], }, - // NOTE: the 'subscription' backend (call Anthropic with the Claude Code OAuth - // session, no API key) is intentionally NOT shipped in this version. Anthropic - // returns 403 "Request not allowed" for that gray-area use of the OAuth token - // (it only succeeds by routing around the TLS-fingerprint gate via curl), so - // it is too fragile/inappropriate for a public extension. The transport code - // stays in advisor.ts, dormant, to re-enable if direct calls become allowed. + // Claude Code OAuth/subscription credentials remain quota-only. AI requests + // have no dormant subscription transport: both surfaces require an exact + // Prepared preview plus a separate Send click through configured BYOK. { key: 'advice.apiKey', type: 'string', @@ -579,7 +576,7 @@ export const SETTINGS: SettingDef[] = [ storage: 'config', group: 'advice', label: 'API key', - help: 'For the api backend. Stays in VS Code settings.', + help: 'Bring-your-own key for the configured endpoint. It stays in VS Code settings and is never placed in a preview body.', secret: true, }, { @@ -625,7 +622,7 @@ export const SETTINGS: SettingDef[] = [ default: 30, storage: 'state', group: 'advice', - label: 'Prompt sample window (days)', + label: 'Evidence and prompt window (days)', min: 1, max: 365, }, @@ -636,7 +633,7 @@ export const SETTINGS: SettingDef[] = [ storage: 'state', group: 'advice', label: 'Personal/project context', - help: 'Optional background; adds a "Personalised" section.', + help: 'Optional free text. It may leave the machine only with separate prompt-personalization consent and appears verbatim in the exact request preview.', multiline: true, }, { @@ -646,7 +643,7 @@ export const SETTINGS: SettingDef[] = [ storage: 'state', group: 'advice', label: 'Enable Usage Optimizer', - help: 'Show the opt-in Optimizer card on the Content tab.', + help: 'Show the opt-in Optimizer card on the Content tab. Its user-draft-only request is previewed before a separate Send action.', }, ]; diff --git a/src/test/adviceComparisonPairing.test.ts b/src/test/adviceComparisonPairing.test.ts new file mode 100644 index 00000000..8bfc74b4 --- /dev/null +++ b/src/test/adviceComparisonPairing.test.ts @@ -0,0 +1,148 @@ +import { test } from 'node:test'; +import * as assert from 'node:assert/strict'; + +import { + ComparableObservation, + ComparablePairBuildInput, + buildComparableTaskPair, +} from '../adviceEffectiveness/comparisonPairing'; + +function observation(value: number, observedAtEpochMs: number): ComparableObservation { + return { + context: { + scope: 'task-cohort', + taskKind: 'small-change', + complexityBand: 'medium', + provider: 'codex', + modelFamily: 'other', + effort: 'high', + measurementProfileVersion: 'codex-aggregate-v1', + metricDefinitionVersion: 'fresh-tokens-v1', + qualityRubricId: 'task-quality-v1', + }, + metric: { + name: 'fresh-tokens', + unit: 'tokens', + direction: 'lower-is-better', + value, + }, + quality: { score: 0.9, passed: 'passed', evidenceCount: 2 }, + evidence: { coverage: 'complete', confidence: 'high', qualityFlags: [] }, + observedAtEpochMs, + }; +} + +function input(): ComparablePairBuildInput { + return { + adviceId: 'advice-day-2', + recommendationId: 'reduce-fresh-tokens', + recommendationVersion: 'recommendation-v1', + before: observation(1_000, 1_777_000_000_000), + after: observation(800, 1_777_086_400_000), + recordedAtEpochMs: 1_777_086_400_001, + }; +} + +test('builds a deterministic comparable pair from matching sanitized observations only', () => { + const candidate = input(); + const before = JSON.stringify(candidate); + const first = buildComparableTaskPair(candidate); + const second = buildComparableTaskPair(candidate); + assert.equal(first.ok, true); + assert.deepEqual(second, first); + assert.equal(JSON.stringify(candidate), before, 'pairing must not mutate observations'); + if (!first.ok) return; + assert.match(first.value.pairId, /^pair-[a-f0-9]{32}$/); + assert.equal(first.value.context.scope, 'task-cohort'); + assert.equal(first.value.context.measurementProfileVersion, 'codex-aggregate-v1'); + assert.equal(first.value.recommendationVersion, 'recommendation-v1'); + assert.equal(first.value.metric.beforeValue, 1_000); + assert.equal(first.value.metric.afterValue, 800); + assert.doesNotMatch( + JSON.stringify(first.value), + /prompt|session|path|body/i, + 'the persisted pair is aggregate-only and content-free', + ); +}); + +test('returns evidence-insufficient for every mismatched comparison dimension', () => { + const mismatches: Array<(candidate: ComparablePairBuildInput) => void> = [ + (candidate) => { candidate.after.context.scope = 'project'; }, + (candidate) => { candidate.after.context.taskKind = 'feature'; }, + (candidate) => { candidate.after.context.complexityBand = 'high'; }, + (candidate) => { candidate.after.context.provider = 'claude'; }, + (candidate) => { candidate.after.context.modelFamily = 'sonnet'; }, + (candidate) => { candidate.after.context.effort = 'medium'; }, + (candidate) => { candidate.after.context.measurementProfileVersion = 'codex-aggregate-v2'; }, + (candidate) => { candidate.after.context.metricDefinitionVersion = 'fresh-tokens-v2'; }, + (candidate) => { candidate.after.context.qualityRubricId = 'task-quality-v2'; }, + (candidate) => { candidate.after.metric.name = 'processed-tokens'; }, + (candidate) => { candidate.after.metric.unit = 'count'; }, + (candidate) => { candidate.after.metric.direction = 'higher-is-better'; }, + ]; + + for (const change of mismatches) { + const candidate = input(); + change(candidate); + const result = buildComparableTaskPair(candidate); + assert.equal(result.ok, false); + if (!result.ok) assert.equal(result.reason, 'evidence-insufficient'); + } +}); + +test('returns explicit evidence-insufficient issues for unreliable observations', () => { + const cases: Array<{ + issue: string; + change: (candidate: ComparablePairBuildInput) => void; + }> = [ + { issue: 'incomplete-coverage', change: (candidate) => { candidate.before.evidence.coverage = 'partial'; } }, + { issue: 'insufficient-confidence', change: (candidate) => { candidate.after.evidence.confidence = 'low'; } }, + { issue: 'unresolved-quality-flags', change: (candidate) => { candidate.after.evidence.qualityFlags = ['schema-drift']; } }, + { issue: 'unknown-quality-result', change: (candidate) => { candidate.after.quality.passed = 'unknown'; } }, + { issue: 'missing-quality-evidence', change: (candidate) => { candidate.before.quality.evidenceCount = 0; } }, + { issue: 'failed-quality-baseline', change: (candidate) => { candidate.before.quality.passed = 'failed'; } }, + { issue: 'unknown-effort', change: (candidate) => { + candidate.before.context.effort = 'unknown'; + candidate.after.context.effort = 'unknown'; + } }, + { issue: 'zero-metric-baseline', change: (candidate) => { candidate.before.metric.value = 0; } }, + { issue: 'invalid-observation-order', change: (candidate) => { candidate.after.observedAtEpochMs = candidate.before.observedAtEpochMs; } }, + ]; + + for (const { issue, change } of cases) { + const candidate = input(); + change(candidate); + const result = buildComparableTaskPair(candidate); + assert.equal(result.ok, false); + if (!result.ok) { + assert.equal(result.reason, 'evidence-insufficient'); + assert.ok(result.issues.includes(issue as never), `${issue} should be explicit`); + } + } +}); + +test('rejects prompt, session, path, body, and any other unknown field at every boundary', () => { + const hostileValues: unknown[] = [ + { ...input(), prompt: 'PRIVATE_PROMPT' }, + { ...input(), sessionId: 'PRIVATE_SESSION' }, + { ...input(), body: 'PRIVATE_BODY' }, + (() => { + const candidate = input() as ComparablePairBuildInput & { before: ComparableObservation & { projectPath: string } }; + candidate.before.projectPath = '/Users/private/project'; + return candidate; + })(), + (() => { + const candidate = input() as ComparablePairBuildInput & { + after: ComparableObservation & { context: ComparableObservation['context'] & { unknown: string } }; + }; + candidate.after.context.unknown = 'not-allowed'; + return candidate; + })(), + ]; + + for (const hostile of hostileValues) { + const result = buildComparableTaskPair(hostile); + assert.equal(result.ok, false); + if (!result.ok) assert.equal(result.reason, 'invalid-input'); + } +}); diff --git a/src/test/adviceComparisonProduction.test.ts b/src/test/adviceComparisonProduction.test.ts new file mode 100644 index 00000000..867cf0f7 --- /dev/null +++ b/src/test/adviceComparisonProduction.test.ts @@ -0,0 +1,239 @@ +import { test } from 'node:test'; +import * as assert from 'node:assert/strict'; +import Module = require('node:module'); + +import { + CodexComparableTaskProjection, + buildAppliedComparablePairs, +} from '../adviceEffectiveness/comparisonProduction'; +import { adaptCodexLocalAdvice } from '../adviceEffectiveness/adapters'; +import { createClosedAdviceLocalState } from '../adviceEffectiveness/versionedPersistence'; + +function projection(observedAtEpochMs: number, fresh: number): CodexComparableTaskProjection { + return { + observedAtEpochMs, + total: { processed: fresh * 2, fresh }, + structural: { + patchCalls: 1, + toolCalls: 4, + postPatchToolCalls: 2, + taskCompleteCount: 1, + }, + childFreshShare: 0, + approvalReviewerFreshShare: 0, + modelFamilies: ['other'], + efforts: ['high'], + coverage: { complete: true, identityComplete: true, qualityFlags: [] }, + }; +} + +test('pairs five real completed Codex task projections around the applied timestamp', () => { + const input = { + adviceId: 'advice-codex-30d-1', + recommendationId: 'recommendation-codex-effort-comparison', + recommendationVersion: 'codex-local-rule-v1', + appliedAtEpochMs: 1_000, + tasks: [ + ...Array.from({ length: 5 }, (_, index) => projection(100 + index, 100)), + ...Array.from({ length: 5 }, (_, index) => projection(1_100 + index, 80)), + ], + }; + const before = JSON.stringify(input); + const result = buildAppliedComparablePairs(input); + assert.equal(result.ok, true); + assert.equal(JSON.stringify(input), before, 'production must not mutate materialized projections'); + if (!result.ok) return; + assert.equal(result.status, 'comparable'); + assert.equal(result.pairs.length, 5); + assert.equal(result.pairs[0].context.scope, 'task-cohort'); + assert.equal(result.pairs[0].context.taskKind, 'small-change'); + assert.equal(result.pairs[0].context.measurementProfileVersion, 'codex-task-structural-v1'); + assert.equal(result.pairs[0].quality.beforePassed, 'passed'); + assert.equal(result.pairs[0].quality.afterPassed, 'passed'); + assert.doesNotMatch(JSON.stringify(result), /prompt|session|path|body/i); +}); + +test('fails closed to evidence-insufficient when tasks are incomplete, mixed, or unreliable', () => { + const base = { + adviceId: 'advice-codex-30d-1', + recommendationId: 'recommendation-codex-effort-comparison', + recommendationVersion: 'codex-local-rule-v1', + appliedAtEpochMs: 1_000, + }; + const unreliable = projection(1_100, 80); + unreliable.coverage.qualityFlags = ['schema-drift']; + const unknownEffort = projection(1_101, 80); + unknownEffort.efforts = ['unknown']; + const incomplete = projection(1_102, 80); + incomplete.structural.taskCompleteCount = 0; + const differentComplexity = projection(1_103, 500_000); + + const result = buildAppliedComparablePairs({ + ...base, + tasks: [projection(100, 100), unreliable, unknownEffort, incomplete, differentComplexity], + }); + assert.equal(result.ok, true); + if (!result.ok) return; + assert.equal(result.status, 'evidence-insufficient'); + assert.equal(result.pairs.length, 0); +}); + +test('rejects sensitive or unknown fields before creating observations', () => { + const base = { + adviceId: 'advice-codex-30d-1', + recommendationId: 'recommendation-codex-effort-comparison', + recommendationVersion: 'codex-local-rule-v1', + appliedAtEpochMs: 1_000, + tasks: [projection(100, 100), projection(1_100, 80)], + }; + const hostile: unknown[] = [ + { ...base, prompt: 'PRIVATE_PROMPT' }, + { ...base, sessionId: 'PRIVATE_SESSION' }, + { ...base, projectPath: '/Users/private/project' }, + { ...base, body: 'PRIVATE_BODY' }, + (() => { + const value = JSON.parse(JSON.stringify(base)); + value.tasks[0].title = 'PRIVATE_TITLE'; + return value; + })(), + ]; + for (const value of hostile) { + const result = buildAppliedComparablePairs(value); + assert.equal(result.ok, false); + if (!result.ok) assert.equal(result.reason, 'invalid-input'); + } +}); + +function codexAdvice(insightKinds: Array<'effort-comparison' | 'post-patch-tool-intensity'>) { + const result = adaptCodexLocalAdvice({ + adviceId: 'advice-codex-30d-1', + generatedAt: '2026-08-30T00:00:00.000Z', + locale: 'en', + scope: '30d', + insights: insightKinds.map((kind) => ({ + kind, + severity: 'normal' as const, + scope: '30d' as const, + evidence: {}, + proxy: true as const, + })), + behavior: { + childFreshShare: 0, + approvalReviewerFreshShare: 0, + highEffortFreshShare: 1, + processedToFreshRatio: 2, + cacheShare: 0.5, + postPatchToolCallsPerPatchCall: 2, + }, + quality: { + indexComplete: true, + identityComplete: true, + periodComplete: true, + qualityFlags: [], + }, + }); + assert.equal(result.ok, true); + if (!result.ok) throw new Error('advice fixture failed'); + return result.value.contract; +} + +function withWebview(run: (provider: any) => T): T { + const originalLoad = (Module as any)._load; + (Module as any)._load = function(request: string, parent: unknown, isMain: boolean) { + if (request === 'vscode') { + return { workspace: { workspaceFolders: [] } }; + } + return originalLoad.call(this, request, parent, isMain); + }; + try { + const { UsageWebviewProvider } = require('../webview') as typeof import('../webview'); + const provider = new UsageWebviewProvider({} as any) as any; + provider.settings = { get: (key: string) => key === 'advice.effectiveness.enabled' }; + provider.adviceLocalStateStatus = 'ready'; + return run(provider); + } finally { + (Module as any)._load = originalLoad; + } +} + +test('webview materializes bounded pairs and a frozen result from the existing Codex view only', () => { + withWebview((provider) => { + const contract = codexAdvice(['effort-comparison']); + provider.adviceEffectivenessStates = { + codex: { + provider: 'codex', + contract, + remotePreviewEligible: false, + promptSamples: [], + }, + }; + const local = createClosedAdviceLocalState(); + local.featureMode = 'enabled'; + local.feedback.push({ + adviceId: contract.adviceId, + recommendationId: contract.recommendations[0].id, + rating: 'helpful', + applied: 'applied', + appliedAtEpochMs: 1_000, + updatedAtEpochMs: 1_000, + }); + provider.adviceLocalState = local; + provider.codexView = { + recentThreads: [ + ...Array.from({ length: 5 }, (_, index) => ({ + rootTaskViewKey: `before-${index}`, + role: 'root', + observedAt: 100 + index, + total: { processed: 200, fresh: 100, input: 100, cachedInput: 0, output: 0, reasoning: 0 }, + structural: { patchCalls: 1, toolCalls: 4, postPatchToolCalls: 2, compactCount: 0, taskCompleteCount: 1 }, + models: ['gpt-5'], + efforts: ['high'], + })), + ...Array.from({ length: 5 }, (_, index) => ({ + rootTaskViewKey: `after-${index}`, + role: 'root', + observedAt: 1_100 + index, + total: { processed: 160, fresh: 80, input: 80, cachedInput: 0, output: 0, reasoning: 0 }, + structural: { patchCalls: 1, toolCalls: 4, postPatchToolCalls: 2, compactCount: 0, taskCompleteCount: 1 }, + models: ['gpt-5'], + efforts: ['high'], + })), + ], + coverage: { complete: true, identity: { complete: true } }, + qualityFlags: [], + }; + + const next = provider.materializeAdviceComparisonState(2_000); + assert.equal(next.comparablePairs.length, 5); + assert.equal(next.comparisonResults.length, 1); + assert.equal(next.comparisonResults[0].result.status, 'improved'); + assert.doesNotMatch(JSON.stringify(next.comparablePairs), /before-|after-|gpt-5|prompt|session|path|body/i); + }); +}); + +test('advice rendering gives every recommendation its own feedback and comparison boundary', () => { + withWebview((provider) => { + const contract = codexAdvice(['effort-comparison', 'post-patch-tool-intensity']); + provider.adviceEffectivenessStates = { + codex: { + provider: 'codex', + contract, + remotePreviewEligible: false, + promptSamples: [], + }, + }; + provider.adviceLocalState = createClosedAdviceLocalState(); + const html = provider.renderAdviceEffectivenessBody('codex'); + for (const recommendation of contract.recommendations) { + const matches = html.match( + new RegExp(`]+data-recommendation-id="${recommendation.id}"`, 'g'), + ) ?? []; + assert.equal(matches.length, 3, `${recommendation.id} needs three independent feedback buttons`); + } + assert.doesNotMatch( + provider.renderAdviceEffectivenessBody.toString(), + /recommendations\[0\]/, + 'the rendering boundary must not collapse the contract to its first recommendation', + ); + }); +}); diff --git a/src/test/adviceComparisonResult.test.ts b/src/test/adviceComparisonResult.test.ts new file mode 100644 index 00000000..2031cf90 --- /dev/null +++ b/src/test/adviceComparisonResult.test.ts @@ -0,0 +1,189 @@ +import { test } from 'node:test'; +import * as assert from 'node:assert/strict'; + +import { + AdviceComparisonResultBuildInput, + AdviceComparisonResultEnvelope, + buildAdviceComparisonResultEnvelope, + parseAdviceComparisonResultEnvelope, +} from '../adviceEffectiveness/comparisonResult'; +import { + ComparablePairBuildInput, + StoredComparablePair, + buildComparableTaskPair, +} from '../adviceEffectiveness/comparisonPairing'; + +function pair(index: number, afterValue = 80, afterScore = 0.9, afterPassed: 'passed' | 'failed' = 'passed'): StoredComparablePair { + const base = 1_777_000_000_000 + index * 10_000; + const input: ComparablePairBuildInput = { + adviceId: `advice-${index}`, + recommendationId: 'reduce-token-use', + recommendationVersion: 'recommendation-v1', + before: { + context: { + scope: 'task-cohort', + taskKind: 'small-change', + complexityBand: 'medium', + provider: 'claude', + modelFamily: 'sonnet', + effort: 'high', + measurementProfileVersion: 'claude-aggregate-v1', + metricDefinitionVersion: 'processed-tokens-v1', + qualityRubricId: 'acceptance-v1', + }, + metric: { name: 'processed-tokens', unit: 'tokens', direction: 'lower-is-better', value: 100 }, + quality: { score: 0.9, passed: 'passed', evidenceCount: 2 }, + evidence: { coverage: 'complete', confidence: 'high', qualityFlags: [] }, + observedAtEpochMs: base, + }, + after: { + context: { + scope: 'task-cohort', + taskKind: 'small-change', + complexityBand: 'medium', + provider: 'claude', + modelFamily: 'sonnet', + effort: 'high', + measurementProfileVersion: 'claude-aggregate-v1', + metricDefinitionVersion: 'processed-tokens-v1', + qualityRubricId: 'acceptance-v1', + }, + metric: { name: 'processed-tokens', unit: 'tokens', direction: 'lower-is-better', value: afterValue }, + quality: { score: afterScore, passed: afterPassed, evidenceCount: 2 }, + evidence: { coverage: 'complete', confidence: 'high', qualityFlags: [] }, + observedAtEpochMs: base + 1, + }, + recordedAtEpochMs: base + 2, + }; + const result = buildComparableTaskPair(input); + assert.equal(result.ok, true); + if (!result.ok) throw new Error('fixture pair failed validation'); + return result.value; +} + +function buildInput(pairs: StoredComparablePair[]): AdviceComparisonResultBuildInput { + return { + provider: 'claude', + recommendationId: 'reduce-token-use', + recommendationVersion: 'recommendation-v1', + measurementProfileVersion: 'claude-aggregate-v1', + cohort: { + scope: 'task-cohort', + taskKind: 'small-change', + complexityBand: 'medium', + modelFamily: 'sonnet', + effort: 'high', + metricDefinitionVersion: 'processed-tokens-v1', + qualityRubricId: 'acceptance-v1', + metric: { name: 'processed-tokens', unit: 'tokens', direction: 'lower-is-better' }, + }, + guardrail: { + minComparablePairs: 5, + minRelativeImprovement: 0.1, + minAfterQualityScore: 0.8, + maxMeanQualityRegression: 0.02, + allowedQualityFlags: [], + qualityRubricId: 'acceptance-v1', + }, + pairs, + recordedAtEpochMs: 1_778_000_000_000, + }; +} + +test('freezes an improved comparison in a strict versioned, replayable envelope', () => { + const input = buildInput(Array.from({ length: 5 }, (_, index) => pair(index + 1))); + const before = JSON.stringify(input); + const built = buildAdviceComparisonResultEnvelope(input); + assert.equal(built.ok, true); + assert.equal(JSON.stringify(input), before, 'comparison result creation must be pure'); + if (!built.ok) return; + assert.equal(built.value.schemaVersion, 1); + assert.equal(built.value.provider, 'claude'); + assert.equal(built.value.measurementProfileVersion, 'claude-aggregate-v1'); + assert.equal(built.value.recommendationVersion, 'recommendation-v1'); + assert.equal(built.value.sample.pairIds.length, 5); + assert.equal(built.value.result.status, 'improved'); + assert.deepEqual(built.value.replayEligibility, { + status: 'eligible', + reason: 'conclusive-result', + }); + assert.match(built.value.comparisonId, /^comparison-[a-f0-9]{32}$/); + assert.deepEqual(parseAdviceComparisonResultEnvelope(built.value), built.value); + assert.doesNotMatch(JSON.stringify(built.value), /prompt|session|path|body/i); +}); + +test('freezes too-small and incomparable samples as explicit evidence-insufficient results', () => { + const tooSmall = buildAdviceComparisonResultEnvelope( + buildInput(Array.from({ length: 4 }, (_, index) => pair(index + 1))), + ); + assert.equal(tooSmall.ok, true); + if (tooSmall.ok) { + assert.deepEqual(tooSmall.value.replayEligibility, { + status: 'ineligible', + reason: 'evidence-insufficient', + }); + assert.deepEqual(tooSmall.value.result, { + status: 'evidence-insufficient', + reasonCode: 'minimum-sample-not-met', + comparablePairs: 4, + }); + } + + const mixed = Array.from({ length: 5 }, (_, index) => pair(index + 1)); + mixed[4].context.scope = 'project'; + const incomparable = buildAdviceComparisonResultEnvelope(buildInput(mixed)); + assert.equal(incomparable.ok, true); + if (incomparable.ok) { + assert.equal(incomparable.value.result.status, 'evidence-insufficient'); + if (incomparable.value.result.status === 'evidence-insufficient') { + assert.equal(incomparable.value.result.reasonCode, 'incomparable-sample'); + } + } +}); + +test('records a measured quality guardrail failure without claiming improvement', () => { + const pairs = Array.from({ length: 5 }, (_, index) => pair(index + 1, 70, 0.75, 'failed')); + const result = buildAdviceComparisonResultEnvelope(buildInput(pairs)); + assert.equal(result.ok, true); + if (!result.ok) return; + assert.equal(result.value.result.status, 'quality-guardrail-failed'); + assert.equal(result.value.replayEligibility.status, 'eligible'); +}); + +test('strict envelope parsing rejects sensitive, nested, unknown, and inconsistent fields', () => { + const built = buildAdviceComparisonResultEnvelope( + buildInput(Array.from({ length: 5 }, (_, index) => pair(index + 1))), + ); + assert.equal(built.ok, true); + if (!built.ok) return; + const value = built.value; + const hostileValues: unknown[] = [ + { ...value, prompt: 'PRIVATE_PROMPT' }, + { ...value, sessionId: 'PRIVATE_SESSION' }, + { ...value, body: 'PRIVATE_BODY' }, + { ...value, projectPath: '/Users/private/project' }, + { + ...value, + guardrail: { ...value.guardrail, unknown: true }, + }, + { + ...value, + sample: { ...value.sample, qualityRubricId: 'different-rubric' }, + }, + { + ...value, + replayEligibility: { status: 'ineligible', reason: 'evidence-insufficient' }, + }, + ]; + for (const hostile of hostileValues) { + assert.equal(parseAdviceComparisonResultEnvelope(hostile), undefined); + } + + const cloned: AdviceComparisonResultEnvelope = JSON.parse(JSON.stringify(value)); + cloned.sample.pairIds.reverse(); + assert.equal( + parseAdviceComparisonResultEnvelope(cloned), + undefined, + 'comparisonId protects the frozen envelope from silent mutation', + ); +}); diff --git a/src/test/adviceContract.test.ts b/src/test/adviceContract.test.ts index 9f627641..aea79447 100644 --- a/src/test/adviceContract.test.ts +++ b/src/test/adviceContract.test.ts @@ -75,6 +75,18 @@ test('rejects privacy metadata that claims aggregates-only while carrying prompt assert.ok(result.issues.some((issue) => issue.includes('zero prompt samples'))); }); +test('accepts explicit personal context without pretending that prompt samples were included', () => { + const input = validInput(); + input.privacy = { + dataMode: 'aggregates-with-personalization', + promptSampleConsent: 'explicit', + promptSampleCount: 0, + feedbackStorage: 'local-only', + }; + const result = createAdviceContract(input); + assert.equal(result.ok, true); +}); + test('allows an empty recommendation list to represent no supported conclusion', () => { const input = validInput(); input.recommendations = []; diff --git a/src/test/adviceIntegration.test.ts b/src/test/adviceIntegration.test.ts index adca1bcb..e1e48758 100644 --- a/src/test/adviceIntegration.test.ts +++ b/src/test/adviceIntegration.test.ts @@ -6,6 +6,7 @@ import { createAdviceContract } from '../adviceEffectiveness/contract'; import { AdviceEffectivenessProviderState, prepareAdviceSnapshot, + selectAdvicePromptSamples, } from '../adviceEffectiveness/integration'; import { buildAdviceAggregateSnapshot, @@ -45,6 +46,7 @@ function state(provider: 'claude' | 'codex' = 'claude'): AdviceEffectivenessProv remotePreviewEligible: provider === 'claude', aggregate: provider === 'claude' ? buildAdviceAggregateSnapshot(usageFixture, 'overall', 30) : undefined, + userContext: '私有项目约束', promptSamples: [{ text: '请核对这个非 ASCII 样本' }], }; } @@ -65,6 +67,7 @@ test('sealed snapshot requires separate aggregate consent and defaults to no pro assert.equal(result.value.preview.body, result.value.prepared.serializedBody); assert.equal(result.value.preview.dataMode, 'aggregates-only'); assert.equal(result.value.preview.body.includes('promptSamples'), false); + assert.equal(result.value.preview.body.includes('私有项目约束'), false); }); test('prompt text enters only the explicitly opted-in sealed body and UTF-8 bytes are exact', () => { @@ -77,6 +80,7 @@ test('prompt text enters only the explicitly opted-in sealed body and UTF-8 byte assert.equal(result.value.preview.body, result.value.prepared.serializedBody); assert.equal(result.value.preview.dataMode, 'aggregates-with-prompt-samples'); assert.equal(result.value.preview.promptSampleCount, 1); + assert.equal(result.value.preview.body.includes('私有项目约束'), true); assert.equal( result.value.preview.utf8Bytes, Buffer.byteLength(result.value.prepared.serializedBody, 'utf8'), @@ -84,6 +88,20 @@ test('prompt text enters only the explicitly opted-in sealed body and UTF-8 byte assert.ok(result.value.preview.utf8Bytes > result.value.preview.body.length); }); +test('configured context remains usable when no prompt sample is inside the window', () => { + const contextOnly = state(); + contextOnly.promptSamples = []; + const result = prepareAdviceSnapshot(contextOnly, { + aggregate: 'explicit', + promptSamples: 'explicit', + }); + assert.equal(result.ok, true); + if (!result.ok) return; + assert.equal(result.value.preview.dataMode, 'aggregates-with-personalization'); + assert.equal(result.value.preview.promptSampleCount, 0); + assert.equal(result.value.preview.body.includes('私有项目约束'), true); +}); + test('sealed snapshot preview, digest, and future sender share one canonical UTF-8 byte sequence', async () => { const result = prepareAdviceSnapshot(state(), { aggregate: 'explicit', @@ -122,3 +140,13 @@ test('Codex evidence stays local until a provider-discriminated remote aggregate }); assert.deepEqual(result, { ok: false, reason: 'provider-not-eligible' }); }); + +test('prompt personalisation excludes old, future, and untrusted-age samples', () => { + const now = 1_778_000_000_000; + assert.deepEqual(selectAdvicePromptSamples([ + { text: 'inside', observedAtEpochMs: now - 29 * 86_400_000 }, + { text: 'too old', observedAtEpochMs: now - 31 * 86_400_000 }, + { text: 'future', observedAtEpochMs: now + 1 }, + { text: 'missing age', observedAtEpochMs: 0 }, + ], now, 30), [{ text: 'inside' }]); +}); diff --git a/src/test/adviceLegacyBridge.test.ts b/src/test/adviceLegacyBridge.test.ts index b0cfdd77..097a0383 100644 --- a/src/test/adviceLegacyBridge.test.ts +++ b/src/test/adviceLegacyBridge.test.ts @@ -1,8 +1,8 @@ import { test } from 'node:test'; import * as assert from 'node:assert/strict'; -import { AdviceOptions } from '../advisor'; import { + LegacyByokCallOptions, LegacyByokModelCall, requestStructuredAdviceViaLegacyByok, } from '../adviceEffectiveness/legacyBridge'; @@ -19,7 +19,7 @@ const byok = { test('legacy BYOK bridge uses the sealed serialization as the exact user turn and strict-parses the result', async () => { const prepared = prepareAdvicePayload(payloadInputFixture()); let capturedUser = ''; - let capturedOptions: AdviceOptions | undefined; + let capturedOptions: LegacyByokCallOptions | undefined; const invoke: LegacyByokModelCall = async (_system, userContent, options) => { capturedUser = userContent; capturedOptions = options; @@ -39,7 +39,12 @@ test('legacy BYOK bridge uses the sealed serialization as the exact user turn an assert.equal(result.ok, true); assert.equal(capturedUser, prepared.serializedBody); assert.equal(capturedOptions?.summary, ''); - assert.equal(capturedOptions?.userContext, undefined); + assert.equal( + capturedOptions === undefined + ? false + : Object.prototype.hasOwnProperty.call(capturedOptions, 'userContext'), + false, + ); assert.equal(capturedOptions?.backend, 'api'); }); diff --git a/src/test/advicePayload.test.ts b/src/test/advicePayload.test.ts index eb4a86fb..5ecc816c 100644 --- a/src/test/advicePayload.test.ts +++ b/src/test/advicePayload.test.ts @@ -69,19 +69,55 @@ test('prompt text appears only behind the separate explicit opt-in and is bounde const input = payloadInputFixture(); input.promptSamples = { consent: 'explicit', + userContext: ` private project context ${'y'.repeat(2_000)} `, samples: [{ text: `private prompt ${'x'.repeat(2_000)}` }], }; const prepared = prepareAdvicePayload(input); const parsed = JSON.parse(prepared.serializedBody) as { promptSamples: { text: string }[]; - privacy: { promptSampleConsent: string }; + privacy: { promptSampleConsent: string; userContextIncluded: boolean }; + userContext: string; }; assert.equal(prepared.dataMode, 'aggregates-with-prompt-samples'); assert.equal(parsed.privacy.promptSampleConsent, 'explicit'); + assert.equal(parsed.privacy.userContextIncluded, true); + assert.equal(parsed.userContext.length, 1_000); + assert.equal(parsed.userContext.startsWith('private project context'), true); assert.equal(parsed.promptSamples.length, 1); assert.equal(parsed.promptSamples[0].text.length, 1_000); }); +test('configured user context is absent without the separate prompt-personalisation consent', () => { + const input = payloadInputFixture(); + (input as typeof input & { userContext?: string }).userContext = 'must not be accepted here'; + const prepared = prepareAdvicePayload(input); + const parsed = JSON.parse(prepared.serializedBody) as Record; + assert.equal(Object.prototype.hasOwnProperty.call(parsed, 'userContext'), false); + assert.equal( + (parsed.privacy as { userContextIncluded: boolean }).userContextIncluded, + false, + ); +}); + +test('configured context can be the only explicitly consented personalisation field', () => { + const input = payloadInputFixture(); + input.promptSamples = { + consent: 'explicit', + userContext: 'private project constraint', + samples: [], + }; + const prepared = prepareAdvicePayload(input); + const parsed = JSON.parse(prepared.serializedBody) as { + userContext: string; + privacy: { promptSampleConsent: string; promptSampleCount: number }; + }; + assert.equal(prepared.dataMode, 'aggregates-with-personalization'); + assert.equal(prepared.promptSampleCount, 0); + assert.equal(parsed.userContext, 'private project constraint'); + assert.equal(parsed.privacy.promptSampleConsent, 'explicit'); + assert.equal(parsed.privacy.promptSampleCount, 0); +}); + test('invalid aggregates and empty explicit prompt consent fail before a payload exists', () => { const invalid = payloadInputFixture(); invalid.aggregate.totals.inputTokens = Number.NaN; @@ -89,7 +125,7 @@ test('invalid aggregates and empty explicit prompt consent fail before a payload const empty = payloadInputFixture(); empty.promptSamples = { consent: 'explicit', samples: [{ text: ' ' }] }; - assert.throws(() => prepareAdvicePayload(empty), /at least one non-empty/); + assert.throws(() => prepareAdvicePayload(empty), /sample or user context/); }); test('remote observations require reviewed metrics and sufficiently clean source evidence', () => { diff --git a/src/test/advicePersistence.test.ts b/src/test/advicePersistence.test.ts index dd5728d8..a293b296 100644 --- a/src/test/advicePersistence.test.ts +++ b/src/test/advicePersistence.test.ts @@ -6,9 +6,12 @@ import { compareAdviceEffectiveness } from '../adviceEffectiveness/comparison'; import { ADVICE_LOCAL_STATE_KEY, ADVICE_LOCAL_STATE_VERSION, + MAX_PERSISTED_ADVICE_COMPARISON_RESULTS, AdviceLocalStateStorage, StoredComparablePair, + appendAdviceComparisonResult, appendStoredComparablePair, + createClearedAdviceLocalState, createClosedAdviceLocalState, loadAndMigrateAdviceLocalState, saveAdviceLocalState, @@ -16,6 +19,10 @@ import { toComparableTaskPairs, upsertAdviceLocalFeedback, } from '../adviceEffectiveness/versionedPersistence'; +import { + AdviceComparisonResultEnvelope, + buildAdviceComparisonResultEnvelope, +} from '../adviceEffectiveness/comparisonResult'; class MemoryStorage implements AdviceLocalStateStorage { value: unknown; @@ -46,6 +53,13 @@ test('missing local state defaults to a closed, non-consenting state without wri assert.equal(storage.updates.length, 0); }); +test('user privacy reset keeps the surface enabled while erasing consent and every local result', () => { + assert.deepEqual(createClearedAdviceLocalState(), { + ...createClosedAdviceLocalState(), + featureMode: 'enabled', + }); +}); + test('v1 migration preserves validated local feedback but conservatively closes feature and consent', async () => { const storage = new MemoryStorage(); storage.value = { @@ -71,12 +85,14 @@ test('v1 migration preserves validated local feedback but conservatively closes assert.equal(result.value.featureMode, 'disabled'); assert.equal(result.value.aggregateConsent, 'not-granted'); assert.equal(result.value.promptSampleConsent, 'not-granted'); + assert.deepEqual(result.value.comparisonResults, []); assert.deepEqual(result.value.feedback, [ { adviceId: 'advice-1', recommendationId: 'recommendation-1', rating: 'helpful', applied: 'applied', + appliedAtEpochMs: 1_777_000_000_000, updatedAtEpochMs: 1_777_000_000_000, }, ]); @@ -156,6 +172,7 @@ test('save accepts only the exact enum, numeric, and opaque-id state shape', asy recommendationId: 'recommendation-1', rating: 'not-helpful', applied: 'not-applied', + appliedAtEpochMs: null, updatedAtEpochMs: 1_777_000_000_000, }); const saved = await saveAdviceLocalState(storage, state); @@ -199,9 +216,31 @@ test('feedback upsert keeps rating mutually exclusive while applied remains inde recommendationId: 'recommendation-1', rating: 'not-helpful', applied: 'applied', + appliedAtEpochMs: 1_777_000_000_002, updatedAtEpochMs: 1_777_000_000_003, }, ]); + + const ratingRetracted = upsertAdviceLocalFeedback(notHelpful.value, { + adviceId: 'advice-1', + recommendationId: 'recommendation-1', + kind: 'not-helpful', + updatedAtEpochMs: 1_777_000_000_004, + }); + assert.equal(ratingRetracted.ok, true); + if (!ratingRetracted.ok) return; + assert.equal(ratingRetracted.value.feedback[0].rating, 'unrated'); + assert.equal(ratingRetracted.value.feedback[0].applied, 'applied'); + + const applicationRetracted = upsertAdviceLocalFeedback(ratingRetracted.value, { + adviceId: 'advice-1', + recommendationId: 'recommendation-1', + kind: 'applied', + updatedAtEpochMs: 1_777_000_000_005, + }); + assert.equal(applicationRetracted.ok, true); + if (!applicationRetracted.ok) return; + assert.deepEqual(applicationRetracted.value.feedback, []); }); function comparablePair(): StoredComparablePair { @@ -209,12 +248,15 @@ function comparablePair(): StoredComparablePair { pairId: 'pair-1', adviceId: 'advice-1', recommendationId: 'recommendation-1', + recommendationVersion: 'recommendation-v1', context: { + scope: 'task-cohort', taskKind: 'small-change', complexityBand: 'low', provider: 'codex', modelFamily: 'other', effort: 'high', + measurementProfileVersion: 'codex-aggregate-v1', metricDefinitionVersion: 'fresh-tokens-v1', qualityRubricId: 'task-quality-rubric-v1', }, @@ -245,6 +287,43 @@ function comparablePair(): StoredComparablePair { }; } +function comparisonResult(recordedAtEpochMs: number): AdviceComparisonResultEnvelope { + const pairs = Array.from({ length: 5 }, (_, index) => { + const value = comparablePair(); + value.pairId = `pair-${recordedAtEpochMs}-${index}`; + return value; + }); + const result = buildAdviceComparisonResultEnvelope({ + provider: 'codex', + recommendationId: 'recommendation-1', + recommendationVersion: 'recommendation-v1', + measurementProfileVersion: 'codex-aggregate-v1', + cohort: { + scope: 'task-cohort', + taskKind: 'small-change', + complexityBand: 'low', + modelFamily: 'other', + effort: 'high', + metricDefinitionVersion: 'fresh-tokens-v1', + qualityRubricId: 'task-quality-rubric-v1', + metric: { name: 'fresh-tokens', unit: 'tokens', direction: 'lower-is-better' }, + }, + guardrail: { + minComparablePairs: 5, + minRelativeImprovement: 0.1, + minAfterQualityScore: 0.8, + maxMeanQualityRegression: 0.02, + allowedQualityFlags: [], + qualityRubricId: 'task-quality-rubric-v1', + }, + pairs, + recordedAtEpochMs, + }); + assert.equal(result.ok, true); + if (!result.ok) throw new Error('comparison result fixture failed validation'); + return result.value; +} + test('comparable-pair persistence accepts only coarse enums, finite numbers, and opaque IDs', async () => { const state = createClosedAdviceLocalState(); const appended = appendStoredComparablePair(state, comparablePair()); @@ -314,3 +393,44 @@ test('comparison lineage spans rotating advice instances while preserving strict 'lineage selection must not silently drop an incomparable task cohort', ); }); + +test('legacy exact v2 state migrates in-place to the same ledger with empty comparison results', async () => { + const storage = new MemoryStorage(); + storage.value = { + schemaVersion: ADVICE_LOCAL_STATE_VERSION, + featureMode: 'disabled', + aggregateConsent: 'not-granted', + promptSampleConsent: 'not-granted', + feedback: [], + comparablePairs: [], + }; + const result = await loadAndMigrateAdviceLocalState(storage); + assert.equal(result.ok, true); + if (!result.ok) return; + assert.equal(result.migrated, true); + assert.deepEqual(result.value.comparisonResults, []); + assert.deepEqual(storage.updates, [result.value]); +}); + +test('comparison results share v2 state, remain bounded, and reject duplicate or hostile envelopes', () => { + let state = createClosedAdviceLocalState(); + const first = comparisonResult(1_778_000_000_000); + const appended = appendAdviceComparisonResult(state, first); + assert.equal(appended.ok, true); + if (!appended.ok) return; + state = appended.value; + assert.deepEqual(state.comparisonResults, [first]); + assert.equal(appendAdviceComparisonResult(state, first).ok, false, 'duplicate IDs fail closed'); + + const hostile = { ...comparisonResult(1_778_000_000_001), prompt: 'PRIVATE_PROMPT' }; + assert.equal(appendAdviceComparisonResult(state, hostile).ok, false); + + for (let index = 1; index <= MAX_PERSISTED_ADVICE_COMPARISON_RESULTS; index += 1) { + const result = appendAdviceComparisonResult(state, comparisonResult(1_778_000_000_001 + index)); + assert.equal(result.ok, true); + if (!result.ok) return; + state = result.value; + } + assert.equal(state.comparisonResults.length, MAX_PERSISTED_ADVICE_COMPARISON_RESULTS); + assert.notEqual(state.comparisonResults[0].comparisonId, first.comparisonId); +}); diff --git a/src/test/advicePreparedRequest.test.ts b/src/test/advicePreparedRequest.test.ts new file mode 100644 index 00000000..52e85654 --- /dev/null +++ b/src/test/advicePreparedRequest.test.ts @@ -0,0 +1,129 @@ +import { test } from 'node:test'; +import * as assert from 'node:assert/strict'; + +import { + prepareAiInvocation, + previewAiInvocation, + sendPreparedAiInvocation, +} from '../adviceEffectiveness/preparedRequest'; + +test('Anthropic preview and sender consume the exact same canonical HTTP bytes', async () => { + const prepared = prepareAiInvocation({ + kind: 'advice', + apiFormat: 'anthropic', + apiUrl: 'https://api.anthropic.com', + model: 'claude-sonnet-4-5', + systemPrompt: 'Return strict JSON only.', + userContent: '{"evidence":"aggregate-only"}', + dataMode: 'aggregates-only', + sourceRevision: 'rev-42', + consentGeneration: 3, + createdAtEpochMs: 1_000, + }); + const preview = previewAiInvocation(prepared); + let networkCalls = 0; + let sentBytes: Uint8Array | undefined; + + assert.equal(networkCalls, 0, 'preparing and previewing must not send'); + const result = await sendPreparedAiInvocation( + prepared, + { backend: 'api', apiKey: 'byok-secret' }, + async (request) => { + networkCalls += 1; + sentBytes = request.canonicalBytes; + assert.equal(request.headers['x-api-key'], 'byok-secret'); + assert.equal(Object.values(request.headers).some((value) => preview.body.includes(value)), false); + return 'ok'; + }, + ); + + assert.equal(result, 'ok'); + assert.equal(networkCalls, 1); + assert.strictEqual(sentBytes, prepared.canonicalBytes); + assert.equal(Buffer.from(sentBytes!).toString('utf8'), preview.body); + assert.equal(preview.utf8Bytes, prepared.canonicalBytes.byteLength); + assert.match(preview.sha256, /^[a-f0-9]{64}$/); + assert.deepEqual(JSON.parse(preview.body), { + max_tokens: 16_000, + messages: [{ content: '{"evidence":"aggregate-only"}', role: 'user' }], + model: 'claude-sonnet-4-5', + system: 'Return strict JSON only.', + }); +}); + +test('OpenAI-compatible optimizer uses the same boundary without advice parser fallback', async () => { + const prepared = prepareAiInvocation({ + kind: 'optimizer', + apiFormat: 'openai', + apiUrl: 'https://api.deepseek.com/v1', + model: 'deepseek-chat', + reasoningEffort: 'high', + systemPrompt: 'Use the optimizer markers exactly.', + userContent: 'User pasted draft only', + dataMode: 'user-draft-only', + sourceRevision: 'optimizer-draft-1', + consentGeneration: 1, + createdAtEpochMs: 2_000, + }); + const preview = previewAiInvocation(prepared); + const body = JSON.parse(preview.body); + + assert.equal(prepared.endpoint, 'https://api.deepseek.com/chat/completions'); + assert.equal(body.messages[1].content, 'User pasted draft only'); + assert.equal(body.reasoning_effort, 'high'); + assert.deepEqual(body.thinking, { type: 'enabled' }); + assert.equal(JSON.stringify(body).includes('byok-secret'), false); +}); + +test('prepared request fails closed for subscription OAuth, missing BYOK, stale state, or tampering', async () => { + const prepared = prepareAiInvocation({ + kind: 'advice', + apiFormat: 'openai', + apiUrl: 'https://example.invalid/v1', + model: 'safe-model', + systemPrompt: 'strict', + userContent: '{}', + dataMode: 'aggregates-only', + sourceRevision: 'rev-safe', + consentGeneration: 7, + createdAtEpochMs: 10_000, + }); + let calls = 0; + const sender = async (): Promise => { + calls += 1; + return 'unexpected'; + }; + + await assert.rejects( + sendPreparedAiInvocation(prepared, { backend: 'subscription', apiKey: 'x' } as never, sender), + /BYOK API/, + ); + await assert.rejects( + sendPreparedAiInvocation(prepared, { backend: 'api', apiKey: ' ' }, sender), + /API key/, + ); + await assert.rejects( + sendPreparedAiInvocation( + prepared, + { backend: 'api', apiKey: 'x', expectedSourceRevision: 'different' }, + sender, + ), + /stale/, + ); + await assert.rejects( + sendPreparedAiInvocation( + prepared, + { backend: 'api', apiKey: 'x', expectedConsentGeneration: 8 }, + sender, + ), + /stale/, + ); + + const mutated = prepared as unknown as { serializedBody: string }; + mutated.serializedBody = '{"changed":true}'; + await assert.rejects( + sendPreparedAiInvocation(prepared, { backend: 'api', apiKey: 'x' }, sender), + /integrity/, + ); + assert.equal(calls, 0); +}); diff --git a/src/test/adviceRemoteInvocation.test.ts b/src/test/adviceRemoteInvocation.test.ts new file mode 100644 index 00000000..d69f3d73 --- /dev/null +++ b/src/test/adviceRemoteInvocation.test.ts @@ -0,0 +1,106 @@ +import { test } from 'node:test'; +import * as assert from 'node:assert/strict'; + +import { + prepareStructuredAdviceInvocation, + requestStructuredAdvice, +} from '../adviceEffectiveness/remoteAdvice'; +import { prepareAdvicePayload } from '../adviceEffectiveness/payload'; +import { previewAiInvocation } from '../adviceEffectiveness/preparedRequest'; +import { payloadInputFixture, structuredOutputFixture } from './adviceTestFixtures'; + +const config = { + apiFormat: 'openai' as const, + apiUrl: 'https://example.invalid/v1', + model: 'private-model', +}; + +test('structured advice preview is the complete provider body later sent byte-for-byte', async () => { + const evidence = prepareAdvicePayload(payloadInputFixture()); + const prepared = prepareStructuredAdviceInvocation(evidence, { + ...config, + sourceRevision: 'claude-index-42', + consentGeneration: 3, + createdAtEpochMs: 1_000, + }); + const preview = previewAiInvocation(prepared); + const parsedBody = JSON.parse(preview.body); + let sentBytes: Uint8Array | undefined; + + assert.equal(parsedBody.messages[1].content, evidence.serializedBody); + assert.equal(parsedBody.messages[0].role, 'system'); + assert.equal(prepared.dataMode, 'aggregates-only'); + + const result = await requestStructuredAdvice( + prepared, + { + observationIds: ['observation-cache-share'], + evidenceIds: ['evidence-cache-share'], + }, + { + apiKey: 'PRIVATE_API_KEY', + expectedSourceRevision: 'claude-index-42', + expectedConsentGeneration: 3, + transport: async (request) => { + sentBytes = request.canonicalBytes; + return { + status: 200, + body: JSON.stringify({ + choices: [{ message: { content: JSON.stringify(structuredOutputFixture()) } }], + }), + }; + }, + }, + ); + + assert.equal(result.ok, true); + assert.strictEqual(sentBytes, prepared.canonicalBytes); + assert.equal(Buffer.from(sentBytes!).toString('utf8'), preview.body); + assert.doesNotMatch(preview.body, /PRIVATE_API_KEY/); +}); + +test('structured advice fails closed on stale consent or malformed strict output', async () => { + const prepared = prepareStructuredAdviceInvocation( + prepareAdvicePayload(payloadInputFixture()), + { + ...config, + sourceRevision: 'claude-index-42', + consentGeneration: 3, + createdAtEpochMs: 1_000, + }, + ); + let calls = 0; + const stale = await requestStructuredAdvice( + prepared, + { observationIds: ['observation-cache-share'], evidenceIds: ['evidence-cache-share'] }, + { + apiKey: 'PRIVATE_API_KEY', + expectedConsentGeneration: 4, + transport: async () => { + calls += 1; + return { status: 200, body: '{}' }; + }, + }, + ); + assert.deepEqual(stale, { + ok: false, + code: 'transport-error', + issues: ['configured BYOK transport failed'], + }); + assert.equal(calls, 0); + + const malformed = await requestStructuredAdvice( + prepared, + { observationIds: ['observation-cache-share'], evidenceIds: ['evidence-cache-share'] }, + { + apiKey: 'PRIVATE_API_KEY', + expectedConsentGeneration: 3, + transport: async () => ({ + status: 200, + body: JSON.stringify({ choices: [{ message: { content: '```json\n{}\n```' } }] }), + }), + }, + ); + assert.equal(malformed.ok, false); + if (!malformed.ok) assert.equal(malformed.code, 'invalid-json'); +}); diff --git a/src/test/adviceRuntimePolicy.test.ts b/src/test/adviceRuntimePolicy.test.ts index 5df6b659..30d08a00 100644 --- a/src/test/adviceRuntimePolicy.test.ts +++ b/src/test/adviceRuntimePolicy.test.ts @@ -14,28 +14,75 @@ test('extension advice and optimizer runtime stay API/BYOK-only', () => { assert.match(types, /adviceBackend:\s*'api'/); assert.doesNotMatch(types, /adviceSubscriptionModel/); - const getAdviceStart = source.indexOf('private async getAdvice()'); - const optimizerEnd = source.indexOf('private loadConfiguration()', getAdviceStart); - assert.ok(getAdviceStart >= 0 && optimizerEnd > getAdviceStart); - const runtimeAdvice = source.slice(getAdviceStart, optimizerEnd); - assert.match(runtimeAdvice, /apiKey:\s*config\.adviceApiKey/); - assert.match(runtimeAdvice, /apiUrl:\s*config\.adviceApiUrl/); - assert.doesNotMatch(runtimeAdvice, /getAccessToken\s*\(/); + // Advice and optimizer now share the prepared-request hooks installed during + // activation. Keep the production credential/endpoint assertions attached to + // those hooks instead of the legacy command-method boundaries. + const hooksStart = source.indexOf('this.webviewProvider.onSendOptimizerInvocation'); + const hooksEnd = source.indexOf('this.webviewProvider.settings = this.settings', hooksStart); + assert.ok(hooksStart >= 0 && hooksEnd > hooksStart); + const runtimeHooks = source.slice(hooksStart, hooksEnd); + assert.match(runtimeHooks, /apiKey:\s*config\.adviceApiKey/); + assert.match(runtimeHooks, /apiUrl:\s*config\.adviceApiUrl/); + assert.match(runtimeHooks, /requestPreparedOptimizer\s*\(/); + assert.match(runtimeHooks, /requestStructuredAdvice\s*\(/); + assert.doesNotMatch(runtimeHooks, /getAccessToken\s*\(/); }); -test('future advice seams remain unreachable from production host and webview code', () => { - const production = [ - readFileSync(path.join(root, 'src', 'extension.ts'), 'utf8'), - readFileSync(path.join(root, 'src', 'webview.ts'), 'utf8'), - ].join('\n'); +test('optimizer preview does not create a second persisted authorization path', () => { + const source = readFileSync(path.join(root, 'src', 'extension.ts'), 'utf8'); + const start = source.indexOf('private async prepareOptimizerRequest'); + const end = source.indexOf('private loadConfiguration', start); + assert.ok(start >= 0 && end > start); + const preparation = source.slice(start, end); + assert.doesNotMatch(preparation, /optimizerConsented/); + assert.doesNotMatch(preparation, /showWarningMessage/); + assert.doesNotMatch(preparation, /globalState\.update/); +}); + +test('packaged advisor exposes no prepare-and-send shortcut around exact preview', () => { + const source = readFileSync(path.join(root, 'src', 'advisor.ts'), 'utf8'); + assert.doesNotMatch(source, /export\s+async\s+function\s+callModel\b/); + assert.doesNotMatch(source, /export\s+async\s+function\s+getUsageAdvice\b/); + assert.doesNotMatch(source, /getSubscriptionToken|subscriptionModel/); + assert.match(source, /export\s+async\s+function\s+sendPreparedModelRequest\b/); +}); + +test('only the reviewed comparison writer reaches the bounded local ledger', () => { + const extension = readFileSync(path.join(root, 'src', 'extension.ts'), 'utf8'); + const webview = readFileSync(path.join(root, 'src', 'webview.ts'), 'utf8'); + const production = [extension, webview].join('\n'); for (const seam of [ 'planAdviceEvidencePreparation', 'buildLegacyPersonalizationDraft', 'projectLegacyPersonalization', 'requestStructuredAdviceViaLegacyByok', - 'appendStoredComparablePair', ]) { assert.doesNotMatch(production, new RegExp(`\\b${seam}\\b`)); } + + assert.doesNotMatch(extension, /\bappendStoredComparablePair\b/); + const materializerStart = webview.indexOf('private materializeAdviceComparisonState'); + const materializerEnd = webview.indexOf( + 'private currentAdviceComparisonProductionRevision', + materializerStart, + ); + assert.ok(materializerStart >= 0 && materializerEnd > materializerStart); + const materializer = webview.slice(materializerStart, materializerEnd); + assert.match(materializer, /\bbuildAppliedComparablePairs\b/); + assert.match(materializer, /\bappendStoredComparablePair\b/); + assert.match(materializer, /\bbuildAdviceComparisonResultEnvelope\b/); + assert.match(materializer, /\bappendAdviceComparisonResult\b/); +}); + +test('Claude advice consumes the materialized index snapshot instead of rebuilding from records', () => { + const source = readFileSync(path.join(root, 'src', 'extension.ts'), 'utf8'); + const start = source.indexOf('private buildAdviceEffectivenessProviderStates'); + const end = source.indexOf('private codexHistoricalWorkPending', start); + assert.ok(start >= 0 && end > start); + const builder = source.slice(start, end); + assert.match(builder, /claudeUsageDashboardSnapshot\s*\(/); + assert.match(builder, /adviceWindowDays:\s*windowDays/); + assert.doesNotMatch(builder, /this\.cache\.records/); + assert.doesNotMatch(builder, /ClaudeDataLoader\.(?:getAllTimeData|getSessionBreakdown)/); }); diff --git a/src/test/adviceStateSerialization.test.ts b/src/test/adviceStateSerialization.test.ts new file mode 100644 index 00000000..ff95b854 --- /dev/null +++ b/src/test/adviceStateSerialization.test.ts @@ -0,0 +1,310 @@ +import { test } from 'node:test'; +import * as assert from 'node:assert/strict'; +import Module = require('node:module'); + +import { + ADVICE_LOCAL_STATE_KEY, + AdviceLocalState, + AdviceLocalStateStorage, + createClosedAdviceLocalState, +} from '../adviceEffectiveness/versionedPersistence'; + +class ControlledStorage implements AdviceLocalStateStorage { + public value: AdviceLocalState; + public readonly pending: Array<{ + value: AdviceLocalState; + resolve: () => void; + }> = []; + + constructor(value: AdviceLocalState) { + this.value = value; + } + + get(key: string): T | undefined { + return key === ADVICE_LOCAL_STATE_KEY ? this.value as T : undefined; + } + + update(key: string, value: unknown): Promise { + assert.equal(key, ADVICE_LOCAL_STATE_KEY); + return new Promise((resolve) => { + this.pending.push({ + value: value as AdviceLocalState, + resolve: () => { + this.value = value as AdviceLocalState; + resolve(); + }, + }); + }); + } + + releaseNext(): void { + const next = this.pending.shift(); + assert.ok(next, 'expected a pending state write'); + next.resolve(); + } +} + +async function nextTurn(): Promise { + await new Promise((resolve) => setImmediate(resolve)); +} + +async function createProvider(storage: AdviceLocalStateStorage): Promise { + const originalLoad = (Module as any)._load; + (Module as any)._load = function(request: string, parent: unknown, isMain: boolean) { + if (request === 'vscode') { + return { workspace: { workspaceFolders: [] } }; + } + return originalLoad.call(this, request, parent, isMain); + }; + let UsageWebviewProvider: typeof import('../webview').UsageWebviewProvider; + try { + ({ UsageWebviewProvider } = require('../webview') as typeof import('../webview')); + } finally { + (Module as any)._load = originalLoad; + } + const provider = new UsageWebviewProvider({ globalState: storage } as any) as any; + await nextTurn(); + provider.settings = { get: (key: string) => key === 'advice.effectiveness.enabled' }; + provider.adviceLocalStateStatus = 'ready'; + provider.postAdviceMessage = () => undefined; + return provider; +} + +test('clear is serialized after an older consent write and remains the final durable state', async () => { + const initial = createClosedAdviceLocalState(); + initial.featureMode = 'enabled'; + const storage = new ControlledStorage(initial); + const provider = await createProvider(storage); + provider.adviceLocalState = initial; + provider.adviceEffectivenessStates = { + claude: { + provider: 'claude', + remotePreviewEligible: true, + aggregate: { windowDays: 30 }, + promptSamples: [], + contract: { adviceId: 'advice-claude-test', recommendations: [] }, + }, + }; + provider.onAdviceDataCleared = async () => undefined; + + const consent = provider.handleAdviceConsentMessage({ + provider: 'claude', + aggregateConsent: 'explicit', + promptSampleConsent: 'not-granted', + }); + await nextTurn(); + assert.equal(storage.pending.length, 1); + + const clear = provider.handleClearAdviceLocalDataMessage(); + await nextTurn(); + assert.equal( + storage.pending.length, + 1, + 'clear must share the one state-writer queue instead of racing the consent write', + ); + + storage.releaseNext(); + await nextTurn(); + assert.equal(storage.pending.length, 1, 'clear should begin after consent settles'); + storage.releaseNext(); + await Promise.all([consent, clear]); + + assert.equal(storage.value.featureMode, 'enabled'); + assert.equal(storage.value.aggregateConsent, 'not-granted'); + assert.equal(storage.value.promptSampleConsent, 'not-granted'); + assert.deepEqual(storage.value.feedback, []); + assert.deepEqual(storage.value.comparablePairs, []); + assert.deepEqual(storage.value.comparisonResults, []); +}); + +test('discarded optimizer request cannot install a stale result after await', async () => { + const provider = await createProvider({ + get: () => undefined, + update: async () => undefined, + }); + provider.settings = { + get: (key: string) => key === 'advice.optimizer.enabled', + }; + const snapshotId = 'optimizer-stale-test'; + provider.optimizerConsentGeneration = 7; + provider.optimizerState = { + draft: 'old draft', + resolve: true, + distil: false, + aesthetic: false, + }; + const stored = { + prepared: {}, + draft: 'old draft', + sourceRevision: 'optimizer-source', + consentGeneration: 7, + }; + provider.preparedOptimizerRequests.set(snapshotId, stored); + const messages: Array> = []; + provider.postAdviceMessage = (message: Record) => messages.push(message); + let finish!: (value: unknown) => void; + provider.onSendOptimizerInvocation = () => new Promise((resolve) => { + finish = resolve; + }); + + const send = provider.handleSendOptimizerMessage({ snapshotId, draft: 'old draft' }); + await nextTurn(); + provider.discardPreparedOptimizer(); + finish({ ok: true, value: { prompt: 'STALE RESULT', settings: 'stale' } }); + await send; + + assert.equal(provider.optimizerState.prompt, undefined); + assert.equal(messages.some((message) => message.prompt === 'STALE RESULT'), false); + const lastMessage = messages[messages.length - 1]; + assert.equal(lastMessage?.command, 'optimizeResult'); + assert.equal(typeof lastMessage?.error, 'string'); +}); + +test('optimizer feedback uses the same retractable local ledger and rendered controls', async () => { + let durable = createClosedAdviceLocalState(); + const provider = await createProvider({ + get: () => durable as T, + update: async (_key: string, value: unknown) => { + durable = value as AdviceLocalState; + }, + }); + provider.settings = { + get: (key: string) => + key === 'advice.effectiveness.enabled' || key === 'advice.optimizer.enabled', + }; + provider.adviceLocalState = durable; + provider.optimizerState = { + draft: 'host-only draft', + resolve: false, + distil: false, + aesthetic: false, + prompt: 'Paste-ready result', + settings: 'Effort: high', + adviceId: 'advice-optimizer-0123456789abcdef01234567', + }; + + await provider.handleAdviceFeedbackMessage({ + provider: 'optimizer', + adviceId: provider.optimizerState.adviceId, + recommendationId: 'recommendation-optimizer-result-v1', + kind: 'helpful', + }); + + assert.equal(durable.feedback.length, 1); + assert.equal(durable.feedback[0].rating, 'helpful'); + assert.equal(durable.feedback[0].applied, 'not-applied'); + assert.doesNotMatch(JSON.stringify(durable), /host-only draft|Paste-ready result|Effort: high/); + const html = provider.renderOptimizerCard(); + assert.equal((html.match(/data-advice-action="feedback"/g) ?? []).length, 3); + assert.match(html, /data-provider="optimizer"/); + assert.match(html, /recommendation-optimizer-result-v1/); +}); + +test('a failed feedback save returns the validated target identity so only its controls unlock', async () => { + const durable = createClosedAdviceLocalState(); + const provider = await createProvider({ + get: () => durable as T, + update: async () => { + throw new Error('simulated storage failure'); + }, + }); + provider.settings = { + get: (key: string) => key === 'advice.optimizer.enabled', + }; + provider.adviceLocalState = durable; + provider.optimizerState = { + draft: 'host-only draft', + resolve: false, + distil: false, + aesthetic: false, + prompt: 'Paste-ready result', + settings: 'Effort: high', + adviceId: 'advice-optimizer-0123456789abcdef01234567', + }; + const messages: Array> = []; + provider.postAdviceMessage = (message: Record) => messages.push(message); + + await provider.handleAdviceFeedbackMessage({ + provider: 'optimizer', + adviceId: 'advice-optimizer-0123456789abcdef01234567', + recommendationId: 'recommendation-optimizer-result-v1', + kind: 'helpful', + }); + + assert.deepEqual(messages[messages.length - 1], { + command: 'adviceFeedbackResult', + ok: false, + provider: 'optimizer', + adviceId: 'advice-optimizer-0123456789abcdef01234567', + recommendationId: 'recommendation-optimizer-result-v1', + }); +}); + +test('optimizer host seam fails closed while its feature is disabled', async () => { + const provider = await createProvider({ + get: () => undefined, + update: async () => undefined, + }); + provider.settings = { get: () => false }; + let prepares = 0; + let sends = 0; + provider.onPrepareOptimizerInvocation = async () => { + prepares += 1; + return { error: 'must not run' }; + }; + provider.onSendOptimizerInvocation = async () => { + sends += 1; + return { ok: false, code: 'transport-error', issues: [] }; + }; + + await provider.handlePrepareOptimizerMessage({ draft: 'private draft' }); + provider.preparedOptimizerRequests.set('optimizer-disabled', { + prepared: {}, + draft: 'private draft', + sourceRevision: 'disabled', + consentGeneration: provider.optimizerConsentGeneration, + }); + await provider.handleSendOptimizerMessage({ + snapshotId: 'optimizer-disabled', + draft: 'private draft', + }); + + assert.equal(prepares, 0); + assert.equal(sends, 0); +}); + +test('successful optimizer send creates an opaque feedback target without deriving it from content', async () => { + const provider = await createProvider({ + get: () => undefined, + update: async () => undefined, + }); + provider.settings = { get: (key: string) => key === 'advice.optimizer.enabled' }; + provider.optimizerConsentGeneration = 3; + provider.optimizerState = { + draft: 'SENSITIVE_DRAFT', + resolve: false, + distil: false, + aesthetic: false, + }; + const snapshotId = 'optimizer-success'; + provider.preparedOptimizerRequests.set(snapshotId, { + prepared: {}, + draft: 'SENSITIVE_DRAFT', + sourceRevision: 'optimizer-source', + consentGeneration: 3, + }); + provider.onSendOptimizerInvocation = async () => ({ + ok: true, + value: { prompt: 'SENSITIVE_RESULT', settings: 'SENSITIVE_SETTINGS' }, + }); + const messages: Array> = []; + provider.postAdviceMessage = (message: Record) => messages.push(message); + + await provider.handleSendOptimizerMessage({ snapshotId, draft: 'SENSITIVE_DRAFT' }); + + assert.match(provider.optimizerState.adviceId, /^advice-optimizer-[a-f0-9]{24}$/); + assert.doesNotMatch(provider.optimizerState.adviceId, /SENSITIVE/i); + const lastMessage = messages[messages.length - 1]; + assert.equal(lastMessage.recommendationId, 'recommendation-optimizer-result-v1'); + assert.equal(lastMessage.adviceId, provider.optimizerState.adviceId); +}); diff --git a/src/test/backgroundWorkState.test.ts b/src/test/backgroundWorkState.test.ts new file mode 100644 index 00000000..9cc352ec --- /dev/null +++ b/src/test/backgroundWorkState.test.ts @@ -0,0 +1,255 @@ +import { test } from 'node:test'; +import * as assert from 'node:assert/strict'; +import { + BACKGROUND_WORK_STATE_SCHEMA_VERSION, + beginBackgroundWork, + canStartBackgroundWork, + createBackgroundWorkState, + interruptBackgroundWork, + pauseBackgroundWork, + recordBackgroundWorkFailure, + recordBackgroundWorkProgress, + restoreBackgroundWorkState, + type BackgroundWorkBackoffPolicy, + type BackgroundWorkProgress, +} from '../backgroundWorkState'; + +const policy: BackgroundWorkBackoffPolicy = { + failureBaseMs: 100, + failureMaxMs: 400, + noProgressCooldownMs: 250, +}; + +const emptyProgress: BackgroundWorkProgress = { + completedUnits: 0, + totalUnits: 5, + completedBytes: 0, + totalBytes: 50, +}; + +test('successful progress resets failures and is immediately eligible', () => { + const initial = createBackgroundWorkState({ + measurementVersion: 7, + reason: 'first-index', + now: 1_000, + progress: emptyProgress, + }); + const firstStart = beginBackgroundWork(initial, { trigger: 'automatic', now: 1_000 }); + assert.equal(firstStart.started, true); + + const failed = recordBackgroundWorkFailure(firstStart.state, { now: 1_010 }, policy); + assert.equal(failed.failStreak, 1); + const retry = beginBackgroundWork(failed, { trigger: 'manual', now: 1_011, reason: 'resume' }); + assert.equal(retry.started, true); + + const progressed = recordBackgroundWorkProgress(retry.state, { + now: 1_020, + complete: false, + progress: { + completedUnits: 1, + totalUnits: 5, + completedBytes: 10, + totalBytes: 50, + }, + }, policy); + + assert.equal(progressed.status, 'eligible'); + assert.equal(progressed.failStreak, 0); + assert.equal(progressed.nextEligibleAt, 1_020); + assert.equal(progressed.pausedReason, null); + assert.equal(progressed.reason, 'resume'); + assert.equal(canStartBackgroundWork(progressed, { trigger: 'automatic', now: 1_020 }), true); +}); + +test('failures persist an exponential backoff and manual work may bypass it', () => { + let state = createBackgroundWorkState({ + measurementVersion: 1, + reason: 'first-index', + now: 1_000, + progress: emptyProgress, + }); + + state = beginBackgroundWork(state, { trigger: 'automatic', now: 1_000 }).state; + state = recordBackgroundWorkFailure(state, { now: 1_000 }, policy); + assert.equal(state.failStreak, 1); + assert.equal(state.nextEligibleAt, 1_100); + assert.equal(state.pausedReason, 'failure-backoff'); + assert.equal(canStartBackgroundWork(state, { trigger: 'automatic', now: 1_099 }), false); + assert.equal(canStartBackgroundWork(state, { trigger: 'automatic', now: 1_100 }), true); + assert.equal(canStartBackgroundWork(state, { trigger: 'manual', now: 1_001 }), true); + + state = beginBackgroundWork(state, { trigger: 'automatic', now: 1_100 }).state; + state = recordBackgroundWorkFailure(state, { now: 1_100 }, policy); + assert.equal(state.failStreak, 2); + assert.equal(state.nextEligibleAt, 1_300); + + state = beginBackgroundWork(state, { trigger: 'automatic', now: 1_300 }).state; + state = recordBackgroundWorkFailure(state, { now: 1_300 }, policy); + state = beginBackgroundWork(state, { trigger: 'automatic', now: 1_700 }).state; + state = recordBackgroundWorkFailure(state, { now: 1_700 }, policy); + assert.equal(state.failStreak, 4); + assert.equal(state.nextEligibleAt, 2_100, 'backoff is capped by failureMaxMs'); +}); + +test('an attempt with no progress cools down instead of hot-looping', () => { + const initial = createBackgroundWorkState({ + measurementVersion: 1, + reason: 'hourly-history', + now: 10, + progress: emptyProgress, + }); + const running = beginBackgroundWork(initial, { trigger: 'automatic', now: 10 }).state; + const noProgress = recordBackgroundWorkProgress(running, { + now: 20, + complete: false, + progress: emptyProgress, + }, policy); + + assert.equal(noProgress.status, 'cooldown'); + assert.equal(noProgress.pausedReason, 'no-progress'); + assert.equal(noProgress.nextEligibleAt, 270); + assert.equal(noProgress.failStreak, 0, 'no-progress is distinct from an execution failure'); + assert.equal(canStartBackgroundWork(noProgress, { trigger: 'automatic', now: 269 }), false); + assert.equal(canStartBackgroundWork(noProgress, { trigger: 'manual', now: 21 }), true); +}); + +test('a user pause can only be resumed by a manual trigger', () => { + const initial = createBackgroundWorkState({ + measurementVersion: 1, + reason: 'period-migration', + now: 1, + }); + const paused = pauseBackgroundWork(initial, { now: 2 }); + + assert.equal(paused.status, 'paused'); + assert.equal(paused.pausedReason, 'user'); + assert.equal(canStartBackgroundWork(paused, { trigger: 'automatic', now: 10_000 }), false); + + const automatic = beginBackgroundWork(paused, { trigger: 'automatic', now: 10_000 }); + assert.equal(automatic.started, false); + assert.equal(automatic.state, paused); + + const manual = beginBackgroundWork(paused, { trigger: 'manual', now: 3, reason: 'resume' }); + assert.equal(manual.started, true); + assert.equal(manual.state.status, 'running'); + assert.equal(manual.state.pausedReason, null); + assert.equal(manual.state.reason, 'resume'); +}); + +test('an interrupted running attempt resumes without counting as a failure', () => { + const initial = createBackgroundWorkState({ + measurementVersion: 1, + reason: 'first-index', + now: 10, + progress: emptyProgress, + }); + const running = beginBackgroundWork(initial, { + trigger: 'automatic', + now: 10, + }).state; + const interrupted = interruptBackgroundWork(running, { now: 20 }); + + assert.equal(interrupted.status, 'eligible'); + assert.equal(interrupted.reason, 'resume'); + assert.equal(interrupted.failStreak, 0); + assert.equal(interrupted.nextEligibleAt, 20); + assert.equal(interrupted.pausedReason, null); + assert.equal(canStartBackgroundWork(interrupted, { + trigger: 'automatic', + now: 20, + }), true); +}); + +test('complete work stays complete for the same measurement version', () => { + const initial = createBackgroundWorkState({ + measurementVersion: 4, + reason: 'first-index', + now: 100, + progress: emptyProgress, + }); + const running = beginBackgroundWork(initial, { trigger: 'automatic', now: 100 }).state; + const complete = recordBackgroundWorkProgress(running, { + now: 110, + complete: true, + progress: { + completedUnits: 5, + totalUnits: 5, + completedBytes: 50, + totalBytes: 50, + }, + }, policy); + + assert.equal(complete.status, 'complete'); + assert.equal(complete.nextEligibleAt, null); + assert.equal(canStartBackgroundWork(complete, { trigger: 'manual', now: 1_000 }), false); + + const sameVersion = restoreBackgroundWorkState(JSON.parse(JSON.stringify(complete)), { + measurementVersion: 4, + reason: 'first-index', + now: 1_000, + }); + assert.equal(sameVersion.disposition, 'valid'); + assert.equal(sameVersion.state.status, 'complete'); + assert.equal(canStartBackgroundWork(sameVersion.state, { trigger: 'automatic', now: 1_000 }), false); + + const changedVersion = restoreBackgroundWorkState(complete, { + measurementVersion: 5, + reason: 'parser-migration', + now: 1_000, + }); + assert.equal(changedVersion.disposition, 'measurement-changed'); + assert.equal(changedVersion.state.status, 'eligible'); + assert.equal(changedVersion.state.measurementVersion, 5); + assert.deepEqual(changedVersion.state.progress, { + completedUnits: 0, + totalUnits: 0, + completedBytes: 0, + totalBytes: 0, + }); +}); + +test('missing state starts safely while corrupt or content-bearing state fails closed', () => { + const missing = restoreBackgroundWorkState(undefined, { + measurementVersion: 3, + reason: 'hourly-history', + now: 50, + }); + assert.equal(missing.disposition, 'new'); + assert.equal(missing.state.status, 'eligible'); + + const unsafe = { + schemaVersion: BACKGROUND_WORK_STATE_SCHEMA_VERSION, + measurementVersion: 3, + status: 'eligible', + reason: 'hourly-history', + progress: { + completedUnits: 0, + totalUnits: 0, + completedBytes: 0, + totalBytes: 0, + }, + failStreak: 0, + nextEligibleAt: 50, + pausedReason: null, + updatedAt: 50, + path: '/Users/alice/.codex/sessions/private.jsonl', + }; + const corrupt = restoreBackgroundWorkState(unsafe, { + measurementVersion: 3, + reason: 'hourly-history', + now: 60, + }); + + assert.equal(corrupt.disposition, 'corrupt'); + assert.equal(corrupt.state.status, 'paused'); + assert.equal(corrupt.state.pausedReason, 'corrupt-state'); + assert.equal(canStartBackgroundWork(corrupt.state, { trigger: 'manual', now: 61 }), false); + assert.doesNotMatch(JSON.stringify(corrupt.state), /alice|private|path|session/i); + + const invalidNumber = restoreBackgroundWorkState({ ...unsafe, path: undefined, failStreak: -1 }, { + measurementVersion: 3, + reason: 'hourly-history', + now: 60, + }); + assert.equal(invalidNumber.disposition, 'corrupt'); +}); diff --git a/src/test/claudeApiClient.test.ts b/src/test/claudeApiClient.test.ts index 139d342d..464a2f8d 100644 --- a/src/test/claudeApiClient.test.ts +++ b/src/test/claudeApiClient.test.ts @@ -101,3 +101,33 @@ test('refreshed credentials are written back to the selected profile', async () const stored = JSON.parse(fs.readFileSync(path.join(profile, '.credentials.json'), 'utf8')); assert.deepEqual(stored, credentials); }); + +test('quota fetch forwards caller cancellation through the active request', async () => { + const client = new ClaudeApiClient(null) as any; + const controller = new AbortController(); + let observedSignal: AbortSignal | undefined; + client.getValidCredentials = async () => ({ + claudeAiOauth: { + accessToken: 'test-access', + refreshToken: 'test-refresh', + expiresAt: Date.now() + 60_000, + }, + }); + client.callUsageApi = (_token: string, signal?: AbortSignal) => + new Promise((_resolve, reject) => { + observedSignal = signal; + signal?.addEventListener( + 'abort', + () => reject(new Error('Request cancelled')), + { once: true }, + ); + }); + + const pending = client.fetchUsageLimits(controller.signal); + await Promise.resolve(); + controller.abort(); + + assert.equal(await pending, null); + assert.strictEqual(observedSignal, controller.signal); + assert.equal(observedSignal?.aborted, true); +}); diff --git a/src/test/claudeIncrementalIndex.test.ts b/src/test/claudeIncrementalIndex.test.ts index 63e4e573..42d6bc34 100644 --- a/src/test/claudeIncrementalIndex.test.ts +++ b/src/test/claudeIncrementalIndex.test.ts @@ -18,6 +18,7 @@ import { updateClaudeUsageIndex, } from '../claudeIncrementalIndex'; import { ClaudeDataLoader } from '../dataLoader'; +import { I18n } from '../i18n'; import { ClaudeUsageRecord } from '../types'; const roots: string[] = []; @@ -250,6 +251,76 @@ test('materialized dashboard rows match every legacy full-record aggregation', a ); }); +test('advice window is opt-in and comes only from materialized day and session aggregates', async () => { + const { root, first } = await fixture(); + await writeFile( + path.join(path.dirname(first), 'session-old.jsonl'), + `${usageLine('old-advice', 999, 99, { timestamp: '2026-06-01T08:00:00.000Z' })}\n`, + 'utf8', + ); + const loaded = await updateClaudeUsageIndex(createClaudeUsageIndex(), root, { + analyzeContent: false, + }); + const ordinary = claudeUsageDashboardSnapshot(loaded.index, { + now: new Date('2026-08-22T12:00:00.000Z'), + }); + assert.equal(ordinary.adviceWindow, undefined); + + const candidate = claudeUsageDashboardSnapshot(loaded.index, { + now: new Date('2026-08-22T12:00:00.000Z'), + adviceWindowDays: 30, + }); + assert.equal(candidate.adviceWindow?.aggregate.messageCount, 1); + assert.equal(candidate.adviceWindow?.aggregate.totalInputTokens, 30); + assert.equal(candidate.adviceWindow?.totalSessions, 2); + assert.equal(candidate.adviceWindow?.longSessionCount, 0); + assert.equal(candidate.adviceWindow?.largeContextSessionCount, 0); + assert.equal(loaded.diagnostics.bodyReads, 3, 'snapshot materialization performs no extra reads'); +}); + +test('configured timezone rebuckets Claude Today and hours from the in-memory index without body reads', async () => { + const previousTimeZone = I18n.getTimezone(); + const root = await mkdtemp(path.join(os.tmpdir(), 'ccu-claude-timezone-index-')); + roots.push(root); + const project = path.join(root, 'projects', '-fixture-timezone'); + await mkdir(project, { recursive: true }); + const file = path.join(project, 'timezone.jsonl'); + await writeFile(file, `${usageLine('timezone', 10, 2, { + timestamp: '2026-07-20T23:30:00.000Z', + })}\n`, 'utf8'); + + try { + I18n.setTimezone('UTC'); + const cold = await updateClaudeUsageIndex(createClaudeUsageIndex(), root, { + analyzeContent: false, + }); + const utc = claudeUsageDashboardSnapshot(cold.index, { + now: new Date('2026-07-21T00:15:00.000Z'), + }); + assert.equal(utc.today.totalInputTokens, 0); + assert.deepEqual(utc.hourlyForToday, []); + + I18n.setTimezone('Asia/Hong_Kong'); + const shifted = await updateClaudeUsageIndex(cold.index, root, { + analyzeContent: false, + }); + const hongKong = claudeUsageDashboardSnapshot(shifted.index, { + now: new Date('2026-07-21T00:15:00.000Z'), + }); + + assert.equal(shifted.diagnostics.bodyReads, 0); + assert.equal(hongKong.today.totalInputTokens, 10); + assert.deepEqual(hongKong.hourlyForToday.map(({ hour }) => hour), ['07:00']); + assert.deepEqual( + ClaudeDataLoader.getHourlyDataForDate(shifted.records, '2026-07-21') + .map(({ hour }) => hour), + ['07:00'], + ); + } finally { + I18n.setTimezone(previousTimeZone); + } +}); + test('append reads only the verified tail and preserves exact totals', async () => { const { root, first } = await fixture(); const cold = await updateClaudeUsageIndex(createClaudeUsageIndex(), root, { diff --git a/src/test/codexIndex.test.ts b/src/test/codexIndex.test.ts index e6596180..1531ec23 100644 --- a/src/test/codexIndex.test.ts +++ b/src/test/codexIndex.test.ts @@ -31,7 +31,10 @@ import { CodexRuntimeManifestEntry, scanCodexManifest, } from '../providers/codex/codexManifest'; -import { CodexFilePassPool } from '../providers/codex/codexFilePassPool'; +import { + CodexFilePassPool, + CodexFilePassWorkerLike, +} from '../providers/codex/codexFilePassPool'; import { pseudonymousIdentityKey } from '../providers/codex/codexIdentity'; import { parseCodexLine } from '../providers/codex/codexParser'; @@ -877,20 +880,154 @@ test('cold and append scans maintain a sparse exact current-day hourly sidecar', total: { inputTotal: 999, rawResponse: 'today-secret-hour' }, byModel: {}, }; + caller.files[key].aggregate.today.days['2026-07-21']['02'].total.promptBody = + 'rolling-secret-total'; + caller.files[key].aggregate.today.days.notADay = { + '02': { + total: { inputTotal: 999, rawResponse: 'rolling-secret-day' }, + byModel: {}, + }, + }; await saveCodexIndexAtomic(indexPath, caller); const persisted = await readFile(indexPath, 'utf8'); - assert.doesNotMatch(persisted, /today-secret/); + assert.doesNotMatch(persisted, /today-secret|rolling-secret/); const reloaded = await loadCodexIndex(indexPath, 'Asia/Hong_Kong'); assert.equal(reloaded.files[key].aggregate.today?.day, '2026-07-21'); + assert.equal(reloaded.files[key].aggregate.today?.windowDays, 30); assert.deepEqual(Object.keys(reloaded.files[key].aggregate.today!.hours), ['00', '02']); + assert.deepEqual( + Object.keys(reloaded.files[key].aggregate.today?.days ?? {}), + ['2026-07-21'], + ); assert.equal(reloaded.files[key].aggregate.today?.hours['02'].total.inputTotal, 50); assert.equal((reloaded.files[key].aggregate.today?.hours['02'] as any).byEffort, undefined); + const reloadIo = trackingIo(); + const warmReload = await updateCodexIndex(reloaded, await scanCodexManifest(root, SALT), { + salt: SALT, + timeZone: 'Asia/Hong_Kong', + now: () => now, + io: reloadIo, + }); + assert.equal(warmReload.bodyReads, 0); + assert.equal(reloadIo.bodyReads.size, 0); + } finally { + await rm(root, { recursive: true, force: true }); + } +}); + +test('rolling hourly sidecar keeps 30 configured-zone days and evicts day 31 without rereading JSONL', async () => { + const root = await mkdtemp(path.join(os.tmpdir(), 'ccu-codex-index-rolling-hours-')); + try { + const sessions = path.join(root, 'sessions'); + await mkdir(sessions, { recursive: true }); + await writeFile( + path.join(sessions, 'rolling-hours.jsonl'), + [ + tokenLine(100, 20, '2026-06-30T08:05:00.000Z'), + tokenLine(200, 40, '2026-07-01T09:05:00.000Z'), + tokenLine(300, 60, '2026-07-15T10:05:00.000Z'), + tokenLine(400, 80, '2026-07-30T11:05:00.000Z'), + '', + ].join('\n'), + 'utf8', + ); + const manifest = await scanCodexManifest(root, SALT); + const first = await updateCodexIndex(createEmptyCodexIndex('UTC'), manifest, { + salt: SALT, + timeZone: 'UTC', + now: () => Date.parse('2026-07-30T12:00:00.000Z'), + }); + const key = manifest.files[0].fileKey; + const sidecar = first.index.files[key].aggregate.today!; + + assert.equal(sidecar.windowDays, 30); + assert.deepEqual(Object.keys(sidecar.days ?? {}).sort(), [ + '2026-07-01', + '2026-07-15', + '2026-07-30', + ]); + assert.equal(sidecar.days?.['2026-07-01']['09'].total.inputTotal, 100); + assert.equal(sidecar.days?.['2026-07-30']['11'].total.outputTotal, 20); + assert.equal(sidecar.days?.['2026-06-30'], undefined); + assert.equal(first.index.coverage.hourly?.complete, true); + + const io = trackingIo(); + const rolled = await updateCodexIndex(first.index, manifest, { + salt: SALT, + timeZone: 'UTC', + now: () => Date.parse('2026-07-31T12:00:00.000Z'), + io, + }); + const rolledSidecar = rolled.index.files[key].aggregate.today!; + + assert.equal(rolled.bodyReads, 0); + assert.equal(io.bodyReads.size, 0); + assert.equal(rolledSidecar.day, '2026-07-31'); + assert.deepEqual(Object.keys(rolledSidecar.days ?? {}).sort(), [ + '2026-07-15', + '2026-07-30', + ]); + assert.deepEqual(rolledSidecar.hours, {}); + assert.equal(rolled.index.coverage.hourly?.asOfDay, '2026-07-31'); + assert.equal(rolled.index.coverage.hourly?.complete, true); + } finally { + await rm(root, { recursive: true, force: true }); + } +}); + +test('legacy one-day schema-3 sidecars receive one targeted rolling migration', async () => { + const root = await mkdtemp(path.join(os.tmpdir(), 'ccu-codex-index-rolling-legacy-')); + try { + const now = Date.parse('2026-07-20T12:00:00.000Z'); + const sessions = path.join(root, 'sessions'); + await mkdir(sessions, { recursive: true }); + await writeFile( + path.join(sessions, 'legacy-hours.jsonl'), + [ + tokenLine(100, 20, '2026-07-05T08:00:00.000Z'), + tokenLine(200, 40, '2026-07-20T09:00:00.000Z'), + '', + ].join('\n'), + 'utf8', + ); + const manifest = await scanCodexManifest(root, SALT); + const current = await updateCodexIndex(createEmptyCodexIndex('UTC'), manifest, { + salt: SALT, + timeZone: 'UTC', + now: () => now, + }); + const key = manifest.files[0].fileKey; + const legacy = structuredClone(current.index) as any; + const modern = legacy.files[key].aggregate.today; + legacy.files[key].aggregate.today = { + day: modern.day, + timeZone: modern.timeZone, + indexedThrough: modern.indexedThrough, + hours: modern.hours, + }; + delete legacy.coverage.hourly; + const io = trackingIo(); + + const migrated = await updateCodexIndex(legacy, manifest, { + salt: SALT, + timeZone: 'UTC', + now: () => now, + io, + }); + + assert.deepEqual(io.fileKeysRead, [key]); + assert.equal(migrated.index.files[key].aggregate.today?.windowDays, 30); + assert.deepEqual( + Object.keys(migrated.index.files[key].aggregate.today?.days ?? {}).sort(), + ['2026-07-05', '2026-07-20'], + ); + assert.equal(migrated.index.coverage.hourly?.complete, true); } finally { await rm(root, { recursive: true, force: true }); } }); -test('an old schema-3 index backfills today only from canonical files known to contain that day', async () => { +test('an old schema-3 index targets canonical files that intersect the rolling hour window', async () => { const root = await mkdtemp(path.join(os.tmpdir(), 'ccu-codex-index-today-targeted-')); try { const today = Date.parse('2026-07-21T04:00:00.000Z'); @@ -910,6 +1047,11 @@ test('an old schema-3 index backfills today only from canonical files known to c `${tokenLine(300, 60, '2026-07-19T18:05:00.000Z')}\n`, 'utf8', ); + await writeFile( + path.join(sessions, 'outside-window.jsonl'), + `${tokenLine(400, 80, '2026-06-01T18:05:00.000Z')}\n`, + 'utf8', + ); const manifest = await scanCodexManifest(root, SALT); const cold = await updateCodexIndex(createEmptyCodexIndex('Asia/Hong_Kong'), manifest, { salt: SALT, @@ -932,17 +1074,29 @@ test('an old schema-3 index backfills today only from canonical files known to c const todayKeys = manifest.files.filter((entry) => cold.index.files[entry.fileKey].aggregate.period?.days['2026-07-21'] ).map((entry) => entry.fileKey); - const todayKey = io.fileKeysRead[0]; - const oldKey = manifest.files.find((entry) => - !cold.index.files[entry.fileKey].aggregate.period?.days['2026-07-21'] + const todayKey = todayKeys.find((key) => io.bodyReads.has(key))!; + const recentKey = manifest.files.find((entry) => + cold.index.files[entry.fileKey].aggregate.period?.days['2026-07-20'] + )!.fileKey; + const outsideKey = manifest.files.find((entry) => + cold.index.files[entry.fileKey].aggregate.period?.days['2026-06-02'] )!.fileKey; assert.equal(rebuilt.index.schemaVersion, 3); - assert.deepEqual(io.fileKeysRead, [todayKey]); + assert.deepEqual(new Set(io.fileKeysRead), new Set([todayKey, recentKey])); assert.equal(todayKeys.length, 2); assert.equal(todayKeys.filter((key) => io.bodyReads.has(key)).length, 1); - assert.equal(io.bodyReads.has(oldKey), false); + assert.equal(io.bodyReads.has(recentKey), true); + assert.equal(io.bodyReads.has(outsideKey), false); assert.equal(rebuilt.index.files[todayKey].aggregate.today?.hours['02'].total.inputTotal, 100); + assert.equal( + rebuilt.index.files[recentKey].aggregate.today?.days?.['2026-07-20']['02'] + .total.inputTotal, + 300, + ); + assert.equal(rebuilt.index.coverage.hourly?.indexedFiles, 2); + assert.equal(rebuilt.index.coverage.hourly?.totalFiles, 2); + assert.equal(rebuilt.index.coverage.hourly?.complete, true); assert.deepEqual(rebuilt.index.coverage.today, { timeZone: 'Asia/Hong_Kong', day: '2026-07-21', @@ -1184,6 +1338,16 @@ test('timezone changes rebuild the current civil day and then stay warm without assert.equal(hongKong.index.files[key].aggregate.today?.timeZone, 'Asia/Hong_Kong'); assert.equal(hongKong.index.files[key].aggregate.today?.hours['00'].total.inputTotal, 50); assert.equal(hongKong.index.files[key].aggregate.today?.hours['15'], undefined); + assert.equal( + hongKong.index.files[key].aggregate.today?.days?.['2026-07-20']['23'] + .total.inputTotal, + 100, + ); + assert.equal( + hongKong.index.files[key].aggregate.today?.days?.['2026-07-21']['00'] + .total.inputTotal, + 50, + ); assert.equal(hongKong.index.coverage.today.complete, true); const warmIo = trackingIo(); @@ -1363,6 +1527,57 @@ test('parallel file passes preserve the exact sequential index across every back } }); +test('file-pass pool disposal waits for workers removed by a failed batch to terminate', async () => { + let finishTerminate!: (code: number) => void; + const termination = new Promise((resolve) => { + finishTerminate = resolve; + }); + let terminateCalls = 0; + const worker = { + postMessage: () => undefined, + on: () => worker, + terminate: () => { + terminateCalls += 1; + return termination; + }, + } as CodexFilePassWorkerLike; + const pool = new CodexFilePassPool(1, () => worker); + const running = pool.run( + [{ taskId: 'aggregate-only-test' } as any], + async () => undefined, + ); + (pool as any).failActive(new Error('synthetic batch failure')); + await assert.rejects(running, /synthetic batch failure/); + + let disposalSettled = false; + const disposal = pool.dispose().then(() => { + disposalSettled = true; + }); + await Promise.resolve(); + assert.equal(terminateCalls, 1); + assert.equal(disposalSettled, false); + + finishTerminate(0); + await disposal; + assert.equal(disposalSettled, true); +}); + +test('file-pass pool disposal rejects when a worker cannot be terminated', async () => { + const worker = { + postMessage: () => undefined, + on: () => worker, + terminate: () => Promise.reject(new Error('file-pass terminate failed')), + } as CodexFilePassWorkerLike; + const pool = new CodexFilePassPool(1, () => worker); + const running = pool.run( + [{ taskId: 'aggregate-only-termination-failure' } as any], + async () => undefined, + ); + (pool as any).failActive(new Error('synthetic batch failure')); + await assert.rejects(running, /synthetic batch failure/); + await assert.rejects(pool.dispose(), /file-pass terminate failed/); +}); + test('day or timezone changes invalidate the warm no-op metadata shortcut', async () => { const root = await mkdtemp(path.join(os.tmpdir(), 'ccu-codex-index-noop-boundary-')); try { diff --git a/src/test/codexIndexClient.test.ts b/src/test/codexIndexClient.test.ts index ba8ad91e..b69f1401 100644 --- a/src/test/codexIndexClient.test.ts +++ b/src/test/codexIndexClient.test.ts @@ -59,6 +59,22 @@ class FakeWorker extends EventEmitter implements CodexWorkerLike { } } +class DelayedTerminateWorker extends FakeWorker { + private finishTerminate!: (value: number) => void; + readonly terminateSettled = new Promise((resolve) => { + this.finishTerminate = resolve; + }); + + override terminate(): Promise { + this.terminated += 1; + return this.terminateSettled; + } + + finish(): void { + this.finishTerminate(0); + } +} + const request = { codexHome: '/private/runtime-only-codex-home', indexPath: '/private/global-storage/codex-index-v1.json', @@ -201,6 +217,7 @@ test('a worker error rejects all shared callers with a typed safe error', async ); }); } + assert.equal(worker.terminated, 1); }); test('dispose terminates the worker and rejects later refreshes', async () => { @@ -208,13 +225,76 @@ test('dispose terminates the worker and rejects later refreshes', async () => { const client = new CodexIndexClient(() => worker); const pending = client.refresh(request); - client.dispose(); + await client.dispose(); assert.equal(worker.terminated, 1); await assert.rejects(pending, { code: 'disposed' }); await assert.rejects(client.refresh(request), { code: 'disposed' }); }); +test('terminal refresh remains owned until the worker has actually terminated', async () => { + const firstWorker = new DelayedTerminateWorker(); + const secondWorker = new FakeWorker(); + const workers = [firstWorker, secondWorker]; + const client = new CodexIndexClient(() => { + const worker = workers.shift(); + if (!worker) throw new Error('unexpected worker allocation'); + return worker; + }); + let firstSettled = false; + const first = client.refresh(request).finally(() => { + firstSettled = true; + }); + const firstRequest = firstWorker.requests[0]; + if (firstRequest.type !== 'refresh') throw new Error('expected refresh'); + + firstWorker.emitMessage({ + type: 'result', + requestId: firstRequest.requestId, + result: result(), + }); + const sharedWhileStopping = client.refresh(request); + await Promise.resolve(); + assert.equal(firstSettled, false); + assert.equal(secondWorker.requests.length, 0); + + firstWorker.finish(); + await first; + await sharedWhileStopping; + assert.equal(firstWorker.terminated, 1); + assert.equal(secondWorker.requests.length, 0); + await client.dispose(); +}); + +test('dispose resolves only after actual worker termination settles', async () => { + const worker = new DelayedTerminateWorker(); + const client = new CodexIndexClient(() => worker); + const pending = client.refresh(request); + let disposed = false; + const disposal = client.dispose().then(() => { + disposed = true; + }); + + await Promise.resolve(); + assert.equal(worker.terminated, 1); + assert.equal(disposed, false); + await assert.rejects(pending, { code: 'disposed' }); + worker.finish(); + await disposal; + assert.equal(disposed, true); +}); + +test('dispose rejects when worker termination cannot be confirmed', async () => { + const worker = new FakeWorker(); + worker.terminate = () => Promise.reject(new Error('terminate failed')); + const client = new CodexIndexClient(() => worker); + const pending = client.refresh(request); + const pendingRejection = assert.rejects(pending, { code: 'disposed' }); + + await assert.rejects(client.dispose(), /terminate failed/); + await pendingRejection; +}); + test('the compiled worker keeps a safe project basename without raw identifiers or paths', async () => { const root = await mkdtemp(path.join(os.tmpdir(), 'ccu-codex-worker-')); const client = new CodexIndexClient(); @@ -395,6 +475,46 @@ test('unchanged worker refresh skips the final atomic index write', async () => assert.equal((resultMessage.result as any).indexChanged, false); }); +test('an ineligible background state disables historical passes without disabling the main refresh', async () => { + const previous = createEmptyCodexIndex(request.timeZone); + let historicalAllowed: boolean | undefined; + let pools = 0; + const messages: CodexWorkerMessage[] = []; + await runCodexWorkerRefresh( + { + type: 'refresh', + requestId: 'historical-cooldown', + ...request, + allowHistoricalBackfill: false, + }, + { + isCancelled: () => false, + post: (message) => messages.push(message), + acquireCodexIndexLease: async () => ({ release: async () => undefined }), + loadCodexIndex: async () => previous, + scanCodexManifest: async () => ({ files: [], persistable: {} }), + updateCodexIndex: async (_index, _manifest, options) => { + historicalAllowed = options.allowHistoricalBackfill; + return { + index: previous, + indexChanged: false, + bodyReads: 0, + failedFiles: 0, + migration: { filePasses: 0, bytesRead: 0, pending: true }, + }; + }, + saveCodexIndexAtomic: async () => undefined, + createCodexFilePassPool: () => { + pools += 1; + return { run: async () => undefined, dispose: async () => undefined }; + }, + }, + ); + assert.equal(historicalAllowed, false); + assert.equal(pools, 0); + assert.equal(messages.some((message) => message.type === 'result'), true); +}); + test('a refresh already saved by its final checkpoint skips the duplicate final write', async () => { const savedIndex = createEmptyCodexIndex(request.timeZone); let saves = 0; diff --git a/src/test/codexPeriodIndex.test.ts b/src/test/codexPeriodIndex.test.ts index c60a4a1a..6e1ed55a 100644 --- a/src/test/codexPeriodIndex.test.ts +++ b/src/test/codexPeriodIndex.test.ts @@ -2,8 +2,12 @@ import { test } from 'node:test'; import * as assert from 'node:assert/strict'; import { + CODEX_ROLLING_HOURLY_DAYS, CodexDailySlice, + CodexHourlyDays, CodexHourlySlice, + pruneCodexHourlyDays, + reduceCodexRollingHourlySlice, reduceCodexHourlySlice, reduceCodexStructuralSlice, reduceCodexUsageSlice, @@ -158,3 +162,39 @@ test('hourly reducer retains only the requested local day as sparse hour buckets assert.equal(hours['02'].total.outputTotal, 5); assert.equal('byEffort' in hours['00'], false); }); + +test('rolling hourly reducer keeps sparse configured-zone days and prunes day 31', () => { + const days: CodexHourlyDays = {}; + const allowedDays = new Set([ + '2026-06-22', + '2026-07-20', + '2026-07-21', + ]); + + reduceCodexRollingHourlySlice( + days, + usage('2026-06-21T15:55:00.000Z', 25, 5, 3), + allowedDays, + 'Asia/Hong_Kong', + ); + reduceCodexRollingHourlySlice( + days, + usage('2026-06-21T16:05:00.000Z', 50, 10, 6), + allowedDays, + 'Asia/Hong_Kong', + ); + reduceCodexRollingHourlySlice( + days, + usage('2026-07-20T16:05:00.000Z', 75, 15, 9), + allowedDays, + 'Asia/Hong_Kong', + ); + + assert.equal(CODEX_ROLLING_HOURLY_DAYS, 30); + assert.deepEqual(Object.keys(days).sort(), ['2026-06-22', '2026-07-21']); + assert.equal(days['2026-06-22']['00'].total.inputTotal, 50); + assert.equal(days['2026-07-21']['00'].total.outputTotal, 15); + + pruneCodexHourlyDays(days, new Set(['2026-07-21'])); + assert.deepEqual(Object.keys(days), ['2026-07-21']); +}); diff --git a/src/test/codexUsage.test.ts b/src/test/codexUsage.test.ts index 390745f4..2c3ad3ef 100644 --- a/src/test/codexUsage.test.ts +++ b/src/test/codexUsage.test.ts @@ -436,24 +436,66 @@ test('daily and monthly Codex rows retain exact-model API-equivalent cost and co test('Today is a calendar-day scope with exact sparse hourly cost and token composition', () => { const snapshot = snapshotFixture(); - for (const [index, file] of snapshot.files.slice(0, 2).entries()) { - const slice = file.period!.days['2026-07-20']; - const hour = index === 0 ? '10' : '11'; + for (const [index, file] of snapshot.files.slice(0, 3).entries()) { + const day = index < 2 ? '2026-07-20' : '2026-07-10'; + const slice = file.period!.days[day]; + const hours = index === 0 + ? { + '10': { + total: { + inputTotal: 250, + cachedInput: 200, + outputTotal: 50, + reasoningOutput: 30, + sourceTotal: 300, + }, + byModel: { + 'gpt-5.6-sol': { + inputTotal: 250, + cachedInput: 200, + outputTotal: 50, + reasoningOutput: 30, + sourceTotal: 300, + }, + }, + }, + '11': { + total: { + inputTotal: 250, + cachedInput: 200, + outputTotal: 50, + reasoningOutput: 30, + sourceTotal: 300, + }, + byModel: { + 'gpt-5.6-sol': { + inputTotal: 250, + cachedInput: 200, + outputTotal: 50, + reasoningOutput: 30, + sourceTotal: 300, + }, + }, + }, + } + : { + [index === 1 ? '11' : '09']: { + total: { ...slice.total }, + byModel: Object.fromEntries( + Object.entries(slice.byModel).map(([model, tokens]) => [ + model, + { ...tokens }, + ]), + ), + }, + }; file.today = { day: '2026-07-20', timeZone: 'UTC', indexedThrough: file.period!.indexedThrough, - hours: { - [hour]: { - total: { ...slice.total }, - byModel: Object.fromEntries( - Object.entries(slice.byModel).map(([model, tokens]) => [ - model, - { ...tokens }, - ]), - ), - }, - }, + windowDays: 30, + days: { [day]: hours }, + hours: day === '2026-07-20' ? hours : {}, }; } snapshot.coverage.today = { @@ -465,17 +507,64 @@ test('Today is a calendar-day scope with exact sparse hourly cost and token comp totalBytes: 2, complete: true, }; + const hourlyCoverage = { + timeZone: 'UTC', + asOfDay: '2026-07-20', + windowDays: 30, + indexedFiles: 3, + totalFiles: 3, + indexedBytes: 3, + totalBytes: 3, + complete: true, + days: { + '2026-07-10': { + day: '2026-07-10', + indexedFiles: 1, + totalFiles: 1, + indexedBytes: 1, + totalBytes: 1, + complete: true, + }, + '2026-07-20': { + day: '2026-07-20', + indexedFiles: 2, + totalFiles: 2, + indexedBytes: 2, + totalBytes: 2, + complete: true, + }, + }, + }; + snapshot.coverage.hourly = hourlyCoverage; + snapshot.hourlyCoverage = hourlyCoverage; const view = buildCodexUsageView(snapshot, NOW); assert.equal(view.today.total.processed, 1_200); assert.equal(view.today.threads, 2); assert.deepEqual(view.todayHourly.map((row) => row.hour), ['10', '11']); - assert.equal(view.todayHourly[0].total.processed, 600); - assert.equal(view.todayHourly[0].apiEquivalent.equivalentUsd, 0.0037); + assert.deepEqual(view.todayHourly.map((row) => row.label), ['10:00', '11:00']); + assert.equal(view.todayHourly[0].total.processed, 300); + assert.equal(view.todayHourly[0].apiEquivalent.equivalentUsd, 0.00185); assert.equal(view.todayHourly[0].apiEquivalent.pricingCoverage, 1); assert.equal(view.todayHourly[0].threads, 1); assert.equal(view.todayCoverage.complete, true); + assert.equal(view.last30DaysHourlyByDay['2026-07-10'][0].hour, '09'); + assert.equal(view.last30DaysHourlyByDay['2026-07-10'][0].label, '09:00'); + assert.equal(view.last30DaysHourlyByDay['2026-07-10'][0].threads, 1); + assert.equal(view.hourlyCoverage.days['2026-07-10'].complete, true); + + const hourlyProcessed = view.todayHourly.reduce( + (sum, row) => sum + row.total.processed, + 0, + ); + const hourlyThreads = view.todayHourly.reduce( + (sum, row) => sum + row.threads, + 0, + ); + assert.equal(hourlyProcessed, view.today.total.processed); + assert.equal(hourlyThreads, 3); + assert.equal(view.today.threads, 2); }); test('rolling scopes stay anchored to snapshot coverage across Hong Kong midnight', () => { diff --git a/src/test/codexWebviewFixtures.ts b/src/test/codexWebviewFixtures.ts index 35f96f21..f717ea96 100644 --- a/src/test/codexWebviewFixtures.ts +++ b/src/test/codexWebviewFixtures.ts @@ -1,6 +1,8 @@ import { CodexProviderSnapshot } from '../providers/codex/codexProvider'; import { CodexFileAggregate } from '../providers/codex/codexIndex'; import { ProviderTokenCounts } from '../providers/providerTypes'; +import { hourKeyInZone } from '../dateKeys'; +import { CODEX_ROLLING_HOURLY_DAYS } from '../providers/codex/codexPeriodIndex'; import { codexFixtureIdentityKey, snapshotFixture } from './codexFixtures'; export const CODEX_WEBVIEW_NOW = Date.parse('2026-07-20T12:00:00.000Z'); @@ -52,6 +54,11 @@ function generatedFile(index: number, template: CodexFileAggregate): CodexFileAg compactCount: index % 7 === 0 ? 1 : 0, taskCompleteCount: role === 'root' ? 1 : 0, }; + const hour = hourKeyInZone(new Date(endedAt), 'Asia/Hong_Kong'); + const hourlySlice = { + total: { ...total }, + byModel: { [model]: { ...total } }, + }; return { ...structuredClone(template), @@ -86,19 +93,14 @@ function generatedFile(index: number, template: CodexFileAggregate): CodexFileAg }, }, }, - today: dayKey === '2026-07-20' - ? { - day: dayKey, - timeZone: 'Asia/Hong_Kong', - indexedThrough: 1, - hours: { - [String(8 + (index % 4)).padStart(2, '0')]: { - total: { ...total }, - byModel: { [model]: { ...total } }, - }, - }, - } - : undefined, + today: { + day: '2026-07-20', + timeZone: 'Asia/Hong_Kong', + indexedThrough: 1, + windowDays: CODEX_ROLLING_HOURLY_DAYS, + days: { [dayKey]: { [hour]: hourlySlice } }, + hours: dayKey === '2026-07-20' ? { [hour]: hourlySlice } : {}, + }, }; } @@ -113,10 +115,36 @@ export function codexWebviewFixture(): CodexProviderSnapshot { totalBytes: 3_000_000, complete: false, }; + const hourlyDays = Object.fromEntries( + [...new Set(files.flatMap((file) => Object.keys(file.today?.days ?? {})))] + .map((day) => { + const totalFiles = files.filter((file) => file.today?.days?.[day]).length; + return [day, { + day, + indexedFiles: totalFiles, + totalFiles, + indexedBytes: totalFiles, + totalBytes: totalFiles, + complete: true, + }]; + }), + ); + const hourlyCoverage = { + timeZone: 'Asia/Hong_Kong', + asOfDay: '2026-07-20', + windowDays: CODEX_ROLLING_HOURLY_DAYS, + indexedFiles: files.length, + totalFiles: files.length, + indexedBytes: files.length, + totalBytes: files.length, + complete: false, + days: hourlyDays, + }; return { ...base, files, total: sumTotals(files), + hourlyCoverage, weeklyValueInputs: { observations: [ { @@ -171,6 +199,7 @@ export function codexWebviewFixture(): CodexProviderSnapshot { last30Days: { ...periodCoverage }, allTime: { ...periodCoverage }, }, + hourly: hourlyCoverage, today: { timeZone: 'Asia/Hong_Kong', day: '2026-07-20', @@ -232,6 +261,20 @@ export function unknownModelCodexWebviewFixture(): CodexProviderSnapshot { }, ]), ), + days: Object.fromEntries( + Object.entries(file.today.days ?? {}).map(([day, hours]) => [ + day, + Object.fromEntries( + Object.entries(hours).map(([hour, slice]) => [ + hour, + { + ...slice, + byModel: { 'unknown-model-fixture': { ...slice.total } }, + }, + ]), + ), + ]), + ), } : undefined, })), diff --git a/src/test/dataLoader.test.ts b/src/test/dataLoader.test.ts index c1887824..07a96324 100644 --- a/src/test/dataLoader.test.ts +++ b/src/test/dataLoader.test.ts @@ -6,6 +6,7 @@ import * as path from 'node:path'; import { scanUsageManifest } from '../claudeUsageFiles'; import { ClaudeDataLoader } from '../dataLoader'; +import { I18n } from '../i18n'; import { ClaudeUsageRecord } from '../types'; const tempRoots: string[] = []; @@ -48,6 +49,43 @@ test('getCurrentContextInfo keeps the 200K window for pre-4.6 Opus and Sonnet', } }); +test('day attribution uses the configured timezone instead of the host timezone', () => { + const previousTimeZone = I18n.getTimezone(); + const fixedNow = new Date('2026-07-21T02:00:00.000Z'); + const attributionRecord = ( + timestamp: string, + inputTokens: number, + sessionId: string, + ): ClaudeUsageRecord => ({ + timestamp, + _sessionId: sessionId, + message: { + model: 'claude-sonnet-4-5', + usage: { input_tokens: inputTokens, output_tokens: 10 }, + }, + }); + + try { + I18n.setTimezone('America/New_York'); + const attribution = ClaudeDataLoader.getUsageAttribution( + [ + attributionRecord('2026-07-20T03:30:00.000Z', 200, 'previous-day'), + attributionRecord('2026-07-21T01:00:00.000Z', 100, 'configured-today'), + ], + null, + { kind: 'day' }, + fixedNow, + ); + + assert.equal(attribution.totalTokens, 110); + assert.deepEqual(attribution.models.map(({ key, count }) => ({ key, count })), [ + { key: 'claude-sonnet-4-5', count: 1 }, + ]); + } finally { + I18n.setTimezone(previousTimeZone); + } +}); + test('an injected manifest is authoritative and returns anonymous load counters', async () => { const manifestRoot = await mkdtemp(path.join(os.tmpdir(), 'ccu-loader-manifest-')); const emptyArgumentRoot = await mkdtemp(path.join(os.tmpdir(), 'ccu-loader-empty-')); diff --git a/src/test/dateKeys.test.ts b/src/test/dateKeys.test.ts index 390c819a..ee9372fa 100644 --- a/src/test/dateKeys.test.ts +++ b/src/test/dateKeys.test.ts @@ -2,6 +2,7 @@ import { test } from 'node:test'; import * as assert from 'node:assert/strict'; import { dayKeyInZone, + formatHourLabel, hourKeyInZone, monthKeyInZone, resolveTimeZone, @@ -93,6 +94,15 @@ test('hour keys use the same target zone and a stable 00-23 clock', () => { assert.equal(hourKeyInZone(new Date('nonsense'), 'UTC'), ''); }); +test('hour labels use one strict HH:00 formatter for every provider', () => { + assert.equal(formatHourLabel('00'), '00:00'); + assert.equal(formatHourLabel('09'), '09:00'); + assert.equal(formatHourLabel('23'), '23:00'); + assert.equal(formatHourLabel('24'), ''); + assert.equal(formatHourLabel('9'), ''); + assert.equal(formatHourLabel('09:00'), ''); +}); + test('timezone resolution returns a usable canonical zone', () => { assert.equal(resolveTimeZone('Asia/Hong_Kong'), 'Asia/Hong_Kong'); assert.notEqual(resolveTimeZone(''), ''); diff --git a/src/test/extensionRefreshRouting.test.ts b/src/test/extensionRefreshRouting.test.ts index 9b662b2f..d7299725 100644 --- a/src/test/extensionRefreshRouting.test.ts +++ b/src/test/extensionRefreshRouting.test.ts @@ -4,6 +4,7 @@ import * as fs from 'node:fs'; import * as os from 'node:os'; import * as path from 'node:path'; import { WindowActivityGate } from '../refreshPolicy'; +import { ResourceOwnershipRegistry } from '../resourceOwnership'; type ExtensionModule = typeof import('../extension'); @@ -33,7 +34,14 @@ function loadExtensionModule(): ExtensionModule { const { ClaudeCodeUsageExtension } = loadExtensionModule(); function bareExtension(): any { - return Object.create(ClaudeCodeUsageExtension.prototype) as any; + const extension = Object.create(ClaudeCodeUsageExtension.prototype) as any; + extension.resourceOwnership = new ResourceOwnershipRegistry(); + extension.codexWatcherLeases = new Map(); + extension.pendingResourceStops = new Set(); + extension.resourceStopFailure = null; + extension.codexWatcherGeneration = 0; + extension.disposed = false; + return extension; } test('Claude watcher refresh stays Claude-only while manual refresh updates both providers', async () => { diff --git a/src/test/extensionWindowActivity.test.ts b/src/test/extensionWindowActivity.test.ts index 727f8e53..8cd01415 100644 --- a/src/test/extensionWindowActivity.test.ts +++ b/src/test/extensionWindowActivity.test.ts @@ -6,6 +6,14 @@ import * as path from 'node:path'; import { ClaudeDataLoader } from '../dataLoader'; import { WindowActivityGate } from '../refreshPolicy'; +import { + beginBackgroundWork, + createBackgroundWorkState, + pauseBackgroundWork, + recordBackgroundWorkFailure, + recordBackgroundWorkProgress, +} from '../backgroundWorkState'; +import { ResourceOwnershipRegistry } from '../resourceOwnership'; import { snapshotFixture } from './codexFixtures'; type ExtensionModule = typeof import('../extension'); @@ -36,7 +44,37 @@ function loadExtensionModule(): ExtensionModule { const { ClaudeCodeUsageExtension } = loadExtensionModule(); function bareExtension(): any { - return Object.create(ClaudeCodeUsageExtension.prototype) as any; + const extension = Object.create(ClaudeCodeUsageExtension.prototype) as any; + extension.resourceOwnership = new ResourceOwnershipRegistry(); + extension.codexWatcherLeases = new Map(); + extension.debounceTimerLeases = new Map(); + extension.activeAdviceNetworks = new Map(); + extension.activeQuotaNetworks = new Map(); + extension.codexProviderRetirements = new Set(); + extension.activeCodexRefreshes = new Set(); + extension.pendingResourceStops = new Set(); + extension.resourceStopFailure = null; + extension.codexProviderRetirementFailure = null; + extension.codexBackgroundStateWrite = Promise.resolve(); + extension.configurationGeneration = 0; + extension.fileWatcherGeneration = 0; + extension.codexWatcherGeneration = 0; + extension.credentialsWatcherGeneration = 0; + extension.disposed = false; + extension.codexBackgroundState = createBackgroundWorkState({ + measurementVersion: 1, + reason: 'first-index', + now: 0, + }); + extension.codexFirstBackfillActive = false; + extension.codexWorkerCancellationRequested = false; + extension.context = { + globalState: { + update: async () => undefined, + }, + }; + extension.outputChannel = { appendLine: () => undefined }; + return extension; } test('background transition stops every Claude and Codex recurring resource once', () => { @@ -196,15 +234,15 @@ test('Codex becomes available in the dashboard before a slow cold index finishes }); const pending = extension.runCodexRefresh('startup'); - await Promise.resolve(); - await Promise.resolve(); + await new Promise((resolve) => setImmediate(resolve)); - assert.deepEqual(states, [{ + assert.deepEqual(states[0], { available: true, hasData: false, refreshing: true, scannedFiles: null, - }]); + }); + assert.ok(states.every((state) => state.available && state.refreshing)); assert.deepEqual(liveProgress, [12]); releaseRefresh?.({ outcome: 'unavailable' }); @@ -245,8 +283,7 @@ test('a verified Codex checkpoint stays visible while its index refresh continue }); const pending = extension.runCodexRefresh('startup'); - await Promise.resolve(); - await Promise.resolve(); + await new Promise((resolve) => setImmediate(resolve)); assert.ok(releaseRefresh, 'the background worker refresh should still be running'); assert.ok( @@ -299,11 +336,17 @@ test('Codex live index progress coalesces dashboard renders', () => { const originalNow = Date.now; let now = 1_000; let renders = 0; + const reasons: unknown[] = []; extension.codexProgress = null; extension.codexProgressLastRenderedAt = 0; + extension.codexBackgroundState = beginBackgroundWork( + extension.codexBackgroundState, + { trigger: 'automatic', now, reason: 'first-index' }, + ).state; extension.webviewProvider = { - updateCodexProgress: () => { + updateCodexProgress: (rendered: { reason?: unknown }) => { renders += 1; + reasons.push(rendered.reason); }, }; const progress = (scannedFiles: number) => ({ @@ -330,11 +373,661 @@ test('Codex live index progress coalesces dashboard renders', () => { extension.onCodexIndexProgress(progress(40)); assert.equal(renders, 2); assert.equal(extension.codexProgress.scannedFiles, 40); + assert.deepEqual(reasons, ['first-index', 'first-index']); } finally { Date.now = originalNow; } }); +test('cooldown, user pause, and completed measurement suppress historical backfill', async () => { + const originalNow = Date.now; + Date.now = () => 1_001; + const snapshot = snapshotFixture(); + snapshot.coverage.complete = false; + snapshot.coverage.indexedFiles = Math.max(0, snapshot.coverage.totalFiles - 1); + const progress = { + completedUnits: 1, + totalUnits: 2, + completedBytes: 10, + totalBytes: 20, + }; + const seed = createBackgroundWorkState({ + measurementVersion: 1, + reason: 'history-backfill', + now: 1_000, + progress, + }); + const running = beginBackgroundWork(seed, { + trigger: 'automatic', + now: 1_000, + }).state; + const states = [ + recordBackgroundWorkFailure(running, { now: 1_000 }), + pauseBackgroundWork(seed, { now: 1_000 }), + recordBackgroundWorkProgress(running, { + now: 1_000, + complete: true, + progress, + }), + ]; + const allowed: boolean[] = []; + + try { + for (const state of states) { + const extension = bareExtension(); + extension.codexBackgroundState = state; + extension.getConfiguration = () => ({ codexEnabled: true }); + extension.webviewProvider = { updateCodexProgress: () => undefined }; + extension.syncProviderUi = () => undefined; + extension.codexProvider = { + isAvailable: async () => true, + loadPersistedSnapshot: async () => snapshot, + refresh: async ( + _profile: string, + _onProgress: unknown, + allowHistoricalBackfill: boolean, + ) => { + allowed.push(allowHistoricalBackfill); + return { + outcome: 'partial', + snapshot, + diagnostic: { + bodyReads: 0, + failedFiles: 0, + metadataMs: 0, + parseMs: 0, + migrationPending: true, + }, + }; + }, + }; + + await extension.runCodexRefresh('poll'); + assert.equal(extension.codexBackgroundState.status, state.status); + } + } finally { + Date.now = originalNow; + } + + assert.deepEqual(allowed, [false, false, false]); +}); + +test('a missing persisted index invalidates same-version complete background work', async () => { + const extension = bareExtension(); + const completeSnapshot = snapshotFixture(); + const seed = createBackgroundWorkState({ + measurementVersion: 1, + reason: 'history-backfill', + now: 1, + progress: { + completedUnits: 1, + totalUnits: 1, + completedBytes: 1, + totalBytes: 1, + }, + }); + extension.codexBackgroundState = recordBackgroundWorkProgress( + beginBackgroundWork(seed, { trigger: 'automatic', now: 1 }).state, + { + now: 2, + complete: true, + progress: seed.progress, + }, + ); + extension.getConfiguration = () => ({ codexEnabled: true }); + extension.windowActivity = new WindowActivityGate(true); + extension.webviewProvider = { updateCodexProgress: () => undefined }; + extension.syncProviderUi = () => undefined; + let historicalAllowed: boolean | undefined; + extension.codexProvider = { + isAvailable: async () => true, + loadPersistedSnapshot: async () => null, + refresh: async ( + _profile: string, + _onProgress: unknown, + allowHistoricalBackfill: boolean, + ) => { + historicalAllowed = allowHistoricalBackfill; + return { + outcome: 'success', + snapshot: completeSnapshot, + diagnostic: { + bodyReads: 1, + failedFiles: 0, + metadataMs: 1, + parseMs: 1, + migrationPending: false, + }, + }; + }, + }; + + await extension.runCodexRefresh('startup'); + + assert.equal(historicalAllowed, true); + assert.equal(extension.codexBackgroundState.status, 'eligible'); +}); + +test('a failed preflight state write never strands in-memory background work as running', async () => { + const extension = bareExtension(); + const snapshot = snapshotFixture(); + snapshot.coverage.complete = false; + snapshot.coverage.indexedFiles = Math.max(0, snapshot.coverage.totalFiles - 1); + let writes = 0; + extension.context.globalState.update = async () => { + writes += 1; + if (writes === 1) throw new Error('persistence unavailable'); + }; + extension.getConfiguration = () => ({ codexEnabled: true }); + extension.windowActivity = new WindowActivityGate(true); + extension.webviewProvider = { updateCodexProgress: () => undefined }; + extension.syncProviderUi = () => undefined; + extension.codexProvider = { + isAvailable: async () => true, + loadPersistedSnapshot: async () => snapshot, + refresh: async () => assert.fail('worker must not start before state is durable'), + }; + + await assert.rejects(extension.runCodexRefresh('poll'), /persistence unavailable/); + + assert.equal(extension.codexBackgroundState.status, 'eligible'); + assert.equal(extension.codexBackgroundState.reason, 'resume'); + assert.ok(writes >= 1); +}); + +test('background state writes are serialized and preserve their captured order', async () => { + const extension = bareExtension(); + let finishFirst!: () => void; + const firstGate = new Promise((resolve) => { finishFirst = resolve; }); + const writes: Array<{ status: string; reason: string }> = []; + extension.context.globalState.update = async (_key: string, value: any) => { + writes.push({ status: value.status, reason: value.reason }); + if (writes.length === 1) await firstGate; + }; + + const first = extension.saveCodexBackgroundState(); + extension.codexBackgroundState = beginBackgroundWork( + extension.codexBackgroundState, + { trigger: 'automatic', now: 1, reason: 'first-index' }, + ).state; + const second = extension.saveCodexBackgroundState(); + await new Promise((resolve) => setImmediate(resolve)); + assert.deepEqual(writes, [{ status: 'eligible', reason: 'first-index' }]); + + finishFirst(); + await Promise.all([first, second]); + assert.deepEqual(writes, [ + { status: 'eligible', reason: 'first-index' }, + { status: 'running', reason: 'first-index' }, + ]); +}); + +test('a recovered index invalidates complete state and queues historical reconciliation', async () => { + const extension = bareExtension(); + const pending = snapshotFixture(); + pending.coverage.complete = false; + pending.coverage.indexedFiles = Math.max(0, pending.coverage.totalFiles - 1); + const seed = createBackgroundWorkState({ + measurementVersion: 1, + reason: 'history-backfill', + now: 1, + progress: { + completedUnits: 1, + totalUnits: 1, + completedBytes: 1, + totalBytes: 1, + }, + }); + extension.codexBackgroundState = recordBackgroundWorkProgress( + beginBackgroundWork(seed, { trigger: 'automatic', now: 1 }).state, + { now: 2, complete: true, progress: seed.progress }, + ); + extension.getConfiguration = () => ({ codexEnabled: true }); + extension.windowActivity = new WindowActivityGate(true); + extension.webviewProvider = { updateCodexProgress: () => undefined }; + extension.syncProviderUi = () => undefined; + const continuations: string[] = []; + extension.refreshCodexData = async (trigger: string) => { + continuations.push(trigger); + }; + extension.codexProvider = { + isAvailable: async () => true, + loadPersistedSnapshot: async () => pending, + refresh: async () => ({ + outcome: 'partial', + snapshot: pending, + diagnostic: { + bodyReads: 0, + failedFiles: 0, + metadataMs: 1, + parseMs: 1, + migrationPending: true, + indexRecovery: { reason: 'invalid-json' }, + }, + }), + }; + + await extension.runCodexRefresh('poll'); + await Promise.resolve(); + await Promise.resolve(); + + assert.equal(extension.codexBackgroundState.status, 'eligible'); + assert.deepEqual(continuations, ['poll']); +}); + +test('successful historical progress queues one immediate continuation without a timer', async () => { + const extension = bareExtension(); + const before = snapshotFixture(); + const after = structuredClone(before); + before.coverage.complete = false; + after.coverage.complete = false; + before.coverage.indexedFiles = Math.max(0, before.coverage.totalFiles - 2); + after.coverage.indexedFiles = Math.max(0, after.coverage.totalFiles - 1); + before.coverage.indexedBytes = Math.max(0, before.coverage.totalBytes - 200); + after.coverage.indexedBytes = Math.max(0, after.coverage.totalBytes - 100); + const continuations: string[] = []; + extension.getConfiguration = () => ({ codexEnabled: true }); + extension.windowActivity = new WindowActivityGate(true); + extension.webviewProvider = { updateCodexProgress: () => undefined }; + extension.syncProviderUi = () => undefined; + extension.refreshCodexData = async (trigger: string) => { + continuations.push(trigger); + }; + extension.codexProvider = { + isAvailable: async () => true, + loadPersistedSnapshot: async () => before, + refresh: async () => ({ + outcome: 'partial', + snapshot: after, + diagnostic: { + bodyReads: 1, + failedFiles: 0, + metadataMs: 1, + parseMs: 1, + migrationPending: true, + }, + }), + }; + + await extension.runCodexRefresh('startup'); + await Promise.resolve(); + await Promise.resolve(); + + assert.equal(extension.codexBackgroundState.status, 'eligible'); + assert.deepEqual(continuations, ['startup']); + assert.equal(extension.resourceOwnership.snapshotForTests().activeCount, 0); +}); + +test('timer and watcher ownership clear only after their real handles stop', async () => { + const extension = bareExtension(); + extension.refreshGen = 0; + extension.watchDebounce = { clear: () => undefined }; + let fired = false; + let watcherClosed = 0; + const timer = setTimeout(() => { fired = true; }, 20); + extension.refreshTimer = timer; + extension.refreshTimerLease = extension.resourceOwnership.register({ + kind: 'timer', + capability: 'refresh', + scope: 'extension', + creator: 'refresh-coordinator', + stopConditions: ['window-blur'], + boundedException: 'none', + }); + extension.fileWatcher = { + close: () => { watcherClosed += 1; }, + }; + extension.fileWatcherLease = extension.resourceOwnership.register({ + kind: 'watcher', + capability: 'refresh', + scope: 'claude', + creator: 'extension', + stopConditions: ['window-blur'], + boundedException: 'none', + }); + + extension.stopAutoRefresh('window-blur'); + extension.stopFileWatching('window-blur'); + await Promise.resolve(); + await Promise.resolve(); + + assert.equal(watcherClosed, 1); + assert.equal(extension.resourceOwnership.snapshotForTests().activeCount, 0); + await new Promise((resolve) => setTimeout(resolve, 30)); + assert.equal(fired, false); +}); + +test('watch quiet-delay timers are owned and actually cancelled', async () => { + const extension = bareExtension(); + extension.debounceTimerLeases = new Map(); + const debounce = extension.createOwnedRefreshDebounce('codex'); + let fired = false; + + debounce.push(20, () => { fired = true; }); + assert.equal(extension.resourceOwnership.snapshotForTests().byKind.timer, 1); + debounce.clear(); + await Promise.resolve(); + await Promise.resolve(); + assert.equal(extension.resourceOwnership.snapshotForTests().byKind.timer, 0); + await new Promise((resolve) => setTimeout(resolve, 30)); + assert.equal(fired, false); +}); + +test('backgrounding aborts an active quota network before releasing ownership', async () => { + const extension = bareExtension(); + let observedSignal: AbortSignal | undefined; + extension.cache = { + usageLimits: null, + usageLimitsLastUpdate: new Date(0), + usageLimitsBackoffUntil: new Date(0), + usageLimitsFailStreak: 0, + }; + extension.isActive = () => false; + extension.apiClient = { + fetchUsageLimits: (signal?: AbortSignal) => new Promise((resolve) => { + observedSignal = signal; + assert.ok(signal); + signal.addEventListener('abort', () => resolve(null), { once: true }); + }), + }; + + const pending = extension.maybeFetchUsageLimits({ usageLimitTracking: true }); + await Promise.resolve(); + assert.equal(extension.resourceOwnership.snapshotForTests().byKind.network, 1); + + await extension.cancelQuotaNetworks('window-blur'); + assert.equal(observedSignal?.aborted, true); + await pending; + assert.equal(extension.resourceOwnership.snapshotForTests().byKind.network, 0); + assert.equal(extension.cache.usageLimitsFailStreak, 0); +}); + +test('network ownership remains active until an aborted request actually settles', async () => { + const extension = bareExtension(); + let releaseRequest!: () => void; + let aborted = false; + const pending = extension.runAdviceNetwork((signal: AbortSignal) => + new Promise((_resolve, reject) => { + signal.addEventListener('abort', () => { + aborted = true; + releaseRequest = () => reject(new Error('request finally settled')); + }, { once: true }); + }), + ); + await Promise.resolve(); + assert.equal(extension.resourceOwnership.snapshotForTests().byKind.network, 1); + + let cancellationSettled = false; + const cancellation = extension.cancelAdviceNetworks('cancelled').then(() => { + cancellationSettled = true; + }); + await Promise.resolve(); + await Promise.resolve(); + + assert.equal(aborted, true); + assert.equal(cancellationSettled, false); + assert.equal(extension.resourceOwnership.snapshotForTests().byKind.network, 1); + + releaseRequest(); + await assert.rejects(pending, /finally settled/); + await cancellation; + assert.equal(extension.resourceOwnership.snapshotForTests().byKind.network, 0); +}); + +test('quota cold retry timer is owned and cleared on blur', async () => { + const extension = bareExtension(); + const originalSetTimeout = global.setTimeout; + const originalClearTimeout = global.clearTimeout; + let cleared = 0; + const fakeTimer = { fake: true } as unknown as NodeJS.Timeout; + global.setTimeout = ((_callback: () => void, _ms: number) => + fakeTimer) as typeof setTimeout; + global.clearTimeout = ((handle: NodeJS.Timeout) => { + assert.equal(handle, fakeTimer); + cleared += 1; + }) as typeof clearTimeout; + extension.windowActivity = new WindowActivityGate(true); + extension.getConfiguration = () => ({ usageLimitTracking: true }); + extension.maybeFetchUsageLimits = async () => null; + + try { + extension.scheduleQuotaColdRetry(); + assert.equal(extension.resourceOwnership.snapshotForTests().byKind.timer, 1); + extension.stopQuotaColdRetry('window-blur'); + await Promise.resolve(); + await Promise.resolve(); + assert.equal(cleared, 1); + assert.equal(extension.resourceOwnership.snapshotForTests().byKind.timer, 0); + } finally { + global.setTimeout = originalSetTimeout; + global.clearTimeout = originalClearTimeout; + } +}); + +test('extension disposal releases worker and backfill only after provider termination', async () => { + const extension = bareExtension(); + let finishProviderDispose!: () => void; + const providerDisposed = new Promise((resolve) => { + finishProviderDispose = resolve; + }); + let providerCancelCalls = 0; + let hostDisposals = 0; + extension.stopAutoRefresh = () => undefined; + extension.stopFileWatching = () => undefined; + extension.stopCodexWatching = () => undefined; + extension.stopCredentialsWatching = () => undefined; + extension.codexProviderRetirements = new Set(); + extension.codexProvider = { + cancel: () => { providerCancelCalls += 1; }, + dispose: () => providerDisposed, + }; + extension.statusBar = { dispose: () => { hostDisposals += 1; } }; + extension.webviewProvider = { dispose: () => { hostDisposals += 1; } }; + extension.codexWorkerLease = extension.resourceOwnership.register({ + kind: 'worker', + capability: 'codex-index', + scope: 'codex', + creator: 'codex-index-client', + stopConditions: ['extension-dispose'], + boundedException: 'none', + }); + extension.codexBackfillLease = extension.resourceOwnership.register({ + kind: 'backfill', + capability: 'codex-history', + scope: 'codex', + creator: 'refresh-coordinator', + stopConditions: ['extension-dispose'], + boundedException: 'first-codex-history', + }); + + let disposalSettled = false; + const disposal = extension.dispose().then(() => { + disposalSettled = true; + }); + await new Promise((resolve) => setImmediate(resolve)); + + assert.equal(providerCancelCalls, 1); + assert.equal(disposalSettled, false); + assert.equal(extension.resourceOwnership.snapshotForTests().activeCount, 2); + assert.equal(hostDisposals, 0); + + finishProviderDispose(); + await disposal; + assert.equal(extension.resourceOwnership.snapshotForTests().activeCount, 0); + assert.equal(hostDisposals, 2); +}); + +test('extension disposal keeps worker ownership active when termination fails', async () => { + const extension = bareExtension(); + let hostDisposals = 0; + extension.stopAutoRefresh = () => undefined; + extension.stopFileWatching = () => undefined; + extension.stopCodexWatching = () => undefined; + extension.stopCredentialsWatching = () => undefined; + extension.codexProvider = { + cancel: () => undefined, + dispose: async () => { throw new Error('provider terminate failed'); }, + }; + extension.statusBar = { dispose: () => { hostDisposals += 1; } }; + extension.webviewProvider = { dispose: () => { hostDisposals += 1; } }; + extension.codexWorkerLease = extension.resourceOwnership.register({ + kind: 'worker', + capability: 'codex-index', + scope: 'codex', + creator: 'codex-index-client', + stopConditions: ['extension-dispose'], + boundedException: 'none', + }); + extension.codexBackfillLease = extension.resourceOwnership.register({ + kind: 'backfill', + capability: 'codex-history', + scope: 'codex', + creator: 'refresh-coordinator', + stopConditions: ['extension-dispose'], + boundedException: 'first-codex-history', + }); + + await assert.rejects(extension.dispose(), /provider terminate failed/); + + assert.equal(extension.resourceOwnership.snapshotForTests().activeCount, 2); + assert.equal(hostDisposals, 2); +}); + +test('a refresh termination failure retains its worker and backfill leases for disposal', async () => { + const extension = bareExtension(); + const snapshot = snapshotFixture(); + snapshot.coverage.complete = false; + snapshot.coverage.indexedFiles = Math.max(0, snapshot.coverage.totalFiles - 1); + extension.getConfiguration = () => ({ codexEnabled: true }); + extension.windowActivity = new WindowActivityGate(true); + extension.webviewProvider = { updateCodexProgress: () => undefined }; + extension.syncProviderUi = () => undefined; + extension.codexProvider = { + isAvailable: async () => true, + loadPersistedSnapshot: async () => snapshot, + refresh: async () => { throw new Error('worker could not terminate'); }, + }; + + await assert.rejects(extension.runCodexRefresh('poll'), /could not terminate/); + + assert.equal(extension.resourceOwnership.snapshotForTests().byKind.worker, 1); + assert.equal(extension.resourceOwnership.snapshotForTests().byKind.backfill, 1); + assert.equal(extension.codexWorkerLease?.active, true); + assert.equal(extension.codexBackfillLease?.active, true); +}); + +test('disposal drains every resource kind and stale callbacks cannot recreate work', async () => { + const extension = bareExtension(); + let releaseNetwork!: () => void; + let releaseProvider!: () => void; + let releaseStaleCallbacks!: () => void; + let watcherClosed = 0; + let providerCancelled = 0; + let hostDisposals = 0; + const providerDisposal = new Promise((resolve) => { + releaseProvider = resolve; + }); + const staleCallbacks = new Promise((resolve) => { + releaseStaleCallbacks = resolve; + }); + + extension.windowActivity = new WindowActivityGate(true); + extension.refreshGen = 0; + extension.refreshTimer = setTimeout(() => assert.fail('disposed timer fired'), 60_000); + extension.refreshTimerLease = extension.resourceOwnership.register({ + kind: 'timer', + capability: 'refresh', + scope: 'extension', + creator: 'refresh-coordinator', + stopConditions: ['extension-dispose'], + boundedException: 'none', + }); + extension.watchDebounce = { clear: () => undefined }; + extension.codexWatchDebounce = { clear: () => undefined }; + extension.fileWatcher = { close: () => { watcherClosed += 1; } }; + extension.fileWatcherLease = extension.resourceOwnership.register({ + kind: 'watcher', + capability: 'refresh', + scope: 'claude', + creator: 'extension', + stopConditions: ['extension-dispose'], + boundedException: 'none', + }); + extension.codexWatchers = []; + extension.codexWorkerLease = extension.resourceOwnership.register({ + kind: 'worker', + capability: 'codex-index', + scope: 'codex', + creator: 'codex-index-client', + stopConditions: ['extension-dispose'], + boundedException: 'none', + }); + extension.codexBackfillLease = extension.resourceOwnership.register({ + kind: 'backfill', + capability: 'codex-history', + scope: 'codex', + creator: 'refresh-coordinator', + stopConditions: ['extension-dispose'], + boundedException: 'first-codex-history', + }); + extension.codexProvider = { + cancel: () => { providerCancelled += 1; }, + dispose: () => providerDisposal, + }; + extension.statusBar = { dispose: () => { hostDisposals += 1; } }; + extension.webviewProvider = { dispose: () => { hostDisposals += 1; } }; + extension.stopQuotaColdRetry = ClaudeCodeUsageExtension.prototype['stopQuotaColdRetry']; + extension.stopAutoRefresh = ClaudeCodeUsageExtension.prototype['stopAutoRefresh']; + extension.stopFileWatching = ClaudeCodeUsageExtension.prototype['stopFileWatching']; + extension.stopCodexWatching = ClaudeCodeUsageExtension.prototype['stopCodexWatching']; + extension.stopCredentialsWatching = () => undefined; + + const network = extension.runAdviceNetwork((signal: AbortSignal) => + new Promise((_resolve, reject) => { + signal.addEventListener('abort', () => { + releaseNetwork = () => reject(new Error('network closed')); + }, { once: true }); + }), + ).catch(() => undefined); + await Promise.resolve(); + assert.deepEqual(extension.resourceOwnership.snapshotForTests().byKind, { + timer: 1, + watcher: 1, + worker: 1, + network: 1, + backfill: 1, + }); + + const stale = staleCallbacks.then(async () => { + extension.startAutoRefresh(); + await extension.startFileWatching(); + extension.startCodexWatching(); + extension.scheduleQuotaColdRetry(); + await extension.refreshData(false, 'poll'); + }); + const disposal = extension.dispose(); + await Promise.resolve(); + await Promise.resolve(); + assert.equal(extension.resourceOwnership.snapshotForTests().byKind.network, 1); + + releaseStaleCallbacks(); + releaseNetwork(); + releaseProvider(); + await Promise.all([network, stale, disposal]); + await Promise.resolve(); + await Promise.resolve(); + + assert.equal(watcherClosed, 1); + assert.equal(providerCancelled, 1); + assert.equal(hostDisposals, 2); + assert.equal(extension.resourceOwnership.snapshotForTests().activeCount, 0); + await assert.rejects( + extension.runAdviceNetwork(async () => undefined), + /disposed/i, + ); + assert.equal(extension.resourceOwnership.snapshotForTests().activeCount, 0); +}); + test('Claude watcher is not created when the window loses focus during directory lookup', async () => { const extension = bareExtension(); const root = fs.mkdtempSync(path.join(os.tmpdir(), 'ccu-claude-watch-focus-')); @@ -373,6 +1066,114 @@ test('Claude watcher is not created when the window loses focus during directory } }); +test('Claude watcher is not created after disposal or a stale settings generation', async () => { + const extension = bareExtension(); + const root = fs.mkdtempSync(path.join(os.tmpdir(), 'ccu-claude-watch-dispose-')); + fs.mkdirSync(path.join(root, 'projects')); + const originalFind = ClaudeDataLoader.findClaudeDataDirectory; + const originalWatch = fs.watch; + let resolveDirectory!: (value: string) => void; + let watchCalls = 0; + extension.windowActivity = new WindowActivityGate(true); + extension.watchDebounce = { clear: () => undefined }; + extension.fileWatcher = undefined; + extension.watchedDir = null; + extension.getConfiguration = () => ({ + fileWatchSeconds: 30, + dataDirectory: '', + }); + (ClaudeDataLoader as any).findClaudeDataDirectory = () => new Promise((resolve) => { + resolveDirectory = resolve; + }); + (fs as any).watch = () => { + watchCalls += 1; + return { close: () => undefined }; + }; + + try { + const pending = extension.startFileWatching(); + extension.disposed = true; + extension.fileWatcherGeneration += 1; + resolveDirectory(root); + await pending; + assert.equal(watchCalls, 0); + assert.equal(extension.fileWatcher, undefined); + } finally { + (ClaudeDataLoader as any).findClaudeDataDirectory = originalFind; + (fs as any).watch = originalWatch; + fs.rmSync(root, { recursive: true, force: true }); + } +}); + +test('rapid settings changes wait for every provider retirement and only latest generation restarts', async () => { + const extension = bareExtension(); + let finishFirst!: () => void; + let finishSecond!: () => void; + const firstDisposal = new Promise((resolve) => { finishFirst = resolve; }); + const secondDisposal = new Promise((resolve) => { finishSecond = resolve; }); + const calls: string[] = []; + const provider = (name: string, disposal: Promise) => ({ + cancel: () => calls.push(`${name}:cancel`), + dispose: () => disposal, + }); + const first = provider('first', firstDisposal); + const second = provider('second', secondDisposal); + const third = provider('third', Promise.resolve()); + const replacements = [second, third]; + extension.codexProvider = first; + extension.windowActivity = new WindowActivityGate(true); + extension.webviewProvider = { + invalidatePreparedAiRequests: () => undefined, + }; + extension.statusBar = { + setVisibility: () => undefined, + }; + extension.getConfiguration = () => ({ + language: 'en', + decimalPlaces: 2, + tokenDecimalPlaces: 0, + compactNumbers: true, + timezone: 'UTC', + showCost: true, + showContext: true, + usageLimitTracking: false, + statusBarMetric: 'tokens', + showScopedWeekly: false, + quotaFiveHourOnly: false, + showResetInStatusBar: false, + resetCountdownFormat: 'short', + dataDirectory: '', + }); + extension.cancelAdviceNetworks = async () => undefined; + extension.cancelQuotaNetworks = async () => undefined; + extension.stopQuotaColdRetry = () => undefined; + extension.startAutoRefresh = () => undefined; + extension.stopFileWatching = () => undefined; + extension.stopCodexWatching = () => undefined; + extension.stopCredentialsWatching = () => undefined; + extension.selectClaudeProfile = () => undefined; + extension.createCodexProvider = () => replacements.shift(); + extension.refreshData = async () => { calls.push('refresh'); }; + extension.startFileWatching = async () => { calls.push('claude:start'); }; + extension.startCodexWatching = () => { calls.push('codex:start'); }; + extension.startCredentialsWatching = () => { calls.push('credentials:start'); }; + + extension.onConfigurationChanged(); + extension.onConfigurationChanged(); + finishSecond(); + await Promise.resolve(); + await Promise.resolve(); + assert.deepEqual(calls.filter((call) => call === 'refresh'), []); + + finishFirst(); + await new Promise((resolve) => setImmediate(resolve)); + await new Promise((resolve) => setImmediate(resolve)); + assert.deepEqual(calls.filter((call) => call === 'refresh'), ['refresh']); + assert.equal(calls.filter((call) => call === 'claude:start').length, 1); + assert.equal(calls.filter((call) => call === 'codex:start').length, 1); + assert.equal(calls.filter((call) => call === 'credentials:start').length, 1); +}); + test('credentials change clears quota failure backoff before refreshing', async () => { const extension = bareExtension(); const calls: string[] = []; diff --git a/src/test/i18nTimezone.test.ts b/src/test/i18nTimezone.test.ts index e9fc4872..689b654c 100644 --- a/src/test/i18nTimezone.test.ts +++ b/src/test/i18nTimezone.test.ts @@ -40,10 +40,22 @@ test('every locale implements complete non-English Codex product copy', () => { I18n.setLanguage(language); const copy = I18n.t.providers.codex; assert.deepEqual(Object.keys(copy).sort(), Object.keys(CODEX_COPY_EN).sort(), language); + assert.deepEqual( + Object.keys(copy.indexingReasons).sort(), + Object.keys(CODEX_COPY_EN.indexingReasons).sort(), + `${language}.indexingReasons`, + ); if (language !== 'en') { for (const key of userFacingKeys) { assert.notEqual(copy[key], CODEX_COPY_EN[key], `${language}.${key} fell back to English`); } + for (const reason of Object.keys(CODEX_COPY_EN.indexingReasons) as Array) { + assert.notEqual( + copy.indexingReasons[reason], + CODEX_COPY_EN.indexingReasons[reason], + `${language}.indexingReasons.${reason} fell back to English`, + ); + } } } } finally { diff --git a/src/test/optimizerPreparedFlow.test.ts b/src/test/optimizerPreparedFlow.test.ts new file mode 100644 index 00000000..931e874c --- /dev/null +++ b/src/test/optimizerPreparedFlow.test.ts @@ -0,0 +1,84 @@ +import { test } from 'node:test'; +import * as assert from 'node:assert/strict'; + +import { + prepareOptimizerInvocation, + requestPreparedOptimizer, +} from '../optimizerRequest'; +import { previewAiInvocation } from '../adviceEffectiveness/preparedRequest'; + +const base = { + apiFormat: 'openai' as const, + apiUrl: 'https://example.invalid/v1', + model: 'private-model', + reasoningEffort: 'high', + systemPrompt: 'Return the exact optimizer markers.', + draft: 'Fix the cache bug without changing public APIs.', + sourceRevision: 'optimizer-draft-42', + consentGeneration: 2, + createdAtEpochMs: 1_000, +}; + +test('optimizer preview and explicit sender share the complete provider bytes', async () => { + const prepared = prepareOptimizerInvocation(base); + const preview = previewAiInvocation(prepared); + let calls = 0; + let sent: Uint8Array | undefined; + const result = await requestPreparedOptimizer(prepared, { + apiKey: 'PRIVATE_API_KEY', + expectedSourceRevision: base.sourceRevision, + expectedConsentGeneration: base.consentGeneration, + transport: async (request) => { + calls += 1; + sent = request.canonicalBytes; + return { + status: 200, + body: JSON.stringify({ + choices: [{ message: { content: + '===PROMPT===\nFix the cache bug. Keep public APIs unchanged.\n' + + '===SETTINGS===\n' + + 'Effort: high — Requires tracing state.\n' + + 'Thinking: on — Verify the invariant.\n' + + 'Model: opus — Ambiguous debugging task.' } }], + }), + }; + }, + }); + assert.equal(result.ok, true); + assert.equal(calls, 1); + assert.strictEqual(sent, prepared.canonicalBytes); + assert.equal(Buffer.from(sent!).toString('utf8'), preview.body); + assert.doesNotMatch(preview.body, /PRIVATE_API_KEY/); +}); + +test('optimizer rejects malformed model output without a marker-free fallback', async () => { + const prepared = prepareOptimizerInvocation(base); + const result = await requestPreparedOptimizer(prepared, { + apiKey: 'PRIVATE_API_KEY', + transport: async () => ({ + status: 200, + body: JSON.stringify({ choices: [{ message: { content: 'Just use a clearer prompt.' } }] }), + }), + }); + assert.deepEqual(result, { + ok: false, + code: 'invalid-output', + issues: ['optimizer output did not match the strict response contract'], + }); +}); + +test('optimizer stale consent fails before transport', async () => { + const prepared = prepareOptimizerInvocation(base); + let calls = 0; + const result = await requestPreparedOptimizer(prepared, { + apiKey: 'PRIVATE_API_KEY', + expectedConsentGeneration: 3, + transport: async () => { + calls += 1; + return { status: 200, body: '{}' }; + }, + }); + assert.equal(result.ok, false); + if (!result.ok) assert.equal(result.code, 'transport-error'); + assert.equal(calls, 0); +}); diff --git a/src/test/providerSelection.test.ts b/src/test/providerSelection.test.ts index 104d5c86..80f97a02 100644 --- a/src/test/providerSelection.test.ts +++ b/src/test/providerSelection.test.ts @@ -208,6 +208,7 @@ test('Codex tab stays visible while its first index is still running', () => { totalFiles: 1_827, indexedBytes: 1_024, totalBytes: 4_096, + reason: 'first-index', }, }, ); @@ -217,6 +218,7 @@ test('Codex tab stays visible while its first index is still running', () => { assert.match(html, /id="provider-tab-codex"/); assert.doesNotMatch(html, /id="provider-tab-compare"/); assert.match(html, /Indexing is still in progress/); + assert.match(html, /First local history setup/); assert.match(html, /Indexed log entries[^<]*1,566\/1,827 \(86%\)/); assert.doesNotMatch(html, /1\.6K\/1\.8K/); assert.match(html, /Indexed storage[^<]*1kB\/4kB/i); @@ -353,7 +355,7 @@ test('provider and Codex view copy is complete in every UI locale', () => { if (typeof value === 'string') { assert.notEqual(value.trim(), '', `${language} has empty Codex copy`); } else { - assert.ok([5, 15, 16, 17, 18].includes(Object.keys(value).length)); + assert.ok([5, 7, 15, 16, 17, 18].includes(Object.keys(value).length)); } } assert.deepEqual( diff --git a/src/test/resourceOwnership.test.ts b/src/test/resourceOwnership.test.ts new file mode 100644 index 00000000..acd2a376 --- /dev/null +++ b/src/test/resourceOwnership.test.ts @@ -0,0 +1,160 @@ +import { test } from 'node:test'; +import * as assert from 'node:assert/strict'; +import { + ResourceOwnershipRegistry, + type ResourceDescriptor, +} from '../resourceOwnership'; + +const descriptors: ResourceDescriptor[] = [ + { + kind: 'timer', + capability: 'refresh', + scope: 'extension', + creator: 'extension', + stopConditions: ['window-blur', 'feature-disabled', 'extension-dispose'], + boundedException: 'none', + }, + { + kind: 'watcher', + capability: 'codex-index', + scope: 'codex', + creator: 'extension', + stopConditions: ['window-blur', 'feature-disabled', 'settings-change', 'extension-dispose'], + boundedException: 'none', + }, + { + kind: 'worker', + capability: 'codex-index', + scope: 'codex', + creator: 'codex-index-client', + stopConditions: ['completed', 'cancelled', 'feature-disabled', 'extension-dispose', 'user-pause'], + boundedException: 'none', + }, + { + kind: 'network', + capability: 'advice-personalization', + scope: 'advice', + creator: 'advice-runtime', + stopConditions: ['settled', 'cancelled', 'feature-disabled', 'extension-dispose'], + boundedException: 'none', + }, + { + kind: 'backfill', + capability: 'codex-history', + scope: 'codex', + creator: 'refresh-coordinator', + stopConditions: ['completed', 'cancelled', 'feature-disabled', 'extension-dispose', 'user-pause'], + boundedException: 'first-codex-history', + }, +]; + +test('registry exposes fixed ownership metadata for every active resource kind', async () => { + const registry = new ResourceOwnershipRegistry(); + const leases = descriptors.map((descriptor) => registry.register(descriptor)); + const snapshot = registry.snapshotForTests(); + + assert.equal(snapshot.activeCount, 5); + assert.deepEqual(snapshot.byKind, { + timer: 1, + watcher: 1, + worker: 1, + network: 1, + backfill: 1, + }); + assert.deepEqual(snapshot.active, descriptors); + assert.equal(Object.isFrozen(snapshot), true); + assert.equal(Object.isFrozen(snapshot.byKind), true); + assert.equal(Object.isFrozen(snapshot.active), true); + assert.equal(Object.isFrozen(snapshot.active[0]), true); + assert.equal(Object.isFrozen(snapshot.active[0].stopConditions), true); + + let finishWorker!: () => void; + const workerStopped = leases[2].stop('completed', () => new Promise((resolve) => { + finishWorker = resolve; + })); + await Promise.resolve(); + assert.equal(registry.snapshotForTests().activeCount, 5, 'ownership remains active until terminate settles'); + finishWorker(); + assert.equal(await workerStopped, true); + assert.equal(registry.snapshotForTests().activeCount, 4); + + await Promise.all([ + leases[0].stop('extension-dispose', () => undefined), + leases[1].stop('extension-dispose', () => undefined), + leases[3].stop('settled', () => undefined), + leases[4].stop('completed', () => undefined), + ]); + assert.equal(registry.snapshotForTests().activeCount, 0); +}); + +test('a lease is idempotent and invokes the actual stopper once', async () => { + const registry = new ResourceOwnershipRegistry(); + const lease = registry.register(descriptors[2]); + let stopCalls = 0; + let finish!: () => void; + const stopper = (): Promise => { + stopCalls += 1; + return new Promise((resolve) => { finish = resolve; }); + }; + + const first = lease.stop('completed', stopper); + const concurrent = lease.stop('completed', stopper); + await Promise.resolve(); + assert.equal(stopCalls, 1); + assert.equal(lease.active, true); + finish(); + assert.equal(await first, true); + assert.equal(await concurrent, true); + assert.equal(lease.active, false); + assert.equal(await lease.stop('completed', stopper), false); + assert.equal(stopCalls, 1); +}); + +test('failed destruction keeps ownership active and permits an explicit retry', async () => { + const registry = new ResourceOwnershipRegistry(); + const lease = registry.register(descriptors[3]); + + await assert.rejects( + lease.stop('settled', async () => { throw new Error('abort failed'); }), + /abort failed/, + ); + assert.equal(lease.active, true); + assert.equal(registry.snapshotForTests().byKind.network, 1); + + assert.equal(await lease.stop('settled', () => undefined), true); + assert.equal(lease.active, false); + assert.equal(registry.snapshotForTests().byKind.network, 0); +}); + +test('registry rejects free-form, extra, duplicate, and invalid exception metadata', async () => { + const registry = new ResourceOwnershipRegistry(); + const base = descriptors[0]; + + assert.throws( + () => registry.register({ ...base, capability: 'read-private-log' } as unknown as ResourceDescriptor), + /capability/, + ); + assert.throws( + () => registry.register({ ...base, path: '/Users/alice/private.jsonl' } as ResourceDescriptor), + /field|metadata/i, + ); + assert.throws( + () => registry.register({ ...base, stopConditions: ['window-blur', 'window-blur'] }), + /duplicate/i, + ); + assert.throws( + () => registry.register({ ...base, boundedException: 'first-codex-history' }), + /bounded exception/i, + ); + + const lease = registry.register(base); + let stopCalls = 0; + await assert.rejects( + lease.stop('completed', () => { stopCalls += 1; }), + /stop condition/i, + ); + assert.equal(stopCalls, 0); + assert.equal(lease.active, true); + await lease.stop('extension-dispose', () => { stopCalls += 1; }); + assert.equal(stopCalls, 1); +}); diff --git a/src/test/resourceOwnershipIntegration.test.ts b/src/test/resourceOwnershipIntegration.test.ts new file mode 100644 index 00000000..4397fde3 --- /dev/null +++ b/src/test/resourceOwnershipIntegration.test.ts @@ -0,0 +1,53 @@ +import { readFileSync } from 'node:fs'; +import * as path from 'node:path'; +import { test } from 'node:test'; +import * as assert from 'node:assert/strict'; + +const root = path.resolve(__dirname, '..', '..'); + +test('production host registers timer, watcher, worker, network, and backfill ownership', () => { + const source = readFileSync(path.join(root, 'src', 'extension.ts'), 'utf8'); + for (const kind of ['timer', 'watcher', 'worker', 'network', 'backfill']) { + assert.match(source, new RegExp(`kind:\\s*'${kind}'`)); + } + for (const actualStop of [ + 'clearTimeout(timer)', + 'watcher.close()', + 'controller.abort()', + 'this.codexProvider.cancel()', + 'this.codexProvider.dispose()', + ]) { + assert.ok(source.includes(actualStop), `missing actual stop: ${actualStop}`); + } + assert.match(source, /onAiSurfaceClosed\s*=.*cancelAdviceNetworks/s); +}); + +test('background eligibility is persisted and controls historical work only', () => { + const extension = readFileSync(path.join(root, 'src', 'extension.ts'), 'utf8'); + const protocol = readFileSync( + path.join(root, 'src', 'providers', 'codex', 'codexWorkerProtocol.ts'), + 'utf8', + ); + const index = readFileSync( + path.join(root, 'src', 'providers', 'codex', 'codexIndex.ts'), + 'utf8', + ); + assert.match(extension, /ccu\.codex\.backgroundWork\.v1/); + assert.match(extension, /beginBackgroundWork/); + assert.match(extension, /recordBackgroundWorkProgress/); + assert.match(extension, /recordBackgroundWorkFailure/); + assert.match(extension, /historicalAttempt/); + assert.match(protocol, /allowHistoricalBackfill\?: boolean/); + assert.match(index, /allowHistoricalBackfill \? manifest\.files : \[\]/); +}); + +test('Codex worker is terminal after every result instead of remaining idle', () => { + const source = readFileSync( + path.join(root, 'src', 'providers', 'codex', 'codexIndexClient.ts'), + 'utf8', + ); + assert.match(source, /terminateSettledWorker/); + assert.match(source, /Promise\.resolve\(worker\.terminate\(\)\)/); + assert.match(source, /await termination/); + assert.match(source, /terminateSettledWorker\(\)[\s\S]*active\.resolve/); +}); diff --git a/src/types.ts b/src/types.ts index b3664c22..09eb1f5b 100644 --- a/src/types.ts +++ b/src/types.ts @@ -208,7 +208,7 @@ export interface ContentAnalysis { totalEstimatedTokens: number; // Recent user prompts (last 30 days), for the AI-advice feature. Each carries // its working directory so advice can be scoped to a project. - recentPrompts: { cwd: string; text: string }[]; + recentPrompts: { cwd: string; text: string; observedAtEpochMs: number }[]; /** * Numeric-only proxy separating Claude Code framework injection from text * attributed to the user. No source text, path, session ID, or tool ID is diff --git a/src/webview.ts b/src/webview.ts index 0c378226..09cba63a 100644 --- a/src/webview.ts +++ b/src/webview.ts @@ -47,7 +47,7 @@ import { getProviderNavClientScript } from './providerNavClient'; import * as os from 'os'; import * as path from 'path'; import * as https from 'https'; -import { randomBytes } from 'crypto'; +import { createHash, randomBytes } from 'crypto'; import { AdviceEffectivenessProvider, AdviceEffectivenessProviderState, @@ -55,21 +55,35 @@ import { PreparedAdviceSnapshot, prepareAdviceSnapshot, } from './adviceEffectiveness/integration'; +import type { PreparedAiInvocation } from './adviceEffectiveness/preparedRequest'; +import { previewAiInvocation } from './adviceEffectiveness/preparedRequest'; +import type { + StructuredAdviceRequestResult, +} from './adviceEffectiveness/remoteAdvice'; +import type { StructuredAdviceReferences } from './adviceEffectiveness/structuredOutput'; +import { AdviceRecommendation, createAdviceContract } from './adviceEffectiveness/contract'; +import type { PreparedOptimizerResult } from './optimizerRequest'; +import type { BackgroundWorkReason } from './backgroundWorkState'; import { AdviceLocalState, AdviceLocalStateStorage, + StoredComparablePair, + appendAdviceComparisonResult, + appendStoredComparablePair, + createClearedAdviceLocalState, createClosedAdviceLocalState, loadAndMigrateAdviceLocalState, saveAdviceLocalState, - selectStoredComparablePairLineage, - toComparableTaskPairs, upsertAdviceLocalFeedback, } from './adviceEffectiveness/versionedPersistence'; import { - AdviceComparisonResult, - DEFAULT_ADVICE_COMPARISON_POLICY, - compareAdviceEffectiveness, -} from './adviceEffectiveness/comparison'; + CODEX_COMPARISON_MEASUREMENT_PROFILE_VERSION, + CODEX_LOCAL_RECOMMENDATION_VERSION, + CodexComparableTaskProjection, + buildAppliedComparablePairs, + isComparableCodexRecommendationId, +} from './adviceEffectiveness/comparisonProduction'; +import { buildAdviceComparisonResultEnvelope } from './adviceEffectiveness/comparisonResult'; import { AttributionScope, BranchUsage, @@ -90,8 +104,11 @@ interface CodexRenderProgress { totalFiles: number; indexedBytes: number; totalBytes: number; + reason?: BackgroundWorkReason; } +const OPTIMIZER_FEEDBACK_RECOMMENDATION_ID = 'recommendation-optimizer-result-v1'; + export class UsageWebviewProvider { private panel: vscode.WebviewPanel | undefined; private currentSessionData: SessionData | null = null; @@ -154,13 +171,33 @@ export class UsageWebviewProvider { private usageLimits: ClaudeApiUsageResponse | null = null; private claudeWeeklyQuotaHistory: WeeklyQuotaObservation[] = []; private quotaWarnDismissed: boolean = false; - // Set by extension.ts: runs the Usage Optimizer round-trip (model lives there - // with the config + OAuth client). Returns the optimised prompt + settings - // recommendation, or an error string. - public onOptimize?: ( + // Extension-host hooks make the optimizer use the same prepare -> preview -> + // explicit send boundary as advice, without exposing keys to the webview. + public onPrepareOptimizerInvocation?: ( draft: string, - options: { resolve: boolean; distil: boolean; aesthetic: boolean } - ) => Promise<{ prompt?: string; settings?: string; error?: string }>; + options: { resolve: boolean; distil: boolean; aesthetic: boolean }, + sourceRevision: string, + consentGeneration: number, + ) => Promise<{ prepared?: PreparedAiInvocation; error?: string }>; + public onSendOptimizerInvocation?: ( + prepared: PreparedAiInvocation, + expectedSourceRevision: string, + expectedConsentGeneration: number, + ) => Promise; + /** Extension-host hooks keep API configuration and keys out of the webview. */ + public onPrepareAdviceInvocation?: ( + snapshot: PreparedAdviceSnapshot, + sourceRevision: string, + consentGeneration: number, + ) => PreparedAiInvocation; + public onSendAdviceInvocation?: ( + prepared: PreparedAiInvocation, + references: StructuredAdviceReferences, + expectedSourceRevision: string, + expectedConsentGeneration: number, + ) => Promise; + public onAiSurfaceClosed?: () => void; + public onAdviceDataCleared?: () => Promise; // Shared settings store + a callback to let extension.ts re-apply config when // the user edits a setting in the dashboard's ⚙ Settings tab. Both are set by // extension.ts right after construction. @@ -177,7 +214,23 @@ export class UsageWebviewProvider { prompt?: string; settings?: string; error?: string; + snapshotId?: string; + previewBody?: string; + previewSha256?: string; + previewBytes?: number; + /** Random opaque run ID; never derived from the draft or model output. */ + adviceId?: string; } | null = null; + private preparedOptimizerRequests = new Map< + string, + { + prepared: PreparedAiInvocation; + draft: string; + sourceRevision: string; + consentGeneration: number; + } + >(); + private optimizerConsentGeneration = 0; /** * Experimental v2.3.1 state. Raw prompt samples never enter rendered HTML: * they stay in the provider state until one explicit sealed-snapshot request. @@ -185,10 +238,21 @@ export class UsageWebviewProvider { private adviceEffectivenessStates: AdviceEffectivenessProviderStates = {}; private preparedAdviceSnapshots = new Map< string, - { provider: AdviceEffectivenessProvider; snapshot: PreparedAdviceSnapshot } + { + provider: AdviceEffectivenessProvider; + snapshot: PreparedAdviceSnapshot; + invocation: PreparedAiInvocation; + sourceRevision: string; + consentGeneration: number; + references: StructuredAdviceReferences; + } >(); + private adviceConsentGeneration = 0; private adviceLocalState: AdviceLocalState = createClosedAdviceLocalState(); private adviceLocalStateStatus: 'loading' | 'ready' | 'degraded' = 'loading'; + private adviceLocalStateGeneration = 0; + private adviceLocalStateWrite: Promise = Promise.resolve(); + private adviceComparisonProductionRevision = ''; constructor(private context: vscode.ExtensionContext) { const storage = this.adviceStateStorage(); @@ -199,6 +263,7 @@ export class UsageWebviewProvider { void loadAndMigrateAdviceLocalState(storage).then((loaded) => { this.adviceLocalState = loaded.value; this.adviceLocalStateStatus = loaded.ok ? 'ready' : 'degraded'; + this.scheduleAdviceComparisonProduction(); if (this.panel) { this.updateWebview(); } @@ -232,6 +297,47 @@ export class UsageWebviewProvider { return this.setting('advice.effectiveness.enabled', false); } + /** + * Serialize every persisted advice mutation through one writer. The mutation + * is derived from the latest committed state only when its turn starts, so a + * delayed consent, feedback, comparison, or clear write cannot resurrect an + * older snapshot over a newer user decision. + */ + private enqueueAdviceLocalStateWrite( + mutate: (current: AdviceLocalState) => AdviceLocalState | undefined, + ): Promise< + | { ok: true; changed: boolean; value: AdviceLocalState } + | { ok: false } + > { + const operation = this.adviceLocalStateWrite.then(async () => { + const storage = this.adviceStateStorage(); + if (!storage || this.adviceLocalStateStatus !== 'ready') { + return { ok: false as const }; + } + let next: AdviceLocalState | undefined; + try { + next = mutate(this.adviceLocalState); + } catch { + return { ok: false as const }; + } + if (!next) { + return { ok: true as const, changed: false, value: this.adviceLocalState }; + } + const saved = await saveAdviceLocalState(storage, next); + if (!saved.ok) { + this.adviceLocalStateStatus = 'degraded'; + return { ok: false as const }; + } + this.adviceLocalState = saved.value; + return { ok: true as const, changed: true, value: saved.value }; + }); + this.adviceLocalStateWrite = operation.then( + () => undefined, + () => undefined, + ); + return operation; + } + private adviceProviderState(value: unknown): AdviceEffectivenessProviderState | undefined { if (!this.adviceExperimentEnabled() || (value !== 'claude' && value !== 'codex')) { return undefined; @@ -255,11 +361,49 @@ export class UsageWebviewProvider { } } + /** Opaque content revision; no path/session/title is retained or exposed. */ + private adviceSourceRevision(state: AdviceEffectivenessProviderState): string { + const safeShape = { + provider: state.provider, + aggregate: state.aggregate, + observations: state.contract.observations.map((item) => ({ + id: item.id, + metric: item.metric, + value: item.value, + unit: item.unit, + method: item.method, + sourceId: item.sourceId, + })), + evidence: state.contract.evidence.map((item) => ({ + id: item.id, + observationIds: item.observationIds, + strength: item.strength, + })), + sourceQuality: state.contract.provenance.sources.map((source) => ({ + id: source.id, + confidence: source.confidence, + qualityFlags: source.qualityFlags, + window: source.window, + })), + // Prompt text remains host-only. Its digest merely invalidates a preview + // if the separately consented sample set changes before send. + promptSampleDigest: createHash('sha256') + .update(state.promptSamples.map((sample) => sample.text).join('\u0000'), 'utf8') + .digest('hex'), + userContextDigest: createHash('sha256') + .update(state.userContext ?? '', 'utf8') + .digest('hex'), + }; + return `advice-${createHash('sha256') + .update(JSON.stringify(safeShape), 'utf8') + .digest('hex')}`; + } + private async handleAdviceConsentMessage(message: Record): Promise { const providerState = this.adviceProviderState(message.provider); const aggregateConsent = message.aggregateConsent; const promptSampleConsent = message.promptSampleConsent; - const storage = this.adviceStateStorage(); + const stateGeneration = this.adviceLocalStateGeneration; if ( !providerState || providerState.provider !== 'claude' || @@ -268,26 +412,33 @@ export class UsageWebviewProvider { (aggregateConsent !== 'explicit' && aggregateConsent !== 'not-granted') || (promptSampleConsent !== 'explicit' && promptSampleConsent !== 'not-granted') || (promptSampleConsent === 'explicit' && aggregateConsent !== 'explicit') || - (promptSampleConsent === 'explicit' && providerState.promptSamples.length === 0) || + (promptSampleConsent === 'explicit' && + providerState.promptSamples.length === 0 && + !providerState.userContext?.trim()) || this.adviceLocalStateStatus !== 'ready' || - !storage + !this.adviceStateStorage() ) { this.postAdviceMessage({ command: 'adviceConsentResult', ok: false, provider: message.provider }); return; } - const next: AdviceLocalState = { - ...this.adviceLocalState, - featureMode: 'enabled', - aggregateConsent, - promptSampleConsent, - }; - const saved = await saveAdviceLocalState(storage, next); + const saved = await this.enqueueAdviceLocalStateWrite((current) => { + if (stateGeneration !== this.adviceLocalStateGeneration) return undefined; + return { + ...current, + featureMode: 'enabled', + aggregateConsent, + promptSampleConsent, + }; + }); if (!saved.ok) { - this.adviceLocalStateStatus = 'degraded'; this.postAdviceMessage({ command: 'adviceConsentResult', ok: false, provider: message.provider }); return; } - this.adviceLocalState = saved.value; + if (!saved.changed) { + this.postAdviceMessage({ command: 'adviceConsentResult', ok: false, provider: message.provider }); + return; + } + this.adviceConsentGeneration += 1; this.clearPreparedAdviceSnapshots(providerState.provider); this.postAdviceMessage({ command: 'adviceConsentResult', @@ -329,73 +480,279 @@ export class UsageWebviewProvider { }); return; } + let invocation: PreparedAiInvocation; + const sourceRevision = this.adviceSourceRevision(providerState); + try { + if (!this.onPrepareAdviceInvocation) { + throw new Error('AI preparation is unavailable'); + } + invocation = this.onPrepareAdviceInvocation( + result.value, + sourceRevision, + this.adviceConsentGeneration, + ); + } catch { + this.postAdviceMessage({ + command: 'adviceSnapshotResult', + ok: false, + provider: providerState.provider, + reason: 'invalid-evidence', + }); + return; + } + const preview = previewAiInvocation(invocation); this.clearPreparedAdviceSnapshots(providerState.provider); const snapshotId = `snapshot-${randomBytes(12).toString('hex')}`; this.preparedAdviceSnapshots.set(snapshotId, { provider: providerState.provider, snapshot: result.value, + invocation, + sourceRevision, + consentGeneration: this.adviceConsentGeneration, + references: { + observationIds: providerState.contract.observations.map((item) => item.id), + evidenceIds: providerState.contract.evidence.map((item) => item.id), + }, }); this.postAdviceMessage({ command: 'adviceSnapshotResult', ok: true, snapshotId, provider: providerState.provider, - contentType: result.value.preview.contentType, - dataMode: result.value.preview.dataMode, + contentType: preview.contentType, + dataMode: preview.dataMode, promptSampleCount: result.value.preview.promptSampleCount, - utf8Bytes: result.value.preview.utf8Bytes, - sha256: result.value.preview.sha256, - body: result.value.preview.body, + utf8Bytes: preview.utf8Bytes, + sha256: preview.sha256, + body: preview.body, }); } - private async handleAdviceFeedbackMessage(message: Record): Promise { + private async handleSendAdviceSnapshotMessage(message: Record): Promise { + const snapshotId = message.snapshotId; const providerState = this.adviceProviderState(message.provider); - const storage = this.adviceStateStorage(); + const stored = typeof snapshotId === 'string' + ? this.preparedAdviceSnapshots.get(snapshotId) + : undefined; + if ( + !providerState || + !stored || + stored.provider !== providerState.provider || + stored.sourceRevision !== this.adviceSourceRevision(providerState) || + stored.consentGeneration !== this.adviceConsentGeneration || + !this.onSendAdviceInvocation + ) { + this.postAdviceMessage({ + command: 'adviceSendResult', + ok: false, + provider: message.provider, + reason: 'stale-preview', + }); + return; + } + const result = await this.onSendAdviceInvocation( + stored.invocation, + stored.references, + stored.sourceRevision, + stored.consentGeneration, + ); + if ( + this.preparedAdviceSnapshots.get(snapshotId as string) !== stored || + stored.sourceRevision !== this.adviceSourceRevision(providerState) || + stored.consentGeneration !== this.adviceConsentGeneration + ) { + this.postAdviceMessage({ + command: 'adviceSendResult', + ok: false, + provider: providerState.provider, + reason: 'stale-preview', + }); + return; + } + if (!result.ok) { + this.postAdviceMessage({ + command: 'adviceSendResult', + ok: false, + provider: providerState.provider, + reason: result.code, + }); + return; + } + const next = createAdviceContract({ + adviceId: providerState.contract.adviceId, + observations: providerState.contract.observations, + evidence: providerState.contract.evidence, + recommendations: result.value.recommendations, + privacy: { + dataMode: stored.invocation.dataMode === 'aggregates-with-prompt-samples' + ? 'aggregates-with-prompt-samples' + : stored.invocation.dataMode === 'aggregates-with-personalization' + ? 'aggregates-with-personalization' + : 'aggregates-only', + promptSampleConsent: stored.invocation.dataMode === 'aggregates-only' + ? 'not-granted' + : 'explicit', + promptSampleCount: stored.snapshot.prepared.promptSampleCount, + feedbackStorage: 'local-only', + }, + provenance: { + ...providerState.contract.provenance, + generatedBy: { kind: 'remote-model' }, + generatedAt: new Date().toISOString(), + }, + }); + if (!next.ok) { + this.postAdviceMessage({ + command: 'adviceSendResult', + ok: false, + provider: providerState.provider, + reason: 'invalid-schema', + }); + return; + } + providerState.contract = next.value; + this.preparedAdviceSnapshots.delete(snapshotId as string); + this.postAdviceMessage({ + command: 'adviceSendResult', + ok: true, + provider: providerState.provider, + recommendationCount: next.value.recommendations.length, + }); + this.updateWebview(); + } + + private async handleClearAdviceLocalDataMessage(): Promise { + if ( + !this.adviceExperimentEnabled() || + this.adviceLocalStateStatus !== 'ready' || + !this.adviceStateStorage() + ) { + this.postAdviceMessage({ command: 'adviceClearResult', ok: false }); + return; + } + this.adviceLocalStateGeneration += 1; + this.adviceConsentGeneration += 1; + this.clearPreparedAdviceSnapshots(); + this.adviceComparisonProductionRevision = ''; + const write = this.enqueueAdviceLocalStateWrite(() => createClearedAdviceLocalState()); + const cancelled = Promise.resolve(this.onAdviceDataCleared?.()).catch(() => undefined); + const saved = await write; + await cancelled; + if (!saved.ok) { + this.postAdviceMessage({ command: 'adviceClearResult', ok: false }); + return; + } + this.postAdviceMessage({ command: 'adviceClearResult', ok: true }); + this.updateWebview(); + } + + private async handleAdviceFeedbackMessage(message: Record): Promise { + const optimizerTarget = + message.provider === 'optimizer' && + this.setting('advice.optimizer.enabled', false) && + typeof this.optimizerState?.adviceId === 'string' && + this.optimizerState.adviceId === message.adviceId && + message.recommendationId === OPTIMIZER_FEEDBACK_RECOMMENDATION_ID && + typeof this.optimizerState.prompt === 'string' && + this.optimizerState.prompt.length > 0; + const providerState = optimizerTarget + ? undefined + : this.adviceProviderState(message.provider); + const stateGeneration = this.adviceLocalStateGeneration; const adviceId = message.adviceId; const recommendationId = message.recommendationId; const kind = message.kind; const recommendation = providerState?.contract.recommendations.find( (candidate) => candidate.id === recommendationId, ); + const targetAdviceId = optimizerTarget + ? this.optimizerState!.adviceId as string + : providerState?.contract.adviceId; + const targetRecommendationId = optimizerTarget + ? OPTIMIZER_FEEDBACK_RECOMMENDATION_ID + : recommendation?.id; + const responseIdentity = + typeof targetAdviceId === 'string' && + typeof targetRecommendationId === 'string' && + targetAdviceId === adviceId && + targetRecommendationId === recommendationId + ? { adviceId: targetAdviceId, recommendationId: targetRecommendationId } + : {}; if ( - !providerState || - providerState.contract.adviceId !== adviceId || - !recommendation || + (!optimizerTarget && !providerState) || + targetAdviceId !== adviceId || + targetRecommendationId !== recommendationId || (kind !== 'helpful' && kind !== 'not-helpful' && kind !== 'applied') || this.adviceLocalStateStatus !== 'ready' || - !storage + !this.adviceStateStorage() ) { - this.postAdviceMessage({ command: 'adviceFeedbackResult', ok: false, provider: message.provider }); + this.postAdviceMessage({ + command: 'adviceFeedbackResult', + ok: false, + provider: message.provider, + ...responseIdentity, + }); return; } - const changed = upsertAdviceLocalFeedback(this.adviceLocalState, { - adviceId: providerState.contract.adviceId, - recommendationId: recommendation.id, - kind, - updatedAtEpochMs: Date.now(), + const updatedAtEpochMs = Date.now(); + let mutationAccepted = true; + const saved = await this.enqueueAdviceLocalStateWrite((current) => { + if (stateGeneration !== this.adviceLocalStateGeneration) return undefined; + const changed = upsertAdviceLocalFeedback(current, { + adviceId: targetAdviceId as string, + recommendationId: targetRecommendationId as string, + kind, + updatedAtEpochMs, + }); + if (!changed.ok) { + mutationAccepted = false; + return undefined; + } + const changedFeedback = changed.value.feedback.find( + (item) => item.adviceId === adviceId && item.recommendationId === recommendationId, + ); + return kind === 'applied' && changedFeedback?.applied !== 'applied' + ? { + ...changed.value, + featureMode: 'enabled' as const, + comparablePairs: changed.value.comparablePairs.filter( + (pair) => pair.adviceId !== adviceId || pair.recommendationId !== recommendationId, + ), + comparisonResults: changed.value.comparisonResults.filter( + (result) => result.recommendationId !== recommendationId, + ), + } + : { ...changed.value, featureMode: 'enabled' as const }; }); - if (!changed.ok) { - this.postAdviceMessage({ command: 'adviceFeedbackResult', ok: false, provider: providerState.provider }); + if (!saved.ok) { + this.postAdviceMessage({ + command: 'adviceFeedbackResult', + ok: false, + provider: message.provider, + ...responseIdentity, + }); return; } - const saved = await saveAdviceLocalState(storage, { - ...changed.value, - featureMode: 'enabled', - }); - if (!saved.ok) { - this.adviceLocalStateStatus = 'degraded'; - this.postAdviceMessage({ command: 'adviceFeedbackResult', ok: false, provider: providerState.provider }); + if (!saved.changed || !mutationAccepted) { + this.postAdviceMessage({ + command: 'adviceFeedbackResult', + ok: false, + provider: message.provider, + ...responseIdentity, + }); return; } - this.adviceLocalState = saved.value; + if (kind === 'applied' && !optimizerTarget) { + this.adviceComparisonProductionRevision = ''; + this.scheduleAdviceComparisonProduction(); + } const feedback = saved.value.feedback.find( (item) => item.adviceId === adviceId && item.recommendationId === recommendationId, ); this.postAdviceMessage({ command: 'adviceFeedbackResult', ok: true, - provider: providerState.provider, + provider: message.provider, adviceId, recommendationId, rating: feedback?.rating ?? 'unrated', @@ -403,9 +760,145 @@ export class UsageWebviewProvider { }); } - show(): void { + private async handlePrepareOptimizerMessage(message: Record): Promise { + if (!this.setting('advice.optimizer.enabled', false)) { + this.discardPreparedOptimizer(); + this.postAdviceMessage({ command: 'optimizePreviewResult', ok: false, error: I18n.t.popup.adviceEffectiveness.strictOutputRejected }); + return; + } + const draft = typeof message.draft === 'string' ? message.draft : ''; + const options = { + resolve: message.resolve === true, + distil: message.distil === true, + aesthetic: message.aesthetic === true, + }; + this.optimizerConsentGeneration += 1; + this.preparedOptimizerRequests.clear(); + const sourceRevision = `optimizer-${createHash('sha256') + .update(JSON.stringify({ draft, options }), 'utf8') + .digest('hex')}`; + const consentGeneration = this.optimizerConsentGeneration; + const baseState = { draft, ...options }; + if (!this.onPrepareOptimizerInvocation) { + this.optimizerState = { ...baseState, error: 'Optimizer is not available.' }; + this.postAdviceMessage({ command: 'optimizePreviewResult', ok: false, error: 'Optimizer is not available.' }); + return; + } + const result = await this.onPrepareOptimizerInvocation( + draft, + options, + sourceRevision, + consentGeneration, + ); + if (consentGeneration !== this.optimizerConsentGeneration) { + this.postAdviceMessage({ command: 'optimizePreviewResult', ok: false, error: I18n.t.popup.adviceEffectiveness.strictOutputRejected }); + return; + } + if (!result.prepared) { + this.optimizerState = { ...baseState, error: result.error ?? '' }; + this.postAdviceMessage({ command: 'optimizePreviewResult', ok: false, error: result.error ?? '' }); + return; + } + const preview = previewAiInvocation(result.prepared); + const snapshotId = `optimizer-${randomBytes(12).toString('hex')}`; + this.preparedOptimizerRequests.set(snapshotId, { + prepared: result.prepared, + draft, + sourceRevision, + consentGeneration, + }); + this.optimizerState = { + ...baseState, + snapshotId, + previewBody: preview.body, + previewSha256: preview.sha256, + previewBytes: preview.utf8Bytes, + }; + this.postAdviceMessage({ + command: 'optimizePreviewResult', + ok: true, + snapshotId, + contentType: preview.contentType, + dataMode: preview.dataMode, + body: preview.body, + sha256: preview.sha256, + utf8Bytes: preview.utf8Bytes, + }); + } + + private async handleSendOptimizerMessage(message: Record): Promise { + if (!this.setting('advice.optimizer.enabled', false)) { + this.discardPreparedOptimizer(); + this.postAdviceMessage({ command: 'optimizeResult', error: I18n.t.popup.adviceEffectiveness.strictOutputRejected }); + return; + } + const snapshotId = typeof message.snapshotId === 'string' ? message.snapshotId : ''; + const stored = this.preparedOptimizerRequests.get(snapshotId); + if ( + !stored || + !this.onSendOptimizerInvocation || + stored.consentGeneration !== this.optimizerConsentGeneration || + message.draft !== stored.draft + ) { + this.postAdviceMessage({ command: 'optimizeResult', error: I18n.t.popup.adviceEffectiveness.strictOutputRejected }); + return; + } + const result = await this.onSendOptimizerInvocation( + stored.prepared, + stored.sourceRevision, + stored.consentGeneration, + ); + if ( + this.preparedOptimizerRequests.get(snapshotId) !== stored || + stored.consentGeneration !== this.optimizerConsentGeneration + ) { + this.postAdviceMessage({ command: 'optimizeResult', error: I18n.t.popup.adviceEffectiveness.strictOutputRejected }); + return; + } + this.preparedOptimizerRequests.delete(snapshotId); + if (!result.ok) { + const error = I18n.t.popup.adviceEffectiveness.strictOutputRejected; + this.optimizerState = this.optimizerState + ? { ...this.optimizerState, error } + : null; + this.postAdviceMessage({ command: 'optimizeResult', error }); + return; + } + this.optimizerState = this.optimizerState + ? { + draft: this.optimizerState.draft, + resolve: this.optimizerState.resolve, + distil: this.optimizerState.distil, + aesthetic: this.optimizerState.aesthetic, + prompt: result.value.prompt, + settings: result.value.settings, + adviceId: `advice-optimizer-${randomBytes(12).toString('hex')}`, + } + : null; + this.postAdviceMessage({ + command: 'optimizeResult', + ...result.value, + adviceId: this.optimizerState?.adviceId, + recommendationId: OPTIMIZER_FEEDBACK_RECOMMENDATION_ID, + }); + } + + private discardPreparedOptimizer(): void { + this.optimizerConsentGeneration += 1; + this.preparedOptimizerRequests.clear(); + if (this.optimizerState) { + const { draft, resolve, distil, aesthetic } = this.optimizerState; + this.optimizerState = { draft, resolve, distil, aesthetic }; + } + } + + show(tab?: string): void { + if (tab) { + this.currentTab = tab; + } if (this.panel) { this.panel.reveal(); + this.updateWebview(); return; } @@ -417,6 +910,8 @@ export class UsageWebviewProvider { this.panel.onDidDispose(() => { this.panel = undefined; this.clearPreparedAdviceSnapshots(); + this.discardPreparedOptimizer(); + this.onAiSurfaceClosed?.(); // Force a fresh render into the next panel (the lastHtml guard must not // suppress the first paint after the panel was closed and reopened). this.lastHtml = ''; @@ -442,6 +937,9 @@ export class UsageWebviewProvider { case 'prepareAdviceSnapshot': this.handlePrepareAdviceSnapshotMessage(message as Record); break; + case 'sendAdviceSnapshot': + await this.handleSendAdviceSnapshotMessage(message as Record); + break; case 'discardAdviceSnapshot': { const provider = message.provider; if (provider === 'claude' || provider === 'codex') { @@ -452,6 +950,9 @@ export class UsageWebviewProvider { case 'recordAdviceFeedback': await this.handleAdviceFeedbackMessage(message as Record); break; + case 'clearAdviceLocalData': + await this.handleClearAdviceLocalDataMessage(); + break; case 'exportHeatmap': vscode.commands.executeCommand('claudeCodeUsage.exportHeatmap'); break; @@ -672,26 +1173,15 @@ export class UsageWebviewProvider { if (!this.panel) { break; } - const draft = String(message.draft || ''); - const opts = { - resolve: !!message.resolve, - distil: !!message.distil, - aesthetic: !!message.aesthetic, - }; - // Persist the inputs immediately so a refresh mid-request keeps them. - this.optimizerState = { draft, ...opts }; - let result: { prompt?: string; settings?: string; error?: string }; - if (this.onOptimize) { - result = await this.onOptimize(draft, opts); - } else { - result = { error: 'Optimizer is not available.' }; - } - // Persist the result too, so re-rendering the webview (auto-refresh) - // restores it instead of wiping a prompt the user is still reading. - this.optimizerState = { draft, ...opts, ...result }; - if (this.panel) { - this.panel.webview.postMessage({ command: 'optimizeResult', ...result }); - } + await this.handlePrepareOptimizerMessage(message as Record); + break; + } + case 'sendOptimizerRequest': { + await this.handleSendOptimizerMessage(message as Record); + break; + } + case 'discardOptimizerRequest': { + this.discardPreparedOptimizer(); break; } case 'getAttribution': { @@ -812,6 +1302,7 @@ export class UsageWebviewProvider { codex: providerAvailability.codex, codexData: providerAvailability.codexData ?? providerAvailability.codex, }; + this.scheduleAdviceComparisonProduction(); if (!this.providerSelectionInitialized) { this.currentProvider = defaultDashboardProvider( providerAvailability.claude, @@ -847,8 +1338,32 @@ export class UsageWebviewProvider { * it with updateProviderData, so one data refresh still produces one paint. */ updateAdviceEffectivenessData(states: AdviceEffectivenessProviderStates): void { + for (const provider of ['claude', 'codex'] as const) { + const previous = this.adviceEffectivenessStates[provider]; + const next = states[provider]; + if (!previous || !next) { + this.clearPreparedAdviceSnapshots(provider); + continue; + } + const unchanged = + this.adviceSourceRevision(previous) === this.adviceSourceRevision(next); + if (!unchanged) { + this.clearPreparedAdviceSnapshots(provider); + continue; + } + // Keep a strictly parsed remote recommendation across ordinary refreshes + // of the same materialized source revision. Evidence remains host-owned. + if (previous.contract.provenance.generatedBy.kind === 'remote-model') { + next.contract = previous.contract; + } + } this.adviceEffectivenessStates = states; + } + + invalidatePreparedAiRequests(): void { + this.adviceConsentGeneration += 1; this.clearPreparedAdviceSnapshots(); + this.discardPreparedOptimizer(); } setLoading(loading: boolean): void { @@ -1652,9 +2167,22 @@ export class UsageWebviewProvider { } private renderCodexTodayHourly(view: CodexUsageView): string { - const rows = view.todayHourly; - const coverage = view.todayCoverage; - if (rows.length === 0 && coverage.complete) { + return this.renderCodexHourlyBreakdown( + view.todayHourly, + view.todayCoverage, + ); + } + + private renderCodexHourlyBreakdown( + rows: CodexHourlyUsageView[], + coverage: { + indexedFiles: number; + totalFiles: number; + complete: boolean; + }, + day?: string, + ): string { + if (rows.length === 0 && coverage.complete && !day) { return ''; } const copy = I18n.t.providers.codex; @@ -1672,10 +2200,10 @@ export class UsageWebviewProvider { '' + ''; const chart = rows.length > 0 - ? '
    ' + + ? '
    ' + this.renderCodexHourlyChart(rows) + '
    ' + this.renderCompositionChart( - rows.map((row) => ({ label: row.hour, data: row.total })), + rows.map((row) => ({ label: row.label, data: row.total })), 'codex', ) : '
    ' + this.escapeHtml(copy.noDailyData) + '
    '; @@ -1688,7 +2216,7 @@ export class UsageWebviewProvider { '' + this.escapeHtml(copy.output) + '' + this.escapeHtml(copy.reasoning) + '' + '' + this.escapeHtml(copy.threads) + '' + rows.map((row) => - '' + this.escapeHtml(row.hour) + '' + + '' + this.escapeHtml(row.label) + '' + '' + (row.apiEquivalent.pricedTokens > 0 ? I18n.formatCurrency(row.apiEquivalent.equivalentUsd) : '—') + '' + '' + I18n.formatNumber(row.total.processed) + '' + @@ -1700,8 +2228,15 @@ export class UsageWebviewProvider { '' + I18n.formatNumber(row.threads) + '', ).join('') + '
    ' : ''; - return '
    ' + - '

    ' + this.escapeHtml(I18n.t.popup.hourlyBreakdown) + '

    ' + partial + tabs + chart + table + '
    '; + const heading = day + ? '

    ' + this.escapeHtml(this.formatDate(day)) + ' · ' + + this.escapeHtml(I18n.t.popup.hourlyBreakdown) + '

    ' + : '

    ' + this.escapeHtml(I18n.t.popup.hourlyBreakdown) + '

    '; + return '
    ' + + heading + partial + tabs + chart + table + '
    '; } /** Opt-in (showEfficiency) efficiency chips appended to a usage summary: @@ -1791,7 +2326,10 @@ export class UsageWebviewProvider { I18n.getLocale(), I18n.getTimezone(), ); - return this.escapeHtml(copy.indexedLogEntries) + ': ' + + const reason = progress.reason + ? this.escapeHtml(copy.indexingReasons[progress.reason]) + ' · ' + : ''; + return reason + this.escapeHtml(copy.indexedLogEntries) + ': ' + exactCount.format(progress.scannedFiles) + '/' + exactCount.format(progress.totalFiles) + (progress.totalFiles > 0 ? ' (' + completedPercent + '%)' : '') + ' · ' + @@ -2129,8 +2667,28 @@ export class UsageWebviewProvider { '' + this.escapeHtml(copy.fresh) + '' + this.escapeHtml(copy.input) + '' + '' + this.escapeHtml(copy.cachedInput) + '' + this.escapeHtml(copy.output) + '' + '' + this.escapeHtml(copy.reasoning) + '' + this.escapeHtml(copy.threads) + '' + - '' + rows.map((row) => - '' + this.escapeHtml(row.day) + '' + + '' + rows.map((row) => { + const hourly = view.last30DaysHourlyByDay[row.day]; + const canExpand = hourly !== undefined; + const dayCoverage = view.hourlyCoverage.days[row.day] ?? view.hourlyCoverage; + const day = this.escapeHtml(row.day); + const detailButton = canExpand + ? '' + : ''; + const detailRow = canExpand + ? '
    ' + + this.renderCodexHourlyBreakdown(hourly, dayCoverage, row.day) + + '
    ' + : ''; + return '' + day + '' + '' + (row.apiEquivalent.pricedTokens > 0 ? I18n.formatCurrency(row.apiEquivalent.equivalentUsd) : '—') + '' + '' + I18n.formatNumber(row.total.processed) + '' + @@ -2139,8 +2697,9 @@ export class UsageWebviewProvider { '' + I18n.formatNumber(row.total.cachedInput) + '' + '' + I18n.formatNumber(row.total.output) + '' + '' + I18n.formatNumber(row.total.reasoning) + '' + - '' + I18n.formatNumber(row.threads) + '', - ).join('') + ''; + '' + I18n.formatNumber(row.threads) + '' + + '' + detailButton + '' + detailRow; + }).join('') + ''; return this.renderUsageData(null, provider, view.last30Days) + breakdown; } if (!this.monthData) { @@ -3813,25 +4372,372 @@ export class UsageWebviewProvider { ); } + private codexComparisonModelFamily(model: string): 'opus' | 'sonnet' | 'haiku' | 'fable' | 'other' { + const value = model.toLowerCase(); + if (value.includes('opus')) return 'opus'; + if (value.includes('sonnet')) return 'sonnet'; + if (value.includes('haiku')) return 'haiku'; + if (value.includes('fable')) return 'fable'; + return 'other'; + } + + private codexComparisonEffort( + effort: string, + ): 'low' | 'medium' | 'high' | 'xhigh' | 'max' | 'ultra' | 'unknown' { + const value = effort.toLowerCase(); + return value === 'low' || value === 'medium' || value === 'high' || value === 'xhigh' || + value === 'max' || value === 'ultra' + ? value + : 'unknown'; + } + + /** + * Project the existing bounded Codex view into numeric task aggregates. The + * rootTaskViewKey is used transiently only to join already-materialized child + * rows; it has no destination in the returned or persisted representation. + */ + private codexComparableTaskProjections(): CodexComparableTaskProjection[] { + const view = this.codexView; + if (!view) return []; + interface Group { + hasRoot: boolean; + observedAtEpochMs: number; + processed: number; + fresh: number; + childFresh: number; + approvalReviewerFresh: number; + patchCalls: number; + toolCalls: number; + postPatchToolCalls: number; + taskCompleteCount: number; + modelFamilies: Set<'opus' | 'sonnet' | 'haiku' | 'fable' | 'other'>; + efforts: Set<'low' | 'medium' | 'high' | 'xhigh' | 'max' | 'ultra' | 'unknown'>; + } + const groups = new Map(); + for (const thread of view.recentThreads.slice(0, 1_000)) { + const current = groups.get(thread.rootTaskViewKey) ?? { + hasRoot: false, + observedAtEpochMs: 0, + processed: 0, + fresh: 0, + childFresh: 0, + approvalReviewerFresh: 0, + patchCalls: 0, + toolCalls: 0, + postPatchToolCalls: 0, + taskCompleteCount: 0, + modelFamilies: new Set<'opus' | 'sonnet' | 'haiku' | 'fable' | 'other'>(), + efforts: new Set<'low' | 'medium' | 'high' | 'xhigh' | 'max' | 'ultra' | 'unknown'>(), + }; + current.hasRoot ||= thread.role === 'root'; + current.observedAtEpochMs = Math.max(current.observedAtEpochMs, thread.observedAt); + current.processed += Math.max(0, thread.total.processed); + current.fresh += Math.max(0, thread.total.fresh); + if (thread.role === 'subagent') current.childFresh += Math.max(0, thread.total.fresh); + if (thread.role === 'approval-reviewer') { + current.approvalReviewerFresh += Math.max(0, thread.total.fresh); + } + current.patchCalls += Math.max(0, thread.structural.patchCalls); + current.toolCalls += Math.max(0, thread.structural.toolCalls); + current.postPatchToolCalls += Math.max(0, thread.structural.postPatchToolCalls); + current.taskCompleteCount += Math.max(0, thread.structural.taskCompleteCount); + for (const model of thread.models) { + current.modelFamilies.add(this.codexComparisonModelFamily(model)); + } + for (const effort of thread.efforts) { + current.efforts.add(this.codexComparisonEffort(effort)); + } + groups.set(thread.rootTaskViewKey, current); + } + const qualityFlags = view.qualityFlags.map((item) => item.flag); + return [...groups.values()] + .filter((group) => group.hasRoot) + .map((group) => ({ + observedAtEpochMs: group.observedAtEpochMs, + total: { processed: group.processed, fresh: group.fresh }, + structural: { + patchCalls: group.patchCalls, + toolCalls: group.toolCalls, + postPatchToolCalls: group.postPatchToolCalls, + taskCompleteCount: group.taskCompleteCount, + }, + childFreshShare: group.fresh > 0 ? group.childFresh / group.fresh : 0, + approvalReviewerFreshShare: + group.fresh > 0 ? group.approvalReviewerFresh / group.fresh : 0, + modelFamilies: [...group.modelFamilies].sort(), + efforts: [...group.efforts].sort(), + coverage: { + complete: view.coverage.complete, + identityComplete: view.coverage.identity.complete, + qualityFlags: [...qualityFlags], + }, + })) + .sort((left, right) => left.observedAtEpochMs - right.observedAtEpochMs); + } + + private adviceComparablePairSampleKey(pair: StoredComparablePair): string { + return JSON.stringify({ + recommendationId: pair.recommendationId, + recommendationVersion: pair.recommendationVersion, + context: pair.context, + metric: pair.metric, + quality: pair.quality, + evidence: pair.evidence, + recordedAtEpochMs: pair.recordedAtEpochMs, + }); + } + + private adviceComparablePairCohortKey(pair: StoredComparablePair): string { + return JSON.stringify({ + context: pair.context, + metric: { + name: pair.metric.name, + unit: pair.metric.unit, + direction: pair.metric.direction, + }, + }); + } + + /** Pure state projection used by the production writer and focused tests. */ + private materializeAdviceComparisonState( + _now: number = Date.now(), + baseState: AdviceLocalState = this.adviceLocalState, + ): AdviceLocalState { + const providerState = this.adviceEffectivenessStates.codex; + if ( + !this.adviceExperimentEnabled() || + !providerState || + !this.codexView || + this.adviceLocalStateStatus !== 'ready' + ) { + return baseState; + } + const tasks = this.codexComparableTaskProjections(); + let next = baseState; + for (const recommendation of providerState.contract.recommendations) { + if (!isComparableCodexRecommendationId(recommendation.id)) continue; + const applied = [...next.feedback] + .filter( + (item) => + item.applied === 'applied' && + item.recommendationId === recommendation.id && + item.adviceId.startsWith('advice-codex-'), + ) + .sort( + (left, right) => + (right.appliedAtEpochMs ?? right.updatedAtEpochMs) - + (left.appliedAtEpochMs ?? left.updatedAtEpochMs) || + right.adviceId.localeCompare(left.adviceId), + )[0]; + if (!applied) continue; + const built = buildAppliedComparablePairs({ + adviceId: applied.adviceId, + recommendationId: recommendation.id, + recommendationVersion: CODEX_LOCAL_RECOMMENDATION_VERSION, + appliedAtEpochMs: applied.appliedAtEpochMs ?? applied.updatedAtEpochMs, + tasks, + }); + if (!built.ok) continue; + const existingSamples = new Set( + next.comparablePairs.map((pair) => this.adviceComparablePairSampleKey(pair)), + ); + for (const pair of built.pairs) { + const sampleKey = this.adviceComparablePairSampleKey(pair); + if (existingSamples.has(sampleKey)) continue; + const appended = appendStoredComparablePair(next, pair); + if (!appended.ok) continue; + next = appended.value; + existingSamples.add(sampleKey); + } + + const criterion = recommendation.successCriteria.find( + (item) => + item.target.kind === 'relative-change' && + item.direction === 'decrease' && + item.qualityGuardrail.rubricId === 'task-quality-rubric-v1', + ); + if (!criterion) continue; + const lineage = next.comparablePairs.filter( + (pair) => + pair.context.provider === 'codex' && + pair.recommendationId === recommendation.id && + pair.recommendationVersion === CODEX_LOCAL_RECOMMENDATION_VERSION && + pair.context.measurementProfileVersion === CODEX_COMPARISON_MEASUREMENT_PROFILE_VERSION, + ); + const cohorts = new Map(); + for (const pair of lineage) { + const key = this.adviceComparablePairCohortKey(pair); + const cohort = cohorts.get(key) ?? []; + cohort.push(pair); + cohorts.set(key, cohort); + } + for (const pairs of cohorts.values()) { + const first = pairs[0]; + const frozen = buildAdviceComparisonResultEnvelope({ + provider: 'codex', + recommendationId: recommendation.id, + recommendationVersion: CODEX_LOCAL_RECOMMENDATION_VERSION, + measurementProfileVersion: CODEX_COMPARISON_MEASUREMENT_PROFILE_VERSION, + cohort: { + scope: first.context.scope, + taskKind: first.context.taskKind, + complexityBand: first.context.complexityBand, + modelFamily: first.context.modelFamily, + effort: first.context.effort, + metricDefinitionVersion: first.context.metricDefinitionVersion, + qualityRubricId: first.context.qualityRubricId, + metric: { + name: first.metric.name, + unit: first.metric.unit, + direction: first.metric.direction, + }, + }, + guardrail: { + minComparablePairs: criterion.minimumComparableTasks, + minRelativeImprovement: criterion.target.value, + minAfterQualityScore: criterion.qualityGuardrail.minimumScore, + maxMeanQualityRegression: criterion.qualityGuardrail.maximumRegression, + allowedQualityFlags: [], + qualityRubricId: criterion.qualityGuardrail.rubricId, + }, + pairs, + recordedAtEpochMs: Math.max(...pairs.map((pair) => pair.recordedAtEpochMs)), + }); + if ( + !frozen.ok || + next.comparisonResults.some( + (result) => result.comparisonId === frozen.value.comparisonId, + ) + ) { + continue; + } + const appended = appendAdviceComparisonResult(next, frozen.value); + if (appended.ok) next = appended.value; + } + } + return next; + } + + private currentAdviceComparisonProductionRevision(): string { + const view = this.codexView; + const safe = { + enabled: this.adviceExperimentEnabled(), + stateStatus: this.adviceLocalStateStatus, + applied: this.adviceLocalState.feedback + .filter((item) => item.applied === 'applied') + .map((item) => ({ + adviceId: item.adviceId, + recommendationId: item.recommendationId, + appliedAtEpochMs: item.appliedAtEpochMs ?? item.updatedAtEpochMs, + })), + comparablePairCount: this.adviceLocalState.comparablePairs.length, + comparisonResultCount: this.adviceLocalState.comparisonResults.length, + codex: view + ? { + indexedFiles: view.coverage.indexedFiles, + indexedBytes: view.coverage.indexedBytes, + complete: view.coverage.complete, + identityComplete: view.coverage.identity.complete, + totalThreadCount: view.totalThreadCount, + lastActiveAt: view.lastTaskIdentity?.lastActiveAt ?? 0, + lastTaskTotal: view.lastTask?.total, + lastTaskStructural: view.lastTask?.structural, + qualityFlags: view.qualityFlags, + } + : null, + }; + return createHash('sha256').update(JSON.stringify(safe), 'utf8').digest('hex'); + } + + private scheduleAdviceComparisonProduction(): void { + if ( + !this.adviceExperimentEnabled() || + this.adviceLocalStateStatus !== 'ready' || + !this.codexView || + !this.adviceStateStorage() + ) { + return; + } + const revision = this.currentAdviceComparisonProductionRevision(); + if (revision === this.adviceComparisonProductionRevision) return; + this.adviceComparisonProductionRevision = revision; + void this.enqueueAdviceLocalStateWrite((current) => { + const next = this.materializeAdviceComparisonState(Date.now(), current); + if ( + next.comparablePairs.length === current.comparablePairs.length && + next.comparisonResults.length === current.comparisonResults.length + ) { + return undefined; + } + return next; + }) + .then((saved) => { + if (!saved.ok) { + if (this.adviceComparisonProductionRevision === revision) { + this.adviceComparisonProductionRevision = ''; + } + return; + } + this.adviceComparisonProductionRevision = + this.currentAdviceComparisonProductionRevision(); + if (saved.changed && this.panel) this.updateWebview(); + }) + .catch(() => { + if (this.adviceComparisonProductionRevision === revision) { + this.adviceComparisonProductionRevision = ''; + } + }); + } + private adviceComparison( state: AdviceEffectivenessProviderState, recommendationId: string | undefined, - ): AdviceComparisonResult { + ): { + status: + | 'evidence-insufficient' + | 'quality-guardrail-failed' + | 'improved' + | 'no-demonstrated-improvement'; + comparablePairs: number; + minimumComparablePairs: number; + } { + const minimumComparablePairs = state.contract.recommendations + .find((item) => item.id === recommendationId) + ?.successCriteria[0]?.minimumComparableTasks ?? 5; if (!recommendationId) { + return { status: 'evidence-insufficient', comparablePairs: 0, minimumComparablePairs }; + } + const recommendationVersion = state.provider === 'codex' && + isComparableCodexRecommendationId(recommendationId) + ? CODEX_LOCAL_RECOMMENDATION_VERSION + : undefined; + const frozen = this.adviceLocalState.comparisonResults + .filter( + (item) => + item.provider === state.provider && + item.recommendationId === recommendationId && + (!recommendationVersion || item.recommendationVersion === recommendationVersion), + ) + .sort( + (left, right) => + right.sample.pairCount - left.sample.pairCount || + right.recordedAtEpochMs - left.recordedAtEpochMs || + right.comparisonId.localeCompare(left.comparisonId), + )[0]; + if (frozen) { return { - status: 'insufficient-evidence', - reasons: ['no evidence-backed recommendation'], - comparablePairs: 0, + status: frozen.result.status, + comparablePairs: frozen.sample.pairCount, + minimumComparablePairs: frozen.guardrail.minComparablePairs, }; } - const stored = selectStoredComparablePairLineage( - this.adviceLocalState.comparablePairs, - { provider: state.provider, recommendationId }, - ); - return compareAdviceEffectiveness( - toComparableTaskPairs(stored), - DEFAULT_ADVICE_COMPARISON_POLICY, - ); + const comparablePairs = this.adviceLocalState.comparablePairs.filter( + (pair) => + pair.context.provider === state.provider && + pair.recommendationId === recommendationId && + (!recommendationVersion || pair.recommendationVersion === recommendationVersion), + ).length; + return { status: 'evidence-insufficient', comparablePairs, minimumComparablePairs }; } /** @@ -3851,10 +4757,8 @@ export class UsageWebviewProvider { const percent = (value: number): string => `${Math.round(Math.max(0, Math.min(1, value)) * 100)}%`; const contract = state?.contract; - const recommendation = contract?.recommendations[0]; + const recommendations = contract?.recommendations ?? []; const observations: string[] = []; - let actionText = t.noEvidenceAdvice; - let recommendationText = t.noEvidenceAdvice; let limitationText = t.qualityGuardrailPending; if (state && provider === 'claude') { @@ -3872,19 +4776,15 @@ export class UsageWebviewProvider { observations.push(replace(t.frameworkOverheadSignal, 'share', percent(framework.value))); } limitationText = `${t.elapsedTimeProxy} ${t.qualityGuardrailPending}`; - if (recommendation) { - recommendationText = t.clearBoundaryRecommendation; - actionText = t.clearBoundaryAction; - } } else if (state && provider === 'codex') { const copy = I18n.t.providers.codex; - const kind = recommendation?.id.replace('recommendation-codex-', '') as - | keyof typeof copy.insightObservations - | undefined; - if (kind && Object.prototype.hasOwnProperty.call(copy.insightObservations, kind)) { - observations.push(copy.insightObservations[kind]); - recommendationText = t.codexLocalRecommendation; - actionText = copy.insightTips[kind]; + for (const recommendation of recommendations) { + const kind = recommendation.id.replace('recommendation-codex-', '') as + keyof typeof copy.insightObservations; + if (Object.prototype.hasOwnProperty.call(copy.insightObservations, kind)) { + const observation = copy.insightObservations[kind]; + if (!observations.includes(observation)) observations.push(observation); + } } limitationText = `${copy.structuralProxy} ${t.qualityGuardrailPending}`; } @@ -3892,36 +4792,15 @@ export class UsageWebviewProvider { observations.push(t.noEvidenceAdvice); } - const comparison = state - ? this.adviceComparison(state, recommendation?.id) - : { - status: 'insufficient-evidence' as const, - reasons: ['provider contract unavailable'], - comparablePairs: 0, - }; - const pairCount = comparison.status === 'insufficient-evidence' - ? comparison.comparablePairs - : comparison.stats.comparablePairs; - const resultText = comparison.status === 'quality-guardrail-failed' - ? t.qualityGuardrailFailed - : comparison.status === 'improved' - ? t.improved - : comparison.status === 'no-demonstrated-improvement' - ? t.noDemonstratedImprovement - : t.insufficientEvidence; - const pairText = replace(t.comparablePairs, 'count', String(pairCount)); - const minimumText = replace( - t.minimumComparablePairs, - 'minimum', - String(DEFAULT_ADVICE_COMPARISON_POLICY.minComparablePairs), - ); const evidenceText = state ? `${t.source}: ${provider === 'claude' ? 'Claude' : 'Codex'} · ${t.evidence}: ${state.contract.evidence.length}` : t.noEvidenceAdvice; const bodyId = `advice-effectiveness-${provider}`; - - let feedbackHtml = ''; - if (state && recommendation) { + const step = (index: number, label: string, content: string): string => + '
  • ' + + '
    ' + html(label) + '

    ' + content + '

  • '; + const feedbackFor = (recommendation: AdviceRecommendation): string => { + if (!state) return ''; const feedback = this.adviceLocalState.feedback.find( (item) => item.adviceId === state.contract.adviceId && @@ -3939,7 +4818,7 @@ export class UsageWebviewProvider { ' data-recommendation-id="' + html(recommendation.id) + '"' + ' data-feedback-kind="' + kind + '" aria-pressed="' + String(selected) + '"' + disabled + '>' + ' ' + html(label) + ''; - feedbackHtml = + return ( '
    ' + '
    ' + html(t.feedbackTitle) + '
    ' + '
    ' + @@ -3948,15 +4827,74 @@ export class UsageWebviewProvider { button('applied', t.applied, feedback?.applied === 'applied') + '
    ' + '

    ' + html(t.feedbackLocalOnly) + '

    ' + - '

    ' + - '
    '; - } + '

    ' + + '' + ); + }; + const recommendationCopy = ( + recommendation: AdviceRecommendation, + ): { recommendation: string; action: string } => { + if (provider === 'claude') { + return { recommendation: t.clearBoundaryRecommendation, action: t.clearBoundaryAction }; + } + const copy = I18n.t.providers.codex; + const kind = recommendation.id.replace('recommendation-codex-', '') as keyof typeof copy.insightTips; + return { + recommendation: t.codexLocalRecommendation, + action: Object.prototype.hasOwnProperty.call(copy.insightTips, kind) + ? copy.insightTips[kind] + : t.noEvidenceAdvice, + }; + }; + const recommendationHtml = state && recommendations.length > 0 + ? recommendations.map((recommendation) => { + const copy = recommendationCopy(recommendation); + const comparison = this.adviceComparison(state, recommendation.id); + const resultText = comparison.status === 'quality-guardrail-failed' + ? t.qualityGuardrailFailed + : comparison.status === 'improved' + ? t.improved + : comparison.status === 'no-demonstrated-improvement' + ? t.noDemonstratedImprovement + : t.insufficientEvidence; + const pairText = replace( + t.comparablePairs, + 'count', + String(comparison.comparablePairs), + ); + const minimumText = replace( + t.minimumComparablePairs, + 'minimum', + String(comparison.minimumComparablePairs), + ); + return ( + '
    ' + + '
      ' + + step(3, t.recommendation, html(copy.recommendation)) + + step(4, t.action, html(copy.action)) + + step( + 5, + t.result, + html(resultText) + '
      ' + + html(pairText + ' · ' + minimumText) + '', + ) + + '
    ' + feedbackFor(recommendation) + '
    ' + ); + }).join('') + : '
      ' + + step(3, t.recommendation, html(t.noEvidenceAdvice)) + + step(4, t.action, html(t.noEvidenceAdvice)) + + step(5, t.result, html(t.insufficientEvidence)) + + '
    '; let payloadHtml = '

    ' + html(t.codexPreviewUnavailable) + '

    '; if (state?.provider === 'claude' && state.remotePreviewEligible && state.aggregate) { const ready = this.adviceLocalStateStatus === 'ready'; const aggregateChecked = ready && this.adviceLocalState.aggregateConsent === 'explicit'; - const promptAvailable = state.promptSamples.length > 0; + const promptAvailable = state.promptSamples.length > 0 || Boolean(state.userContext?.trim()); const promptChecked = aggregateChecked && promptAvailable && this.adviceLocalState.promptSampleConsent === 'explicit'; const aggregateDisabled = ready ? '' : ' disabled'; @@ -3974,7 +4912,12 @@ export class UsageWebviewProvider { '' + + html(t.promptConsentLabel) + '' + + html(t.promptConsentHelp + ' ' + replace( + t.promptWindowHelp, + 'days', + String(state.aggregate.windowDays), + )) + '' + '' + '
    ' + '
    ' +
    +        '
    ' + '
    '; } - const step = (index: number, label: string, content: string): string => - '
  • ' + - '
    ' + html(label) + '

    ' + content + '

  • '; return ( '
    ' + '

    ' + html(t.title) + @@ -4000,11 +4943,11 @@ export class UsageWebviewProvider { '
      ' + step(1, t.observation, observations.map((item) => html(item)).join('
      ')) + step(2, t.evidence, html(evidenceText) + '
      ' + html(limitationText) + '') + - step(3, t.recommendation, html(recommendationText)) + - step(4, t.action, html(actionText)) + - step(5, t.result, html(resultText) + '
      ' + html(pairText + ' · ' + minimumText) + '') + '
    ' + - feedbackHtml + payloadHtml + + recommendationHtml + payloadHtml + + '
    ' + '

    ' ); } @@ -4034,12 +4977,13 @@ export class UsageWebviewProvider { /** * Usage Optimizer card (Phase 9c). Default OFF — until the user opts in via * settings it shows only a description + an "enable" button. When enabled it - * exposes a textarea + toggles; the round-trip runs in extension.ts via the - * onOptimize hook (consent modal lives there). The result skeleton is baked + * exposes a textarea + toggles; preparation and explicit sending run through + * the same sealed-request hooks as advice. The result skeleton is baked * in (hidden) so the webview JS only fills text — no labels passed to JS. */ private renderOptimizerCard(): string { const t = I18n.t.popup; + const ai = t.adviceEffectiveness; const enabled = this.setting('advice.optimizer.enabled', false); // Shared header: icon badge + title (with an "experimental" pill) + purpose. const head = (action: string): string => @@ -4055,7 +4999,7 @@ export class UsageWebviewProvider { if (!enabled) { return ( - '
    ' + + '
    ' + head( ''; + const optimizerFeedbackHtml = + '
    ' + + '
    ' + this.escapeHtml(ai.feedbackTitle) + '
    ' + + '
    ' + + feedbackButton('helpful', ai.helpful, optimizerFeedback?.rating === 'helpful') + + feedbackButton('not-helpful', ai.notHelpful, optimizerFeedback?.rating === 'not-helpful') + + feedbackButton('applied', ai.applied, optimizerFeedback?.applied === 'applied') + + '

    ' + this.escapeHtml(ai.feedbackLocalOnly) + '

    ' + + '

    '; const lens = (id: string, label: string, hint: string, on?: boolean): string => ''; return ( - '
    ' + + '
    ' + head('') + '

    ' + t.optimizerHowto + '

    ' + '' + @@ -5238,36 +5278,8 @@ export class UsageWebviewProvider { '
    ' + '
    ' + (hasErr ? this.escapeHtml(st!.error as string) : '') + '
    ' + - '
    ' + - '' + this.escapeHtml(ai.payloadTitle) + - 'SHA-256' + - '
    ' + - (hasPreview ? this.escapeHtml(ai.payloadBytes.replace('{bytes}', String(st!.previewBytes))) : '') + - '' + - (hasPreview ? this.escapeHtml(`SHA-256 ${st!.previewSha256}`) : '') + '
    ' + - '
    ' +
    -      (hasPreview ? this.escapeHtml(st!.previewBody as string) : '') + '
    ' + - '
    ' + this.escapeHtml(ai.noNetworkTransport) + '
    ' + - '
    ' + - '
    ' + - '

    ' + t.optimizerPromptHeading + '

    ' + - '
    ' +
    -      (st && st.prompt ? this.escapeHtml(st.prompt) : '') + '
    ' + - '
    ' + - '

    ' + t.optimizerSettingsHeading + '

    ' + - '
    ' + - (st && st.settings ? this.escapeHtml(st.settings) : '') + '
    ' + - optimizerFeedbackHtml + '
    ' + + optimizerPreviewHtml + + optimizerResultHtml + '
    ' ); } From 3a0cd4b2428f728ae5cb20921ca5e30075088175 Mon Sep 17 00:00:00 2001 From: Carl <74130028+Carl723000@users.noreply.github.com> Date: Sun, 30 Aug 2026 23:56:51 +0800 Subject: [PATCH 15/72] fix: preserve mixed snooze migrations Co-authored-by: OpenAI Codex <215057067+openai-codex[bot]@users.noreply.github.com> --- .../versionedPersistence.ts | 4 ++- src/test/advicePersistence.test.ts | 28 +++++++++++++++++++ 2 files changed, 31 insertions(+), 1 deletion(-) diff --git a/src/adviceEffectiveness/versionedPersistence.ts b/src/adviceEffectiveness/versionedPersistence.ts index 08372cb9..ac627b96 100644 --- a/src/adviceEffectiveness/versionedPersistence.ts +++ b/src/adviceEffectiveness/versionedPersistence.ts @@ -340,7 +340,9 @@ function migrateLegacyV3(value: Record): AdviceLocalState | und const suppression: PersistedAdviceSuppression[] = []; const targets = new Set(); for (const raw of value.suppression) { - const migrated = migrateLegacySuppression(raw); + // Mixed v3 writes can contain both the new stable shape and an older + // adviceId record; preserve only records whose provider scope is explicit. + const migrated = parseSuppression(raw) ?? migrateLegacySuppression(raw); if (!migrated) continue; const target = `${migrated.provider}\0${migrated.surface}\0${migrated.recommendationId}`; if (targets.has(target)) continue; diff --git a/src/test/advicePersistence.test.ts b/src/test/advicePersistence.test.ts index 1564d210..67e0acea 100644 --- a/src/test/advicePersistence.test.ts +++ b/src/test/advicePersistence.test.ts @@ -171,6 +171,34 @@ test('v3 suppression migration maps only explicit provider prefixes and drops am }]); }); +test('v3 migration preserves stable suppression records in a mixed-generation write', async () => { + const storage = new MemoryStorage(); + storage.value = { + ...createClosedAdviceLocalState(), + suppression: [ + { + provider: 'optimizer', + surface: 'optimizer', + recommendationId: 'recommendation-optimizer-result-v1', + snoozedUntilEpochMs: 1_777_000_100_000, + updatedAtEpochMs: 1_777_000_000_000, + }, + { + adviceId: 'advice-claude-20260830', + recommendationId: 'recommendation-2', + snoozedUntilEpochMs: 1_777_000_100_100, + updatedAtEpochMs: 1_777_000_000_000, + }, + ], + }; + const result = await loadAndMigrateAdviceLocalState(storage); + assert.equal(result.ok, true); + if (!result.ok) return; + assert.equal(result.value.suppression.length, 2); + assert.equal(result.value.suppression[0]?.provider, 'optimizer'); + assert.equal(result.value.suppression[1]?.provider, 'claude'); +}); + test('unknown, corrupt, or identifying local data fails closed and is never overwritten', async () => { for (const value of [ { schemaVersion: 99, prompt: 'PRIVATE_PROMPT' }, From e79f510c7f00d6acde4ee9f7e12685bbbbb5e5cc Mon Sep 17 00:00:00 2001 From: Carl <74130028+Carl723000@users.noreply.github.com> Date: Mon, 31 Aug 2026 12:03:33 +0800 Subject: [PATCH 16/72] fix: show Codex weekly quota remaining after reset --- src/codexStatus.ts | 69 ++++++++++++++++++++++++++---- src/statusBar.ts | 38 +++++++++++++++-- src/test/codexStatus.test.ts | 67 +++++++++++++++++++++++------ src/test/weeklyValue.test.ts | 81 ++++++++++++++++++++++++++++++++++++ src/weeklyValue.ts | 45 +++++++++++++++++++- 5 files changed, 277 insertions(+), 23 deletions(-) diff --git a/src/codexStatus.ts b/src/codexStatus.ts index dd5cc1c9..17c59b4b 100644 --- a/src/codexStatus.ts +++ b/src/codexStatus.ts @@ -3,12 +3,30 @@ import { ProviderLimitSnapshot } from './providers/providerTypes'; export type CodexStatusMetric = 'fresh' | 'processed' | 'output'; +export interface CodexStatusOptions { + /** Keep the existing opt-in preference meaningful for the Codex bar. */ + quotaFiveHourOnly?: boolean; +} + +export interface CodexStatusLimit { + label: string; + windowMinutes?: number; + usedPercent: number; + remainingPercent: number; + observedAt?: number; + resetsAt?: number; +} + export interface CodexStatusText { text: string; limitText?: string; + limit?: CodexStatusLimit; stale: boolean; } +const FIVE_HOUR_MINUTES = 5 * 60; +const WEEK_MINUTES = 7 * 24 * 60; + function compact(value: number): string { const safe = Math.max(0, value); if (safe >= 1_000_000_000) { @@ -24,6 +42,12 @@ function compact(value: number): string { } function windowLabel(minutes: number | undefined, fallback: string | undefined): string { + if (minutes === FIVE_HOUR_MINUTES) { + return '5h'; + } + if (minutes === WEEK_MINUTES) { + return 'wk'; + } if (minutes !== undefined && minutes > 0) { if (minutes % (24 * 60) === 0) { return `${minutes / (24 * 60)}d`; @@ -36,11 +60,30 @@ function windowLabel(minutes: number | undefined, fallback: string | undefined): return fallback && fallback !== 'primary' ? fallback : 'limit'; } +function clampPercent(value: number): number { + return Math.max(0, Math.min(100, Number.isFinite(value) ? value : 0)); +} + +function selectQuotaWindow( + windows: ProviderLimitSnapshot['windows'], + options: CodexStatusOptions, +): ProviderLimitSnapshot['windows'][number] | undefined { + if (options.quotaFiveHourOnly) { + return windows.find((window) => window.windowMinutes === FIVE_HOUR_MINUTES); + } + // The weekly allowance is the useful default after a reset: it answers how + // much room remains for the week, rather than how much has already gone. + return windows.find((window) => window.windowMinutes === WEEK_MINUTES) + ?? windows.find((window) => window.windowMinutes === FIVE_HOUR_MINUTES) + ?? windows[0]; +} + export function formatCodexStatus( scope: CodexUsageScopeView, metric: CodexStatusMetric, limit: ProviderLimitSnapshot | null, now: number = Date.now(), + options: CodexStatusOptions = {}, ): CodexStatusText { const value = metric === 'processed' @@ -48,17 +91,29 @@ export function formatCodexStatus( : metric === 'output' ? scope.total.output : scope.total.fresh; - const liveWindow = limit?.windows.find( + const liveWindows = limit?.windows.filter( (window) => window.resetsAt === undefined || window.resetsAt > now, - ); - const limitText = liveWindow - ? `${windowLabel(liveWindow.windowMinutes, liveWindow.label)} ${Math.round( - Math.max(0, Math.min(100, liveWindow.usedPercent)), - )}%` + ) ?? []; + const liveWindow = selectQuotaWindow(liveWindows, options); + const selectedLimit = liveWindow + ? { + label: windowLabel(liveWindow.windowMinutes, liveWindow.label), + ...(liveWindow.windowMinutes !== undefined + ? { windowMinutes: liveWindow.windowMinutes } + : {}), + usedPercent: clampPercent(liveWindow.usedPercent), + remainingPercent: 100 - clampPercent(liveWindow.usedPercent), + ...(limit?.observedAt !== undefined ? { observedAt: limit.observedAt } : {}), + ...(liveWindow.resetsAt !== undefined ? { resetsAt: liveWindow.resetsAt } : {}), + } + : undefined; + const limitText = selectedLimit + ? `${selectedLimit.label} ${Math.round(selectedLimit.remainingPercent)}%` : undefined; return { text: `CX ${compact(value)}${scope.indexedSubtotal ? '*' : ''}`, limitText, - stale: Boolean(limit && !liveWindow), + ...(selectedLimit ? { limit: selectedLimit } : {}), + stale: Boolean(limit && liveWindows.length === 0), }; } diff --git a/src/statusBar.ts b/src/statusBar.ts index bb99fa0a..f3e9b477 100644 --- a/src/statusBar.ts +++ b/src/statusBar.ts @@ -369,7 +369,13 @@ export class StatusBarManager { metric: CodexStatusMetric, limit: ProviderLimitSnapshot | null, ): void { - const formatted = formatCodexStatus(scope, metric, limit); + const formatted = formatCodexStatus( + scope, + metric, + limit, + Date.now(), + { quotaFiveHourOnly: this.quotaFiveHourOnly }, + ); const copy = I18n.t.providers.codex; this.isLoading = false; this.statusBarItem.text = formatted.text; @@ -388,9 +394,35 @@ export class StatusBarManager { this.statusBarItem.tooltip = md; this.applyCostVisibility(); - if (formatted.limitText) { + if (formatted.limitText && formatted.limit) { this.quotaItem.text = `$(dashboard) ${formatted.limitText}`; - this.quotaItem.tooltip = `${copy.accountSnapshotLastObserved} — ${formatted.limitText}`; + const windowName = formatted.limit.windowMinutes === 7 * 24 * 60 + ? copy.weeklyWindow + : formatted.limit.windowMinutes === 5 * 60 + ? copy.fiveHourWindow + : formatted.limit.label; + let resetText = ''; + if (formatted.limit.resetsAt !== undefined) { + try { + const reset = new Intl.DateTimeFormat( + I18n.getLocale(), + I18n.dateFormatOptions({ + year: 'numeric', + month: 'short', + day: 'numeric', + hour: '2-digit', + minute: '2-digit', + }), + ).format(new Date(formatted.limit.resetsAt)); + resetText = ` · ${copy.resets}: ${reset}`; + } catch { + resetText = ''; + } + } + this.quotaItem.tooltip = + `${windowName}: ${copy.remaining} ${Math.round(formatted.limit.remainingPercent)}% ` + + `(${copy.used} ${Math.round(formatted.limit.usedPercent)}%)${resetText}\n\n` + + `${copy.accountSnapshotLastObserved}`; this.quotaItem.backgroundColor = undefined; this.quotaItem.show(); } else { diff --git a/src/test/codexStatus.test.ts b/src/test/codexStatus.test.ts index da3c9470..54c060e7 100644 --- a/src/test/codexStatus.test.ts +++ b/src/test/codexStatus.test.ts @@ -35,31 +35,74 @@ const scope: CodexUsageScopeView = { efforts: [], }; -function limit(resetsAt: number): ProviderLimitSnapshot { +function limit( + resetsAt: number, + windows: ProviderLimitSnapshot['windows'] = [ + { + label: 'primary', + usedPercent: 42, + windowMinutes: 300, + resetsAt, + }, + ], +): ProviderLimitSnapshot { return { provider: 'codex', observedAt: NOW - 60_000, source: 'local-log', confidence: 'last-observed', - windows: [ - { - label: 'primary', - usedPercent: 42, - windowMinutes: 300, - resetsAt, - }, - ], + windows, }; } -test('Codex status defaults to fresh and labels last-observed limits', () => { - assert.deepEqual(formatCodexStatus(scope, 'fresh', limit(NOW + 60_000), NOW), { +test('Codex status defaults to fresh and shows remaining weekly quota', () => { + const formatted = formatCodexStatus(scope, 'fresh', limit(NOW + 60_000, [ + { + label: 'primary', + usedPercent: 42, + windowMinutes: 300, + resetsAt: NOW + 60_000, + }, + { + label: 'secondary', + usedPercent: 67, + windowMinutes: 10_080, + resetsAt: NOW + 86_400_000, + }, + ]), NOW); + + assert.deepEqual(formatted, { text: 'CX 400', - limitText: '5h 42%', + limitText: 'wk 33%', + limit: { + label: 'wk', + windowMinutes: 10_080, + usedPercent: 67, + remainingPercent: 33, + observedAt: NOW - 60_000, + resetsAt: NOW + 86_400_000, + }, stale: false, }); }); +test('the existing five-hour-only preference selects the five-hour remaining quota', () => { + const formatted = formatCodexStatus(scope, 'fresh', limit(NOW + 60_000, [ + { label: 'primary', usedPercent: 42, windowMinutes: 300, resetsAt: NOW + 60_000 }, + { label: 'secondary', usedPercent: 67, windowMinutes: 10_080, resetsAt: NOW + 86_400_000 }, + ]), NOW, { quotaFiveHourOnly: true }); + + assert.equal(formatted.limitText, '5h 58%'); + assert.equal(formatted.limit?.windowMinutes, 300); + assert.equal(formatted.limit?.remainingPercent, 58); +}); + +test('the default falls back to a live five-hour window when weekly data is absent', () => { + const formatted = formatCodexStatus(scope, 'fresh', limit(NOW + 60_000), NOW); + assert.equal(formatted.limitText, '5h 58%'); + assert.equal(formatted.limit?.remainingPercent, 58); +}); + test('processed and output metrics stay distinct', () => { assert.equal(formatCodexStatus(scope, 'processed', null, NOW).text, 'CX 1.2k'); assert.equal(formatCodexStatus(scope, 'output', null, NOW).text, 'CX 200'); diff --git a/src/test/weeklyValue.test.ts b/src/test/weeklyValue.test.ts index b7692239..dd40266b 100644 --- a/src/test/weeklyValue.test.ts +++ b/src/test/weeklyValue.test.ts @@ -274,6 +274,87 @@ test('weekly timeline keeps only the latest current reset for one anonymous Code assert.equal(points.reduce((sum, point) => sum + point.usedEquivalentUsd, 0), 65); }); +test('Codex keeps post-reset usage in a new reset-aligned period when the quota sample is stale', () => { + const now = RESET + HOUR; + const points = buildWeeklyValueTimeline('codex', { + // Older than the sample window, but still a trustworthy reset timestamp. + observations: [observation({ + observedAt: RESET - 8 * DAY, + resetAt: RESET, + sourceKey: 'one-file', + })], + usage: [ + usage(40, RESET - HOUR, 'one-file'), + usage(7, RESET + 30 * 60 * 1000, 'one-file'), + ], + }, { now }); + + const previous = points.find((point) => point.resetAt === RESET); + const current = points.find((point) => point.resetAt === RESET + 7 * DAY); + assert.equal(previous?.usedEquivalentUsd, 40); + assert.equal(current?.usedEquivalentUsd, 7); + assert.equal(previous?.basis, 'reset-aligned-usage'); + assert.equal(current?.basis, 'reset-aligned-usage'); + assert.equal(current?.utilizationPercent, null); + assert.equal(current?.fullEquivalentUsd, null); + assert.equal(points.reduce((sum, point) => sum + point.usedEquivalentUsd, 0), 47); +}); + +test('conflicting stale Codex reset schedules do not choose an arbitrary alignment', () => { + const now = RESET + HOUR; + const points = buildWeeklyValueTimeline('codex', { + observations: [ + observation({ + observedAt: RESET - 8 * DAY, + resetAt: RESET, + sourceKey: 'older-login', + }), + observation({ + observedAt: RESET - 8 * DAY + 1, + resetAt: RESET + DAY, + sourceKey: 'other-login', + }), + ], + usage: [usage(7, RESET + 30 * 60 * 1000, 'older-login')], + }, { now }); + + assert.equal(points.some((point) => point.resetAt === RESET + 7 * DAY), false); + assert.equal(points[0]?.basis, 'calendar-usage'); + assert.equal(points[0]?.fullEquivalentUsd, null); +}); + +test('a fresh Codex reset starts a new current period instead of carrying the old week forward', () => { + const now = RESET + HOUR; + const nextReset = RESET + 7 * DAY; + const points = buildWeeklyValueTimeline('codex', { + observations: [ + observation({ + observedAt: RESET - HOUR, + resetAt: RESET, + usedPercent: 80, + sourceKey: 'one-file', + }), + observation({ + observedAt: RESET + 30 * 60 * 1000, + resetAt: nextReset, + usedPercent: 10, + sourceKey: 'one-file', + }), + ], + usage: [ + usage(40, RESET - 30 * 60 * 1000, 'one-file'), + usage(7, RESET + 45 * 60 * 1000, 'one-file'), + ], + }, { now }); + + assert.equal(points.find((point) => point.resetAt === RESET)?.usedEquivalentUsd, 40); + const current = points.find((point) => point.current); + assert.equal(current?.resetAt, nextReset); + assert.equal(current?.usedEquivalentUsd, 7); + assert.equal(current?.utilizationPercent, 10); + assert.equal(current?.fullEquivalentUsd, null); +}); + test('ambiguous overlapping Codex reset observations do not invent a combined allowance', () => { const now = Date.parse('2026-08-27T04:00:00.000Z'); const points = buildWeeklyValueTimeline('codex', { diff --git a/src/weeklyValue.ts b/src/weeklyValue.ts index bfd82c28..6cbe05ca 100644 --- a/src/weeklyValue.ts +++ b/src/weeklyValue.ts @@ -511,6 +511,40 @@ function alignedResetAt(resetAt: number, anchorResetAt: number): number | null { return Math.abs(aligned - resetAt) <= RESET_CLUSTER_MS ? aligned : null; } +/** + * A quota sample can outlive the seven-day interval it describes. When that + * is the only evidence left after a reset, falling back to Monday buckets would + * put the new usage in a different period from the provider's actual window. + * Recover the reset as an alignment hint only when every observed Codex reset + * sits on the same seven-day sequence. This helper never supplies a quota + * observation, so it cannot enable an allowance estimate on its own. + */ +function latestKnownCodexResetAnchor( + observations: WeeklyQuotaObservation[], + now: number, +): number | undefined { + const candidates = observations + .filter((item) => + item.provider === 'codex' && + Number.isFinite(item.observedAt) && + item.observedAt <= now && + Number.isFinite(item.resetAt) && + item.resetAt > 0, + ) + .sort((left, right) => + right.observedAt - left.observedAt || right.resetAt - left.resetAt, + ); + const latest = candidates[0]; + if (!latest) { + return undefined; + } + return candidates.every((item) => + alignedResetAt(item.resetAt, latest.resetAt) !== null, + ) + ? latest.resetAt + : undefined; +} + /** * Builds the dashboard's one authoritative weekly timeline. Token-log usage is * first assigned to non-overlapping [start, reset) buckets exactly once. Quota @@ -557,7 +591,16 @@ export function buildWeeklyValueTimeline( right.latest.observedAt - left.latest.observedAt || right.cluster.resetAt - left.cluster.resetAt, )[0]; - const anchorResetAt = anchorEntry?.cluster.resetAt ?? options.anchorResetAt; + // If the newest reset sample is older than its own seven-day window, it is + // still useful for grouping usage after a rollover. It remains alignment-only + // until a valid in-window sample exists, so no allowance is inferred here. + const staleResetAnchor = !anchorEntry && + options.anchorResetAt === undefined && + provider === 'codex' + ? latestKnownCodexResetAnchor(normalizedObservations, now) + : undefined; + const anchorResetAt = anchorEntry?.cluster.resetAt ?? + options.anchorResetAt ?? staleResetAnchor; const limit = Math.max(1, Math.floor(options.limit ?? 12)); let history = buildWeeklyUsageHistory(provider, inputs.usage, { now, From cc021b7403883c4583d0f311c946606708b30df8 Mon Sep 17 00:00:00 2001 From: Carl <74130028+Carl723000@users.noreply.github.com> Date: Mon, 31 Aug 2026 14:44:16 +0800 Subject: [PATCH 17/72] fix: repair Codex rolling totals and retain quota resets Co-authored-by: OpenAI Codex <215057067+openai-codex[bot]@users.noreply.github.com> --- ARCHITECTURE-zh-CN.md | 23 +- ARCHITECTURE.md | 33 ++- CHANGELOG.md | 8 + README-en.md | 2 + README-id.md | 2 + README-ja.md | 2 + README-ko.md | 2 + README-zh-CN.md | 2 + README-zh-TW.md | 2 + README.md | 9 + src/providers/codex/codexIndex.ts | 285 +++++++++++++++++++++-- src/providers/codex/codexPeriodIndex.ts | 135 +++++++++++ src/providers/codex/codexProvider.ts | 136 +++++++---- src/providers/codex/codexQuotaHistory.ts | 191 +++++++++++++++ src/providers/codex/codexUsage.ts | 66 ++++-- src/test/codexIndex.test.ts | 180 ++++++++++++++ src/test/codexProvider.test.ts | 92 ++++++++ src/test/codexQuotaHistory.test.ts | 127 ++++++++++ src/test/codexUsage.test.ts | 55 +++++ src/test/repositoryPolicy.test.ts | 18 +- 20 files changed, 1264 insertions(+), 106 deletions(-) create mode 100644 src/providers/codex/codexQuotaHistory.ts create mode 100644 src/test/codexQuotaHistory.test.ts diff --git a/ARCHITECTURE-zh-CN.md b/ARCHITECTURE-zh-CN.md index 899aa6e6..a89f822a 100644 --- a/ARCHITECTURE-zh-CN.md +++ b/ARCHITECTURE-zh-CN.md @@ -9,8 +9,8 @@ **Claude Code Usage** 继续保持 local-first、无 runtime dependency 和 read-mostly。 v2.3.0 保留完整 Claude 体验,并增加 provider-specific 的 Codex Beta 用量与优化视图。 v2.3.1 候选在不改变 provider 计量口径的前提下,增加一套默认关闭、 -local-first 的建议有效性闭环、唯一的显式 BYOK 请求边界、可持久的历史任务状态, -以及 Codex 滚动 30 天的日期→小时投影。 +local-first 的建议有效性闭环、唯一的显式 BYOK 请求边界、可持久的历史任务状态、 +Codex 有界的本地每周重置观测历史,以及滚动 30 天的日期→小时投影。 - Claude:精确的本地 token bucket、模型成本估算和 Anthropic OAuth 5 小时/每周配额。 - Codex Beta:本地 processed/fresh/cache/output/reasoning 指标、模型与 effort 拆分、 @@ -39,6 +39,7 @@ Opt-in GitHub 认证和跨设备聚合同步延后到 v2.4.x,届时单独做 | `providers/codex/codexParser.ts` | Codex 精确单次请求解析及 cumulative high-water 回退、伪名 lineage metadata、结构计数、quality flag 和 last-observed limit。 | | `providers/codex/codexManifest.ts` | Codex 允许目录发现、HMAC file key、fingerprint 和 manifest diff。 | | `providers/codex/codexIndex.ts` | schema-3 的 per-file 数字聚合与重放证据持久化、有界 cold/tail parse、独立 aggregate/period/滚动 30 天小时 coverage 和原子存取。 | +| `providers/codex/codexQuotaHistory.ts` | 小型、账户中性的每周重置观测缓存;压缩重复本地样本,不保留凭据、账户名或原始标签。 | | `providers/codex/codexIndexWorker.ts` / `codexIndexClient.ts` | 后台协调器、recent-first progress、cancel、resume、checkpoint 持久化和 single-flight client。 | | `providers/codex/codexFilePassPool.ts` / `codexFilePassWorker.ts` | 未完成回填期间,用于独立 main、lineage、period、current-day 和 identity per-file pass 的自适应受限本地 pool。 | | `providers/codex/codexProvider.ts` | 面向 extension 的 Codex snapshot facade 与 partial/unavailable/error outcome。 | @@ -68,7 +69,8 @@ Claude JSONL ──> manifest metadata ──> background worker ──> schema guard + 精确单次请求 parser(lineage high-water 回退) - ──> per-file 数字聚合索引 + 滚动稀疏 30 天小时 sidecar + ──> per-file 数字聚合索引 + 有界账户中性的每周重置历史 + ──> 滚动稀疏 30 天小时 sidecar ──> CodexProviderSnapshot ──> Codex scope + insight ──> Codex 状态栏 + provider-aware dashboard render input @@ -121,7 +123,10 @@ output 与 reasoning component 是精确的单次请求归因。`last_token_usag parent baseline,因此该路径按 component 与 lineage 维护 high-water。Unknown parent、counter regression 和 schema drift 产生 quality flag,不生成负用量或伪造精度。 -本地日志中的 Codex `rate_limits.primary` 只是 last-observed snapshot,到 reset 时间后隐藏。 +本地日志中的 Codex `rate_limits` 只是 last-observed snapshot,到 reset 时间后隐藏。 +v2.3.1 候选还会保留一份有界、账户中性的 account-wide 每周观测历史:同一重置边界的 +重复样本只保留一个代表,真正不同的重置时刻则分别保留。这份历史用于对齐和复核每周 +估算,不是账户注册表;如果某次重置从未出现在本地日志行里,插件也无法凭空发现它。 v2.3.0 不读 Codex credential,也不发网络请求刷新它。 ## 隐私与持久化 @@ -133,9 +138,9 @@ Codex 发现仅限: - 默认 `$CODEX_HOME`:`~/.codex` 绝不读取 `auth.json`、SQLite、config secret、keychain、浏览器状态或未知文件。 -Raw path/session/parent ID 只留在本地 worker 的短期内存。磁盘只持久化 machine-salted 伪名 key -和按 day/model/effort/session 聚合的数字,绝不存 prompt、response、command、tool arguments、 -raw line 或 raw path。 +Raw path/session/parent ID 只留在本地 worker 的短期内存。磁盘只持久化 machine-salted 伪名 key、 +按 day/model/effort/session 聚合的数字,以及上面所述的有界账户中性额度历史;绝不存 prompt、 +response、command、tool arguments、raw line、raw path、账户名、凭据或原始 provider label。 Advice 只消费已物化聚合,不重读 JSONL,也不重遍历保留的 records。远程默认为 aggregate-only;prompt sample 与可选 user context 需单独明确同意,并必须原样出现在精确请求预览中。 @@ -220,6 +225,10 @@ Codex 按多 GiB 本地历史设计: - cancel checkpoint 会原子保存 per-file contribution 与 migration progress,下一轮从已验证 cursor resume; - 并发 refresh 共享同一 worker run。 +额度历史由同一批 JSONL 解析流程顺便填充。旧的完整索引最多接受一次基于已保存 +last-observed limit 的 metadata-only 播种;不会增加第二套额度扫描器、timer、网络轮询或凭据读取。 +播种完成后,未变化的 warm refresh 仍然读取 0 字节用量正文。 + 历史任务还使用一份精简的可持久控制状态:measurement version、reason、progress、 failure streak、next eligible time 和 pause reason。成功后立即继续,不人为等待;失败或无进展后 进入 backoff,普通 refresh/watch/focus 事件不能重启同一停滞迁移。Resource registry 记录每个 timer、 diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index 47c602d8..8660b367 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -11,8 +11,9 @@ v2.3.0 preserves the complete Claude experience and adds Codex Beta as a provider-specific usage and optimization view. The v2.3.1 candidate adds a default-off, local-first advice-effectiveness loop, -a single explicit BYOK request boundary, durable historical-work state, and a -rolling 30-day Codex date/hour projection without changing provider accounting. +a single explicit BYOK request boundary, durable historical-work state, a +bounded local history of Codex weekly reset observations, and a rolling 30-day +Codex date/hour projection without changing provider accounting. - Claude: exact local token buckets, model pricing estimates, and Anthropic OAuth 5-hour/weekly quota. @@ -46,6 +47,7 @@ aggregate sync are deferred to v2.4.x after a separate privacy review. | `providers/codex/codexParser.ts` | Codex exact-request parsing with cumulative high-water fallback, pseudonymous lineage metadata, structural counters, quality flags, and last-observed limits. | | `providers/codex/codexManifest.ts` | Allowlisted Codex directory discovery, HMAC file keys, fingerprints, and manifest diffing. | | `providers/codex/codexIndex.ts` | Schema-3 persistent per-file numeric aggregates and replay evidence, bounded cold/tail parsing, independent aggregate/period/rolling-30-day-hour coverage, and atomic save/load. | +| `providers/codex/codexQuotaHistory.ts` | Small account-neutral weekly reset-observation cache; compacts repeated local samples without retaining credentials, account names, or raw labels. | | `providers/codex/codexIndexWorker.ts` / `codexIndexClient.ts` | Background coordinator, recent-first progress, cancellation, resume, checkpoint persistence, and single-flight client. | | `providers/codex/codexFilePassPool.ts` / `codexFilePassWorker.ts` | Adaptive bounded local pool for independent per-file main, lineage, period, current-day, and identity passes during incomplete backfills. | | `providers/codex/codexProvider.ts` | Extension-facing Codex snapshot facade and partial/unavailable/error outcomes. | @@ -76,7 +78,8 @@ allowlisted Codex JSONL ──> manifest metadata ──> background worker ──> schema guard + exact-request parser with lineage high-water fallback - ──> per-file numeric aggregate index + rolling sparse 30-day hourly sidecar + ──> per-file numeric aggregate index + bounded neutral weekly-reset history + ──> rolling sparse 30-day hourly sidecar ──> CodexProviderSnapshot ──> Codex scopes + insights ──> Codex status + provider-aware dashboard render inputs @@ -143,9 +146,14 @@ fallback and may include an inherited parent baseline. That path uses per-component, per-lineage high-water marks. Unknown parents, regressions, and schema drift produce quality flags rather than negative or fabricated usage. -Codex `rate_limits.primary` found in local logs is a last-observed snapshot only. -It is hidden once its reset time passes. v2.3.0 does not read Codex credentials -or make a network call to refresh it. +Codex `rate_limits` found in local logs is a last-observed snapshot only. It is +hidden once its reset time passes. The v2.3.1 candidate also keeps a bounded, +account-neutral history of account-wide weekly observations: repeated samples +for one reset boundary collapse to one representative, while genuinely +different reset times remain separate. This history is evidence for aligning +and auditing weekly estimates; it is not an account registry and cannot reveal +a reset that never appears in a local log line. v2.3.0 does not read Codex +credentials or make a network call to refresh it. ## Privacy and persistence @@ -157,9 +165,10 @@ Codex discovery is restricted to: It never reads `auth.json`, SQLite databases, config secrets, keychains, browser state, or unknown files. Raw paths/session/parent IDs stay in short-lived local -worker memory. Disk persistence contains machine-salted pseudonymous keys and -numeric per-day/model/effort/session aggregates only—never prompt, response, -command, tool-argument, raw-line, or raw-path content. +worker memory. Disk persistence contains machine-salted pseudonymous keys, +numeric per-day/model/effort/session aggregates, and the bounded neutral quota +history described above—never prompt, response, command, tool-argument, +raw-line, raw-path, account name, credential, or raw provider-label content. Advice starts from materialized aggregates rather than rereading JSONL or walking retained records. Aggregate-only is the remote default. Prompt samples @@ -284,6 +293,12 @@ Codex history is designed for multi-gigabyte local corpora: progress, so the next run resumes from the verified cursor; - concurrent refresh requests share one worker run. +Quota history is populated opportunistically by those same JSONL passes. An +older complete index may receive one metadata-only seed from its already saved +last-observed limit, but there is no second quota scanner, timer, network poll, +or credential lookup. Once that seed is present, an unchanged warm refresh +continues to read zero usage-record bodies. + Historical work also has a small durable control state: measurement version, reason, progress, failure streak, next eligible time, and pause reason. Progress continues immediately after success. Failure or no progress applies backoff, so diff --git a/CHANGELOG.md b/CHANGELOG.md index 5016264d..dc29032f 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -44,6 +44,14 @@ upstream release: 1.0.8). Format follows [Keep a Changelog](https://keepachangel and backfills expose their creator, stop conditions, and actual disposal to lifecycle tests. A bounded first-index exception may finish after focus loss, but disable, explicit cancellation, and extension disposal still stop it. +- **Safer Codex rolling totals and reset history** — recent 7/30-day views no + longer trust an inflated or still-rebuilding period sidecar; they use the + verified daily aggregate until the configured-zone projection catches up. + The existing index pass also keeps a small, account-neutral history of + account-wide weekly reset observations, preserving irregular reset boundaries + across file replacement or removal without polling, credentials, or a second + scanner. Ambiguous history remains used-value-only rather than producing a + guessed allowance. ### Privacy and packaging - Disabled or unconsented advice adds no timer, watcher, worker, network request, diff --git a/README-en.md b/README-en.md index 9a65b26b..6250bbf2 100644 --- a/README-en.md +++ b/README-en.md @@ -42,6 +42,8 @@ Hover the quota indicator for a breakdown: - Reliable similar tasks may be paired into a versioned local comparison result after a recommendation is applied. The ledger stores coarse metric, quality, coverage, and version fields only—never prompts, bodies, paths, or session IDs. Without a reliable pair, the UI says the evidence is insufficient. - Codex historical work persists progress, failure streak, next-eligible time, and pause reason. First-use work may perform one bounded heavy pass, then does not repeat equivalent work after completion or hot-loop after failure/no progress. - Every populated Codex day in the rolling last 30 days can expand to its already-indexed hourly buckets with zero JSONL reads on click. Claude and Codex share the configured timezone and `HH:00` formatter. +- Weekly Codex quota observations are compacted during those existing index passes into a bounded, account-neutral local history, so separate irregular reset boundaries can survive file replacement or removal. It does not poll, read credentials, or add a second scanner; unchanged warm refreshes still read zero usage-record bodies. If a reset never appears in a local log line, it cannot be inferred. +- Recent 7/30-day views reject stale or inflated period projections and temporarily fall back to the verified daily aggregate until the zone-aware projection is rebuilt. The package version remains unchanged; this is a local validation candidate, not a published release. diff --git a/README-id.md b/README-id.md index 049053cd..9141f1f5 100644 --- a/README-id.md +++ b/README-id.md @@ -42,6 +42,8 @@ Arahkan kursor ke indikator kuota untuk melihat rinciannya: - Setelah saran diterapkan, hanya tugas serupa yang andal yang dapat menjadi perbandingan lokal sebelum / sesudah dengan versi. Ledger hanya menyimpan metrik kasar, kualitas, cakupan, dan versi—tidak pernah prompt, isi, jalur, atau session ID. Tanpa pasangan yang andal, UI menyatakan bahwa buktinya tidak cukup. - Pekerjaan riwayat Codex menyimpan progres, jumlah kegagalan beruntun, waktu berikutnya yang memenuhi syarat, dan alasan jeda. Pemrosesan berat yang terbatas hanya boleh terjadi sekali pada penggunaan pertama; setelah selesai tidak diulang, dan kegagalan atau ketiadaan progres tidak dipicu terus oleh refresh biasa. - Setiap tanggal Codex yang memiliki data dalam 30 hari terakhir dapat membuka bucket per jam yang sudah diindeks. Klik membaca nol JSONL; Claude dan Codex memakai zona waktu konfigurasi serta format `HH:00` yang sama. +- Pengamatan kuota mingguan Codex dipadatkan selama proses indeks yang sudah ada menjadi riwayat lokal yang terbatas dan netral terhadap akun, sehingga batas reset yang tidak teratur tetap dapat dipertahankan. Tidak ada polling, pembacaan kredensial, atau pemindai kedua; setelah seed sekali, refresh tanpa perubahan tetap membaca nol byte isi catatan penggunaan. Reset yang tidak pernah muncul di log lokal tidak dapat disimpulkan. +- Tampilan 7/30 hari terakhir menolak proyeksi periode yang kedaluwarsa atau membengkak, lalu sementara kembali ke agregat harian terverifikasi sampai proyeksi zona waktu selesai dibangun ulang. Versi paket tetap tidak berubah; ini kandidat validasi lokal, bukan rilis yang diterbitkan. diff --git a/README-ja.md b/README-ja.md index 3394cbfa..6b5d3e60 100644 --- a/README-ja.md +++ b/README-ja.md @@ -42,6 +42,8 @@ - 推奨を適用した後、信頼できる類似タスクだけをバージョン付きローカル比較にできます。保存するのは粗い指標、品質、カバレッジ、バージョンのみで、プロンプト、本文、パス、session ID は保存しません。信頼できる組がなければ根拠不足と表示します。 - Codex の履歴処理は進捗、連続失敗回数、次回実行可能時刻、停止理由を永続化します。初回は一度だけ有界な重い処理を許可し、完了後に同等処理を繰り返さず、失敗や進展なしの後も通常更新で再起動を繰り返しません。 - 直近 30 日でデータのある Codex の各日を、既に索引済みの時間別バケットへ展開できます。クリック時の JSONL 読み取りはゼロで、Claude / Codex は設定タイムゾーンと `HH:00` 表記を共有します。 +- 週間 Codex クォータ観測は既存の索引処理中に圧縮され、上限付きでアカウント中立のローカル履歴として保持されます。そのため不規則なリセット境界も残せます。ポーリング、認証情報の読み取り、二重のスキャナーは追加しません。初回シード後の変更のない更新は、使用量本文を 0 バイトしか読みません。ローカルログに現れないリセットは推測できません。 +- 直近 7 / 30 日の表示は、古いまたは膨張した期間投影を拒否し、タイムゾーン投影の再構築が終わるまで検証済みの日次集計へ一時的にフォールバックします。 パッケージのバージョンは変更していません。これはローカル検証候補であり、公開リリースではありません。 diff --git a/README-ko.md b/README-ko.md index 97b4d612..40573acc 100644 --- a/README-ko.md +++ b/README-ko.md @@ -42,6 +42,8 @@ - 권장 사항을 적용한 뒤 신뢰할 수 있는 유사 작업만 버전이 있는 로컬 전후 비교로 묶습니다. 거친 지표, 품질, 적용 범위, 버전만 저장하고 프롬프트, 본문, 경로, session ID는 저장하지 않습니다. 신뢰할 수 있는 쌍이 없으면 근거 부족을 표시합니다. - Codex 기록 작업은 진행률, 연속 실패 횟수, 다음 실행 가능 시각, 일시 중지 이유를 영구 저장합니다. 최초 한 번의 제한된 무거운 처리를 허용하되 완료 후 동등한 작업을 반복하지 않으며, 실패나 진행 없음 뒤 일반 새로고침이 계속 깨우지 않습니다. - 최근 30일 중 데이터가 있는 모든 Codex 날짜를 이미 인덱싱된 시간 버킷으로 펼칠 수 있습니다. 클릭할 때 JSONL을 읽지 않으며 Claude / Codex는 설정 시간대와 `HH:00` 형식을 공유합니다. +- 주간 Codex 쿼터 관측은 기존 인덱스 처리 중에 압축되어, 상한이 있는 계정 중립 로컬 기록으로 보존됩니다. 따라서 불규칙한 재설정 경계를 유지할 수 있습니다. 폴링, 자격 증명 읽기, 두 번째 스캐너는 추가하지 않습니다. 한 번 시드된 뒤 변경 없는 새로 고침은 사용량 본문을 0바이트만 읽으며, 로컬 로그에 나타나지 않은 재설정은 추론하지 않습니다. +- 최근 7/30일 보기는 오래되었거나 부풀려진 기간 투영을 거부하고, 시간대 투영이 재구축될 때까지 검증된 일별 집계로 임시 폴백합니다. 패키지 버전은 그대로입니다. 이는 로컬 검증 후보이며 게시된 릴리스가 아닙니다. diff --git a/README-zh-CN.md b/README-zh-CN.md index 10172064..4666db87 100644 --- a/README-zh-CN.md +++ b/README-zh-CN.md @@ -61,6 +61,8 @@ AI 个性化仍为可选项。自备 `advice.apiKey` 后,默认请求只含聚 - 建议被标记为「已应用」后,只有可靠同类任务才会形成版本化本地前后比较;没有可靠配对时显示证据不足。 - Codex 历史工作持久记录进度、连续失败次数、下次可执行时间与暂停原因。首次可进行一次有界重处理;完成后不重复,失败或无进展后也不会被普通刷新反复唤醒。 - 最近 30 天内任意有数据的 Codex 日期都能直接展开已索引小时桶;点击读取 JSONL 为 0。Claude 与 Codex 共用配置时区和 `HH:00` 格式。 +- 每周 Codex 额度观测会在既有索引流程中压缩为有界、账户中性的本地历史,因此前后不同的官方重置边界可以保留下来;不增加轮询、凭据读取或第二套扫描器。一次性播种后,未变化的刷新仍读取 0 字节用量正文;如果某次重置从未出现在本地日志行中,插件无法推断它。 +- 最近 7 天 / 30 天视图会拒绝过期或膨胀的期间投影,在时区投影重建完成前暂时回退到已验证的每日聚合。 包版本保持不变;这是供本地验证的候选,不是已发布版本。 diff --git a/README-zh-TW.md b/README-zh-TW.md index eb1c9522..1667a00f 100644 --- a/README-zh-TW.md +++ b/README-zh-TW.md @@ -42,6 +42,8 @@ - 套用建議後,只有可靠的同類工作才會形成版本化的本機前後比較;內容只含粗粒度指標、品質、涵蓋率與版本,不保存提示、正文、路徑或 session ID。沒有可靠配對時會明示證據不足。 - Codex 歷史工作會持久保存進度、連續失敗次數、下次可執行時間與暫停原因。首次可進行一次有界的重處理;完成後不重複,失敗或無進展時也不會被一般重新整理反覆喚醒。 - 最近 30 天內每個有資料的 Codex 日期都可直接展開已索引的小時資料;點擊時不讀取 JSONL。Claude 與 Codex 共用設定時區與 `HH:00` 格式。 +- 每週 Codex 額度觀測會在既有索引流程中壓縮成有界、帳戶中性的本機歷史,因此不同的官方重設邊界可以保留下來;不增加輪詢、憑證讀取或第二套掃描器。一次性播種後,未變更的重新整理仍讀取 0 位元組用量正文;若某次重設從未出現在本機日誌行中,擴充功能無法推斷它。 +- 最近 7 天 / 30 天檢視會拒絕過期或膨脹的期間投影,在設定時區的投影重建完成前暫時回退到已驗證的每日聚合。 套件版本維持不變;這是供本機驗證的候選,不是已發佈版本。 diff --git a/README.md b/README.md index aa248660..e49a484f 100644 --- a/README.md +++ b/README.md @@ -132,6 +132,15 @@ same full-request preview and separate explicit Send action as AI advice. - **Thirty-day hour drill-down** — any populated Codex day in the last 30 days expands from the existing index with zero JSONL reads on click. Claude and Codex use the configured timezone and the same `HH:00` labels. +- **Reset-aware local evidence** — weekly Codex quota observations are compacted + during the existing index passes into a bounded, account-neutral history, so + irregular reset boundaries survive file replacement or removal. It adds no + polling, credential read, or second scanner; after the one-time seed, unchanged + refreshes still read zero usage-record bodies. A reset absent from local logs + cannot be inferred. +- **Safe rolling totals** — recent 7/30-day views reject stale or inflated period + projections and temporarily use the verified daily aggregate until the + configured-zone projection is rebuilt. The candidate keeps the package version unchanged and is not a published release. diff --git a/src/providers/codex/codexIndex.ts b/src/providers/codex/codexIndex.ts index 9e33cb6c..4b47b351 100644 --- a/src/providers/codex/codexIndex.ts +++ b/src/providers/codex/codexIndex.ts @@ -26,6 +26,7 @@ import { parseCodexLine, } from './codexParser'; import { + CODEX_PERIOD_LINEAGE_VERSION, CODEX_ROLLING_HOURLY_DAYS, CodexFilePeriodIndex, CodexFileTodayIndex, @@ -33,10 +34,12 @@ import { CodexPeriodMigrationState, CodexStructuralSummary, CodexTodayMigrationState, + isCompatibleCodexPeriodLineage, pruneCodexHourlyDays, reduceCodexRollingHourlySlice, reduceCodexStructuralSlice, reduceCodexUsageSlice, + codexPeriodFitsAggregate, } from './codexPeriodIndex'; import { parseJsonObject, stringField } from './codexSchema'; import { @@ -68,11 +71,20 @@ import { CODEX_LINEAGE_FINGERPRINTS_PER_BLOCK, CodexLineageTrace, } from './codexLineage'; +import { + appendCodexQuotaHistory, + CODEX_FILE_QUOTA_HISTORY_LIMIT, + CODEX_QUOTA_HISTORY_LIMIT, + codexQuotaObservationsFromLimit, + CodexQuotaObservation, + sanitizeCodexQuotaHistory, +} from './codexQuotaHistory'; export { CodexPeriodMigrationState, CodexStructuralSummary, } from './codexPeriodIndex'; +export type { CodexQuotaObservation } from './codexQuotaHistory'; export interface CodexFileAggregate { total: ProviderTokenCounts; @@ -111,6 +123,8 @@ export interface CodexFileContribution { aggregate: CodexFileAggregate; limit?: ProviderLimitSnapshot; limits?: Record; + /** Small neutral history of weekly reset observations found in this file. */ + quotaHistory?: CodexQuotaObservation[]; qualityFlags: string[]; identityChecked?: boolean; periodMigration?: CodexPeriodMigrationState; @@ -236,6 +250,8 @@ export interface CodexIndexV3 { files: Record; aggregate: CodexProviderAggregate; coverage: CodexIndexCoverage; + /** Bounded neutral history retained across file replacement/removal. */ + quotaHistory?: CodexQuotaObservation[]; } /** @deprecated Compatibility name until the remaining v2 consumers are rewired. */ @@ -643,8 +659,12 @@ function promoteCaughtUpPeriod( timeZone: string, ): void { const migration = contribution.periodMigration; + const prefixEvents = contribution.lineage?.desiredPrefixEvents ?? 0; if ( migration?.timeZone !== timeZone || + migration.prefixEvents !== prefixEvents || + !Number.isFinite(migration.tokenEventsSeen) || + (migration.tokenEventsSeen ?? 0) < prefixEvents || migration.offset < contribution.offset || migration.discardingOversizedLine ) { @@ -654,6 +674,8 @@ function promoteCaughtUpPeriod( timeZone, indexedThrough: contribution.offset, days: migration.days, + lineageVersion: CODEX_PERIOD_LINEAGE_VERSION, + lineagePrefixEvents: prefixEvents, }; contribution.qualityFlags = uniqueFlags( contribution.qualityFlags, @@ -836,15 +858,24 @@ async function updateContribution( const pseudonymize = pseudonymizer(options.salt); let limit = contribution.limit; const limits = { ...(contribution.limits ?? {}) }; + let quotaHistory = appendCodexQuotaHistory( + contribution.quotaHistory ?? [], + [], + CODEX_FILE_QUOTA_HISTORY_LIMIT, + ); const timeZone = resolveTimeZone(options.timeZone); const asOfDay = options.asOfDay ?? dayKeyInZone( new Date((options.now ?? Date.now)()), timeZone, ); const rollingDays = rollingHourlyDaySet(asOfDay); + const desiredPrefixEvents = lineage.desiredPrefixEvents ?? 0; const advancePeriod = aggregate.period?.timeZone === timeZone && - aggregate.period.indexedThrough === contribution.offset; + aggregate.period.indexedThrough === contribution.offset && + lineage.appliedPrefixEvents === desiredPrefixEvents && + isCompatibleCodexPeriodLineage(aggregate.period, desiredPrefixEvents) && + codexPeriodFitsAggregate(aggregate.period, aggregate.total); if (aggregate.today && !reanchorRollingToday(aggregate.today, asOfDay, timeZone)) { delete aggregate.today; } @@ -880,6 +911,7 @@ async function updateContribution( aggregate, limit, limits, + ...(quotaHistory.length > 0 ? { quotaHistory } : {}), qualityFlags: uniqueFlags( contribution.qualityFlags, parserState.qualityFlags, @@ -939,6 +971,11 @@ async function updateContribution( limit = parsed.limit ?? limit; } if (parsed.limit) { + quotaHistory = appendCodexQuotaHistory( + quotaHistory, + codexQuotaObservationsFromLimit(parsed.limit), + CODEX_FILE_QUOTA_HISTORY_LIMIT, + ); const limitKey = parsed.limit.limitId ?? parsed.limit.limitName ?? 'default'; const previousLimit = limits[limitKey]; @@ -997,6 +1034,10 @@ function promoteCaughtUpLineage( } contribution.aggregate = reconciliation.aggregate; lineage.appliedPrefixEvents = reconciliation.prefixEvents; + if (contribution.aggregate.period) { + contribution.aggregate.period.lineageVersion = CODEX_PERIOD_LINEAGE_VERSION; + contribution.aggregate.period.lineagePrefixEvents = reconciliation.prefixEvents; + } contribution.qualityFlags = uniqueFlags( contribution.qualityFlags, reconciliation.qualityFlags, @@ -1045,6 +1086,11 @@ async function reconcileContributionLineage( const aggregate = initial.aggregate; let invalidEventTimestamp = false; const pseudonymize = pseudonymizer(options.salt); + let quotaHistory = appendCodexQuotaHistory( + contribution.quotaHistory ?? [], + [], + CODEX_FILE_QUOTA_HISTORY_LIMIT, + ); const snapshot = (cursor: CodexJsonlCursor): CodexFileContribution => { syncSession(aggregate, parserState); @@ -1059,6 +1105,7 @@ async function reconcileContributionLineage( } return { ...contribution, + ...(quotaHistory.length > 0 ? { quotaHistory } : {}), lineageReconciliation: { prefixEvents, tokenEventsSeen, @@ -1089,6 +1136,13 @@ async function reconcileContributionLineage( } const parsed = parseCodexLine(line, parserState, pseudonymize); parserState = parsed.state; + if (parsed.limit) { + quotaHistory = appendCodexQuotaHistory( + quotaHistory, + codexQuotaObservationsFromLimit(parsed.limit), + CODEX_FILE_QUOTA_HISTORY_LIMIT, + ); + } if (parsed.lineageTokenKey) { tokenEventsSeen += 1; } @@ -1154,10 +1208,18 @@ async function migrateContributionPeriod( onChunk: ContributionChunkHandler, ): Promise { const timeZone = resolveTimeZone(options.timeZone); - const initial = contribution.periodMigration?.timeZone === timeZone - ? contribution.periodMigration + const prefixEvents = contribution.lineage?.desiredPrefixEvents ?? 0; + const savedMigration = contribution.periodMigration; + const canResume = savedMigration?.timeZone === timeZone && + savedMigration.prefixEvents === prefixEvents && + Number.isFinite(savedMigration.tokenEventsSeen) && + (savedMigration.tokenEventsSeen ?? 0) >= prefixEvents; + const initial: CodexPeriodMigrationState = canResume + ? savedMigration : { timeZone, + prefixEvents, + tokenEventsSeen: 0, offset: 0, discardingOversizedLine: false, parserState: createCodexParserState(entry.fileKey), @@ -1166,17 +1228,26 @@ async function migrateContributionPeriod( }; let parserState = initial.parserState; const days = initial.days; + let tokenEventsSeen = initial.tokenEventsSeen ?? 0; let invalidEventTimestamp = false; const pseudonymize = pseudonymizer(options.salt); + let quotaHistory = appendCodexQuotaHistory( + contribution.quotaHistory ?? [], + [], + CODEX_FILE_QUOTA_HISTORY_LIMIT, + ); const snapshot = (cursor: CodexJsonlCursor): CodexFileContribution => ({ ...contribution, + ...(quotaHistory.length > 0 ? { quotaHistory } : {}), aggregate: { ...contribution.aggregate, period: undefined, }, periodMigration: { timeZone, + prefixEvents, + tokenEventsSeen, offset: cursor.offset, discardingOversizedLine: cursor.discardingOversizedLine, parserState, @@ -1203,14 +1274,26 @@ async function migrateContributionPeriod( } const parsed = parseCodexLine(line, parserState, pseudonymize); parserState = parsed.state; - for (const event of parsed.events) { - if (!Number.isFinite(event.timestamp) || event.timestamp <= 0) { - invalidEventTimestamp = true; - } else { - reduceCodexUsageSlice(days, event, timeZone); + if (parsed.limit) { + quotaHistory = appendCodexQuotaHistory( + quotaHistory, + codexQuotaObservationsFromLimit(parsed.limit), + CODEX_FILE_QUOTA_HISTORY_LIMIT, + ); + } + if (parsed.lineageTokenKey) { + tokenEventsSeen += 1; + } + if (tokenEventsSeen > prefixEvents) { + for (const event of parsed.events) { + if (!Number.isFinite(event.timestamp) || event.timestamp <= 0) { + invalidEventTimestamp = true; + } else { + reduceCodexUsageSlice(days, event, timeZone); + } } } - if (parsed.structural) { + if (parsed.structural && tokenEventsSeen >= prefixEvents) { if ( !Number.isFinite(parsed.structural.timestamp) || parsed.structural.timestamp <= 0 @@ -1244,6 +1327,8 @@ async function migrateContributionPeriod( timeZone, indexedThrough: contribution.offset, days: updated.periodMigration.days, + lineageVersion: CODEX_PERIOD_LINEAGE_VERSION, + lineagePrefixEvents: prefixEvents, }; updated.qualityFlags = uniqueFlags( updated.qualityFlags, @@ -1296,9 +1381,15 @@ async function migrateContributionToday( pruneCodexHourlyDays(days, rollingDays); let invalidEventTimestamp = false; const pseudonymize = pseudonymizer(options.salt); + let quotaHistory = appendCodexQuotaHistory( + contribution.quotaHistory ?? [], + [], + CODEX_FILE_QUOTA_HISTORY_LIMIT, + ); const snapshot = (cursor: CodexJsonlCursor): CodexFileContribution => ({ ...contribution, + ...(quotaHistory.length > 0 ? { quotaHistory } : {}), aggregate: { ...contribution.aggregate, today: undefined, @@ -1336,6 +1427,13 @@ async function migrateContributionToday( } const parsed = parseCodexLine(line, parserState, pseudonymize); parserState = parsed.state; + if (parsed.limit) { + quotaHistory = appendCodexQuotaHistory( + quotaHistory, + codexQuotaObservationsFromLimit(parsed.limit), + CODEX_FILE_QUOTA_HISTORY_LIMIT, + ); + } if (parsed.lineageTokenKey) { tokenEventsSeen += 1; } @@ -1762,6 +1860,54 @@ function recomputeAggregate( return aggregate; } +/** + * Merge the small per-file observations into the index-level history before + * any manifest removals can discard their last copy. The history is neutral by + * construction, so it can preserve reset boundaries without pretending to + * identify which account produced them. + */ +function mergeCodexQuotaHistory(index: CodexIndexV3): void { + const additions: CodexQuotaObservation[] = []; + for (const contribution of Object.values(index.files)) { + additions.push(...(contribution.quotaHistory ?? [])); + for (const snapshot of [ + ...Object.values(contribution.limits ?? {}), + ...(contribution.limit ? [contribution.limit] : []), + ]) { + additions.push(...codexQuotaObservationsFromLimit(snapshot)); + } + } + const history = appendCodexQuotaHistory( + sanitizeCodexQuotaHistory(index.quotaHistory, CODEX_QUOTA_HISTORY_LIMIT), + additions, + CODEX_QUOTA_HISTORY_LIMIT, + ); + index.quotaHistory = history.length > 0 ? history : undefined; +} + +function hasCodexQuotaEvidence(index: CodexIndexV3): boolean { + return Object.values(index.files).some((contribution) => + (contribution.quotaHistory?.length ?? 0) > 0 || [ + ...Object.values(contribution.limits ?? {}), + ...(contribution.limit ? [contribution.limit] : []), + ].some((snapshot) => codexQuotaObservationsFromLimit(snapshot).length > 0), + ); +} + +export function isReadableCodexPeriod( + contribution: CodexFileContribution | undefined, + timeZone: string, +): boolean { + if (!contribution || !isCodexUsageContributionCurrent(contribution)) { + return false; + } + const period = contribution.aggregate.period; + const prefixEvents = contribution.lineage?.desiredPrefixEvents ?? 0; + return period?.timeZone === timeZone && + isCompatibleCodexPeriodLineage(period, prefixEvents) && + codexPeriodFitsAggregate(period, contribution.aggregate.total); +} + function coverageFor( files: Record, manifest: CodexManifest, @@ -1831,8 +1977,18 @@ function coverageFor( } rangeTotalFiles += 1; rangeTotalBytes += entry.size; - const periodIsCurrent = Boolean( - contribution && isCodexUsageContributionCurrent(contribution), + const periodIsCurrent = isReadableCodexPeriod(contribution, timeZone); + const periodMigration = contribution?.periodMigration; + // A migration draft is intentionally not a readable period yet, but + // its cursor still represents verified work for the progress display. + // Keep this separate from periodIsCurrent so a rejected legacy sidecar + // cannot be rendered as data while an active rebuild remains visible. + const periodMigrationIsCurrent = Boolean( + contribution && + isCodexUsageContributionCurrent(contribution) && + periodMigration?.timeZone === timeZone && + periodMigration.prefixEvents === + (contribution.lineage?.desiredPrefixEvents ?? 0), ); const promoted = periodIsCurrent && contribution?.aggregate.period?.timeZone === timeZone @@ -1842,9 +1998,8 @@ function coverageFor( entry.size, ) : 0; - const draft = periodIsCurrent && - contribution?.periodMigration?.timeZone === timeZone - ? Math.min(contribution.periodMigration.offset, contribution.offset, entry.size) + const draft = periodMigrationIsCurrent && periodMigration + ? Math.min(periodMigration.offset, contribution.offset, entry.size) : 0; const migrated = Math.max(0, Math.max(promoted, draft)); migratedBytes += migrated; @@ -1893,11 +2048,11 @@ function coverageFor( contribution.lineageReconciliation || contribution.lineage?.appliedPrefixEvents !== contribution.lineage?.desiredPrefixEvents || - contribution.aggregate.period?.timeZone !== timeZone + !isReadableCodexPeriod(contribution, timeZone) ) { continue; } - const eligibleDays = Object.keys(contribution.aggregate.period.days) + const eligibleDays = Object.keys(contribution.aggregate.period!.days) .filter((day) => rollingDays.has(day)); if (eligibleDays.length === 0) { continue; @@ -2023,6 +2178,7 @@ function recomputeDerivedIndex( initialDeduplication.canonicalFileKeys, ); } + mergeCodexQuotaHistory(index); const deduplication = classifyCodexSessionDuplicates(index.files); index.aggregate = recomputeAggregate(index.files, deduplication); index.coverage = coverageFor( @@ -2052,6 +2208,12 @@ function isWarmNoOp( timeZone: string, asOfDay: string, ): boolean { + // Older schema-3 indexes have a current `limit` projection but no neutral + // reset history. Give the next refresh one chance to seed the bounded cache + // even when the JSONL manifest itself has not changed. + if (hasCodexQuotaEvidence(previous) && !previous.quotaHistory?.length) { + return false; + } const totalBytes = manifest.files.reduce((sum, entry) => sum + entry.size, 0); const hourlyCoverage = previous.coverage.hourly; if ( @@ -2094,6 +2256,7 @@ function isWarmNoOp( !contribution.lineageReconciliation && contribution.aggregate.period?.timeZone === timeZone && contribution.aggregate.period.indexedThrough >= contribution.offset && + isReadableCodexPeriod(contribution, timeZone) && !contribution.qualityFlags.includes('stale-file') && !contribution.qualityFlags.includes('stale-reset-required'), ); @@ -2177,6 +2340,9 @@ export async function updateCodexIndex( }; } const index = cloneIndex(previous); + // Seed the durable neutral history before moved/removed files disappear. + // This also upgrades an older index without rereading unchanged JSONL. + mergeCodexQuotaHistory(index); if (previous.coverage.period.timeZone !== timeZone) { index.indexGeneration = Math.max(1, Math.floor(previous.indexGeneration ?? 0) + 1); } @@ -2227,10 +2393,29 @@ export async function updateCodexIndex( } for (const contribution of Object.values(index.files)) { promoteCaughtUpLineage(contribution); - if (contribution.aggregate.period?.timeZone !== timeZone) { + const desiredPrefixEvents = contribution.lineage?.desiredPrefixEvents ?? 0; + if (contribution.aggregate.period && ( + !isCompatibleCodexPeriodLineage( + contribution.aggregate.period, + desiredPrefixEvents, + ) || + !codexPeriodFitsAggregate( + contribution.aggregate.period, + contribution.aggregate.total, + ) || + contribution.aggregate.period.timeZone !== timeZone + )) { delete contribution.aggregate.period; } - if (contribution.periodMigration?.timeZone !== timeZone) { + if ( + contribution.periodMigration && + ( + contribution.periodMigration.timeZone !== timeZone || + contribution.periodMigration.prefixEvents !== desiredPrefixEvents || + !Number.isFinite(contribution.periodMigration.tokenEventsSeen) || + (contribution.periodMigration.tokenEventsSeen ?? 0) < desiredPrefixEvents + ) + ) { delete contribution.periodMigration; } promoteCaughtUpPeriod(contribution, timeZone); @@ -2740,6 +2925,14 @@ export async function updateCodexIndex( contribution.offset >= entry.size && !contribution.discardingOversizedLine && ( + !isCompatibleCodexPeriodLineage( + contribution.aggregate.period, + contribution.lineage?.desiredPrefixEvents ?? 0, + ) || + !codexPeriodFitsAggregate( + contribution.aggregate.period, + contribution.aggregate.total, + ) || contribution.aggregate.period?.timeZone !== timeZone || contribution.aggregate.period.indexedThrough !== contribution.offset ), @@ -2765,7 +2958,12 @@ export async function updateCodexIndex( continue; } const base = cloneContribution(prior); - const start = base.periodMigration?.timeZone === timeZone + const start = base.periodMigration?.timeZone === timeZone && + base.periodMigration.prefixEvents === + (base.lineage?.desiredPrefixEvents ?? 0) && + Number.isFinite(base.periodMigration.tokenEventsSeen) && + (base.periodMigration.tokenEventsSeen ?? 0) >= + (base.lineage?.desiredPrefixEvents ?? 0) ? base.periodMigration.offset : 0; const endExclusive = Math.min( @@ -2805,7 +3003,12 @@ export async function updateCodexIndex( continue; } const base = cloneContribution(prior); - const start = base.periodMigration?.timeZone === timeZone + const start = base.periodMigration?.timeZone === timeZone && + base.periodMigration.prefixEvents === + (base.lineage?.desiredPrefixEvents ?? 0) && + Number.isFinite(base.periodMigration.tokenEventsSeen) && + (base.periodMigration.tokenEventsSeen ?? 0) >= + (base.lineage?.desiredPrefixEvents ?? 0) ? base.periodMigration.offset : 0; const endExclusive = Math.min( @@ -3102,6 +3305,7 @@ interface LegacyCodexIndexV1 { readonly files: Readonly>; readonly aggregate?: unknown; readonly coverage?: unknown; + readonly quotaHistory?: unknown; } interface LegacyCodexFileContribution { @@ -3118,6 +3322,7 @@ interface LegacyCodexFileContribution { readonly aggregate?: unknown; readonly limit?: unknown; readonly limits?: unknown; + readonly quotaHistory?: unknown; readonly qualityFlags?: unknown; readonly identityChecked?: unknown; } @@ -3139,6 +3344,7 @@ interface LegacyCodexIndexV2 { readonly files: Readonly>; readonly aggregate?: unknown; readonly coverage?: unknown; + readonly quotaHistory?: unknown; } function isLegacyIndexV2(value: unknown): value is LegacyCodexIndexV2 { @@ -3418,6 +3624,12 @@ function sanitizePeriod(value: unknown): CodexFilePeriodIndex | undefined { timeZone: resolveTimeZone(value.timeZone), indexedThrough: Math.max(0, finiteNumber(value.indexedThrough)), days, + ...(optionalNumber(value.lineageVersion) !== undefined + ? { lineageVersion: optionalNumber(value.lineageVersion) } + : {}), + ...(optionalNumber(value.lineagePrefixEvents) !== undefined + ? { lineagePrefixEvents: Math.max(0, Math.floor(optionalNumber(value.lineagePrefixEvents)!)) } + : {}), }; } @@ -3559,6 +3771,12 @@ function sanitizePeriodMigration( } return { timeZone: period.timeZone, + ...(optionalNumber(value.prefixEvents) !== undefined + ? { prefixEvents: Math.max(0, Math.floor(optionalNumber(value.prefixEvents)!)) } + : {}), + ...(optionalNumber(value.tokenEventsSeen) !== undefined + ? { tokenEventsSeen: Math.max(0, Math.floor(optionalNumber(value.tokenEventsSeen)!)) } + : {}), offset: Math.max(0, finiteNumber(value.offset)), discardingOversizedLine: value.discardingOversizedLine === true, parserState: sanitizeParserState(value.parserState, fileKey), @@ -3932,11 +4150,15 @@ function migrateIndexV1(index: LegacyCodexIndexV1): CodexIndexV3 { false, ); } + const quotaHistory = sanitizeCodexQuotaHistory(index.quotaHistory); return { schemaVersion: 3, files, aggregate: sanitizeProviderAggregate(index.aggregate), coverage: sanitizeCoverage(index.coverage), + ...(quotaHistory.length > 0 + ? { quotaHistory } + : {}), }; } @@ -3959,6 +4181,10 @@ function sanitizeFileContribution( const safeOffset = offsetOverride ?? Math.max(0, finiteNumber(contribution.offset)); const limit = sanitizeLimit(contribution.limit); const limits = sanitizeLimits(contribution.limits); + const quotaHistory = sanitizeCodexQuotaHistory( + contribution.quotaHistory, + CODEX_FILE_QUOTA_HISTORY_LIMIT, + ); const lineage = sanitizeLineage(contribution.lineage); const lineageReconciliation = sanitizeLineageReconciliation( contribution.lineageReconciliation, @@ -3984,6 +4210,20 @@ function sanitizeFileContribution( ); if (periodMigration) { periodMigration.offset = Math.min(periodMigration.offset, safeOffset); + if (lineage) { + if (periodMigration.prefixEvents !== undefined) { + periodMigration.prefixEvents = Math.min( + periodMigration.prefixEvents, + lineage.tokenEvents, + ); + } + if (periodMigration.tokenEventsSeen !== undefined) { + periodMigration.tokenEventsSeen = Math.min( + periodMigration.tokenEventsSeen, + lineage.tokenEvents, + ); + } + } } const todayMigration = sanitizeTodayMigration( contribution.todayMigration, @@ -4029,6 +4269,7 @@ function sanitizeFileContribution( aggregate, ...(limit ? { limit } : {}), ...(limits ? { limits } : {}), + ...(quotaHistory.length > 0 ? { quotaHistory } : {}), qualityFlags: sanitizeQualityFlags(contribution.qualityFlags), ...(typeof contribution.identityChecked === 'boolean' ? { identityChecked: contribution.identityChecked } @@ -4068,6 +4309,7 @@ function sanitizeIndexV2(value: unknown): CodexIndexV3 { delete contribution.todayMigration; } } + const quotaHistory = sanitizeCodexQuotaHistory(index.quotaHistory); return { schemaVersion: 3, ...(typeof index.indexGeneration === 'number' && @@ -4077,6 +4319,9 @@ function sanitizeIndexV2(value: unknown): CodexIndexV3 { files, aggregate: sanitizeProviderAggregate(index.aggregate), coverage, + ...(quotaHistory.length > 0 + ? { quotaHistory } + : {}), }; } diff --git a/src/providers/codex/codexPeriodIndex.ts b/src/providers/codex/codexPeriodIndex.ts index 9553dc34..5f0bf1c9 100644 --- a/src/providers/codex/codexPeriodIndex.ts +++ b/src/providers/codex/codexPeriodIndex.ts @@ -27,6 +27,10 @@ export interface CodexFilePeriodIndex { timeZone: string; indexedThrough: number; days: Record; + /** Semantic marker for period data produced after lineage filtering. */ + lineageVersion?: number; + /** The lineage prefix excluded while this period was built. */ + lineagePrefixEvents?: number; } export interface CodexHourlySlice { @@ -74,11 +78,142 @@ export interface CodexTodayMigrationState extends CodexJsonlCursor { export interface CodexPeriodMigrationState extends CodexJsonlCursor { timeZone: string; + /** Missing on pre-fix drafts; those drafts must restart from byte zero. */ + prefixEvents?: number; + /** Number of lineage token events consumed by the resumable cursor. */ + tokenEventsSeen?: number; parserState: CodexParserState; days: Record; qualityFlags: string[]; } +/** Increment when the period lineage filtering algorithm changes. */ +export const CODEX_PERIOD_LINEAGE_VERSION = 1; + +export function isCurrentCodexPeriodLineage( + period: CodexFilePeriodIndex | undefined, + prefixEvents: number, +): boolean { + return Boolean( + period && + period.lineageVersion === CODEX_PERIOD_LINEAGE_VERSION && + period.lineagePrefixEvents === Math.max(0, Math.floor(prefixEvents)), + ); +} + +/** + * Pre-fix schema-3 indexes did not carry a lineage marker. A bounded + * aggregate-size check can still safely read those legacy slices; once a + * marker exists, it must match the current filtering algorithm exactly. + */ +export function isCompatibleCodexPeriodLineage( + period: CodexFilePeriodIndex | undefined, + prefixEvents: number, +): boolean { + if (!period) { + return false; + } + const hasMarker = + period.lineageVersion !== undefined || + period.lineagePrefixEvents !== undefined; + return !hasMarker || isCurrentCodexPeriodLineage(period, prefixEvents); +} + +const TOKEN_FIELDS = [ + 'inputTotal', + 'cachedInput', + 'cacheWriteInput', + 'outputTotal', + 'reasoningOutput', + 'sourceTotal', +] as const; + +type TokenField = (typeof TOKEN_FIELDS)[number]; + +function tokenValue(tokens: ProviderTokenCounts, field: TokenField): number { + const value = tokens[field]; + return typeof value === 'number' && Number.isFinite(value) + ? Math.max(0, value) + : 0; +} + +function tokenShapeIsValid(tokens: ProviderTokenCounts): boolean { + return TOKEN_FIELDS.every((field) => { + const value = tokens[field]; + return value === undefined || + (typeof value === 'number' && Number.isFinite(value) && value >= 0); + }); +} + +function sumTokenBuckets( + buckets: readonly ProviderTokenCounts[], +): ProviderTokenCounts { + const total: ProviderTokenCounts = { + inputTotal: 0, + cachedInput: 0, + cacheWriteInput: 0, + outputTotal: 0, + reasoningOutput: 0, + sourceTotal: 0, + }; + for (const bucket of buckets) { + for (const field of TOKEN_FIELDS) { + total[field] = tokenValue(total, field) + tokenValue(bucket, field); + } + } + return total; +} + +function tokensFitWithin( + actual: ProviderTokenCounts, + allowed: ProviderTokenCounts, +): boolean { + if (!tokenShapeIsValid(actual) || !tokenShapeIsValid(allowed)) { + return false; + } + // Older persisted DTOs legitimately omit optional buckets such as + // reasoningOutput, sourceTotal, or cacheWriteInput. An omitted allowance is + // unknown, not zero; constraining it to zero would reject otherwise valid + // legacy period projections and make the reader fall back unnecessarily. + return TOKEN_FIELDS.every((field) => { + if (allowed[field] === undefined) { + return true; + } + return tokenValue(actual, field) <= tokenValue(allowed, field); + }); +} + +/** + * A period is a subset of a file's all-time aggregate. This inexpensive + * invariant lets readers reject old/replayed sidecars immediately, before a + * background rebuild has finished. + */ +export function codexPeriodFitsAggregate( + period: CodexFilePeriodIndex | undefined, + aggregateTotal: ProviderTokenCounts, +): boolean { + if (!period || !tokenShapeIsValid(aggregateTotal)) { + return false; + } + const periodTotal = sumTokenBuckets( + Object.values(period.days).map((slice) => slice.total), + ); + if (!tokensFitWithin(periodTotal, aggregateTotal)) { + return false; + } + return Object.values(period.days).every((slice) => + tokenShapeIsValid(slice.total) && + tokensFitWithin( + sumTokenBuckets(Object.values(slice.byModel)), + slice.total, + ) && + tokensFitWithin( + sumTokenBuckets(Object.values(slice.byEffort)), + slice.total, + ), + ); +} + function zeroTokens(): ProviderTokenCounts { return { inputTotal: 0, diff --git a/src/providers/codex/codexProvider.ts b/src/providers/codex/codexProvider.ts index 83ec7c04..af4899d9 100644 --- a/src/providers/codex/codexProvider.ts +++ b/src/providers/codex/codexProvider.ts @@ -16,10 +16,23 @@ import { CodexTodayCoverage, CodexIndexV1, createEmptyCodexIndex, + isReadableCodexPeriod, isCodexUsageContributionCurrent, loadCodexIndex, } from './codexIndex'; -import { CODEX_ROLLING_HOURLY_DAYS } from './codexPeriodIndex'; +import { + CODEX_ROLLING_HOURLY_DAYS, + codexPeriodFitsAggregate, + isCompatibleCodexPeriodLineage, +} from './codexPeriodIndex'; +import { + appendCodexQuotaHistory, + CODEX_QUOTA_HISTORY_LIMIT, + CODEX_QUOTA_RESET_CLUSTER_MS, + CodexQuotaObservation, + weeklyQuotaObservationsFromCodexHistory, + weeklyQuotaObservationsFromCodexLimit, +} from './codexQuotaHistory'; import { CodexIndexClient } from './codexIndexClient'; import { CodexWorkerRefreshInput, @@ -150,57 +163,77 @@ function aggregateTotal( return total; } -function isAccountWideCodexLimit(snapshot: ProviderLimitSnapshot): boolean { - const id = (snapshot.limitId ?? '').trim().toLowerCase(); - const name = (snapshot.limitName ?? '').trim().toLowerCase(); - return id === 'codex' || name === 'codex' || (!name && id === ''); -} - function codexWeeklyValueInputs( files: CodexFileContribution[], + persistedHistory: readonly CodexQuotaObservation[] = [], + timeZone = 'UTC', ): WeeklyValueInputs { - const observations: WeeklyQuotaObservation[] = []; + const currentObservations: WeeklyQuotaObservation[] = []; const usage: WeeklyValueInputs['usage'] = []; + const seenObservations = new Set(); + const fileHistory: CodexQuotaObservation[] = []; for (const file of files) { const sourceKey = file.fileKey; - const seenSnapshots = new Set(); + fileHistory.push(...(file.quotaHistory ?? [])); for (const snapshot of [ ...Object.values(file.limits ?? {}), ...(file.limit ? [file.limit] : []), ]) { - if (!isAccountWideCodexLimit(snapshot)) { - continue; - } - const seriesKey = snapshot.limitId ?? snapshot.limitName ?? 'codex'; - for (const window of snapshot.windows) { - if ( - window.windowMinutes !== 7 * 24 * 60 || - window.resetsAt === undefined - ) { - continue; - } + for (const observation of weeklyQuotaObservationsFromCodexLimit( + snapshot, + sourceKey, + )) { const identity = [ - seriesKey, - snapshot.observedAt, - window.resetsAt, - window.usedPercent, + observation.observedAt, + observation.resetAt, + observation.usedPercent, + observation.sourceKey ?? '', ].join('|'); - if (seenSnapshots.has(identity)) { - continue; + if (!seenObservations.has(identity)) { + seenObservations.add(identity); + currentObservations.push(observation); } - seenSnapshots.add(identity); - observations.push({ - provider: 'codex', - seriesKey, - ...(snapshot.limitName ? { seriesLabel: snapshot.limitName } : {}), - observedAt: snapshot.observedAt, - resetAt: window.resetsAt, - usedPercent: window.usedPercent, - sourceKey, - }); } } - for (const slice of Object.values(file.aggregate.period?.days ?? {})) { + } + + const cachedHistory = appendCodexQuotaHistory( + persistedHistory, + fileHistory, + CODEX_QUOTA_HISTORY_LIMIT, + ); + const cachedObservations = weeklyQuotaObservationsFromCodexHistory( + cachedHistory, + ).filter((cached) => !currentObservations.some((current) => + Math.abs(current.resetAt - cached.resetAt) <= CODEX_QUOTA_RESET_CLUSTER_MS, + )); + const observations = [...currentObservations, ...cachedObservations]; + + for (const file of files) { + const sourceKey = file.fileKey; + const usageReady = isCodexUsageContributionCurrent(file) && + !file.lineageReconciliation && + ( + !file.lineage || + file.lineage.appliedPrefixEvents === file.lineage.desiredPrefixEvents + ); + if (!usageReady) { + continue; + } + const period = file.aggregate.period; + const prefixEvents = file.lineage?.desiredPrefixEvents ?? 0; + if ( + !period || + period.timeZone !== timeZone || + !isCompatibleCodexPeriodLineage(period, prefixEvents) || + !codexPeriodFitsAggregate(period, file.aggregate.total) + ) { + // A pre-fix or still-rebuilding period projection may contain copied + // lineage records. Do not turn the reliable all-time total into a false + // model-priced weekly estimate; the next bounded refresh will rebuild it. + continue; + } + for (const slice of Object.values(period.days)) { const timestamp = slice.lastObservedAt ?? slice.firstObservedAt; if (timestamp === undefined) { continue; @@ -242,13 +275,22 @@ function snapshotFromIndex( ), ); const files = usageContributions - .map((file) => ({ - ...file.aggregate, - session: { - ...file.aggregate.session, - sessionTitle: sessionTitles.get(file.aggregate.session.sessionKey), - }, - })) + .map((file) => { + const periodReadable = isReadableCodexPeriod( + file, + index.coverage.period.timeZone, + ); + return { + ...file.aggregate, + // Keep the trusted all-time aggregate visible, but do not let a + // rejected or rebuilding period projection reach any period consumer. + ...(periodReadable ? {} : { period: undefined }), + session: { + ...file.aggregate.session, + sessionTitle: sessionTitles.get(file.aggregate.session.sessionKey), + }, + }; + }) .sort( (left, right) => (right.session.startedAt ?? 0) - (left.session.startedAt ?? 0), @@ -274,7 +316,11 @@ function snapshotFromIndex( qualityFlags: qualityCounts(contributions), limits, limit: limits[0] ?? null, - weeklyValueInputs: codexWeeklyValueInputs(usageContributions), + weeklyValueInputs: codexWeeklyValueInputs( + usageContributions, + index.quotaHistory ?? [], + index.coverage.period.timeZone, + ), hourlyCoverage, todayCoverage: index.coverage.today, todayPartial: !index.coverage.today.complete, diff --git a/src/providers/codex/codexQuotaHistory.ts b/src/providers/codex/codexQuotaHistory.ts new file mode 100644 index 00000000..08ba50d6 --- /dev/null +++ b/src/providers/codex/codexQuotaHistory.ts @@ -0,0 +1,191 @@ +import type { + ProviderLimitSnapshot, +} from '../providerTypes'; +import type { + WeeklyQuotaObservation, +} from '../../weeklyValue'; + +/** The Codex secondary window is the only window used for weekly estimates. */ +export const CODEX_WEEKLY_WINDOW_MINUTES = 7 * 24 * 60; + +/** Reset timestamps from the same provider window may differ by a few minutes. */ +export const CODEX_QUOTA_RESET_CLUSTER_MS = 5 * 60 * 1000; + +/** + * Keep the per-file copy small. The index-level history below is the durable + * cross-file record; the per-file copy lets a changed or newly discovered file + * contribute its observations again without retaining every quota poll. + */ +export const CODEX_FILE_QUOTA_HISTORY_LIMIT = 16; + +/** A few hundred reset events are only a few kilobytes, but remain bounded. */ +export const CODEX_QUOTA_HISTORY_LIMIT = 256; + +/** + * Deliberately neutral persisted quota evidence. There is no account name, + * source key, token, credential, or raw provider label in this shape. + */ +export interface CodexQuotaObservation { + provider: 'codex'; + seriesKey: 'codex'; + observedAt: number; + resetAt: number; + usedPercent: number; +} + +function finite(value: unknown): number | undefined { + return typeof value === 'number' && Number.isFinite(value) + ? value + : undefined; +} + +function isAccountWideCodexLimit(snapshot: ProviderLimitSnapshot): boolean { + const id = (snapshot.limitId ?? '').trim().toLowerCase(); + const name = (snapshot.limitName ?? '').trim().toLowerCase(); + // Spark has its own rolling allowance and must never be folded into the + // account-wide Codex estimate. + return id === 'codex' || name === 'codex' || (!id && !name); +} + +function normalizeObservation(value: unknown): CodexQuotaObservation | undefined { + if (typeof value !== 'object' || value === null || Array.isArray(value)) { + return undefined; + } + const record = value as Record; + const observedAt = finite(record.observedAt); + const resetAt = finite(record.resetAt); + const usedPercent = finite(record.usedPercent); + if ( + observedAt === undefined || + resetAt === undefined || + usedPercent === undefined || + observedAt <= 0 || + resetAt <= 0 + ) { + return undefined; + } + return { + provider: 'codex', + seriesKey: 'codex', + observedAt, + resetAt, + usedPercent: Math.min(100, Math.max(0, usedPercent)), + }; +} + +/** Extract only account-wide weekly windows from a parsed local snapshot. */ +export function codexQuotaObservationsFromLimit( + snapshot: ProviderLimitSnapshot, +): CodexQuotaObservation[] { + if (snapshot.provider !== 'codex' || !isAccountWideCodexLimit(snapshot)) { + return []; + } + return snapshot.windows.flatMap((window) => { + if (window.windowMinutes !== CODEX_WEEKLY_WINDOW_MINUTES) { + return []; + } + const observation = normalizeObservation({ + observedAt: snapshot.observedAt, + resetAt: window.resetsAt, + usedPercent: window.usedPercent, + }); + return observation ? [observation] : []; + }); +} + +function representative( + items: CodexQuotaObservation[], +): CodexQuotaObservation { + // Prefer the latest sample that was actually observed before the reset. A + // delayed log line carrying an old reset must not replace a useful boundary + // sample with an observation that the weekly timeline will reject. + const beforeReset = items + .filter((item) => item.observedAt < item.resetAt) + .sort((left, right) => + left.observedAt - right.observedAt || left.resetAt - right.resetAt, + ); + if (beforeReset.length > 0) { + return beforeReset[beforeReset.length - 1]; + } + return [...items].sort((left, right) => + left.observedAt - right.observedAt || left.resetAt - right.resetAt, + )[items.length - 1]; +} + +/** + * Compact quota polls to one useful observation per reset boundary. This is + * intentionally different from retaining every percentage step: the product + * needs to know that an irregular reset happened, not to build a second poll + * database. The result is sorted oldest-first and capped by reset count. + */ +export function appendCodexQuotaHistory( + history: readonly CodexQuotaObservation[] = [], + additions: readonly CodexQuotaObservation[] = [], + limit = CODEX_QUOTA_HISTORY_LIMIT, +): CodexQuotaObservation[] { + const all = [...history, ...additions] + .map(normalizeObservation) + .filter((item): item is CodexQuotaObservation => item !== undefined) + .sort((left, right) => + left.resetAt - right.resetAt || left.observedAt - right.observedAt, + ); + const groups: CodexQuotaObservation[][] = []; + for (const item of all) { + const current = groups[groups.length - 1]; + // Compare with the beginning of the group. Comparing with the previous + // item would let a chain of small skews merge two genuinely different + // reset boundaries that are more than the allowed cluster width apart. + const currentResetAt = current?.[0]?.resetAt; + if ( + current && + currentResetAt !== undefined && + item.resetAt - currentResetAt <= CODEX_QUOTA_RESET_CLUSTER_MS + ) { + current.push(item); + } else { + groups.push([item]); + } + } + const compacted = groups.map(representative).sort((left, right) => + left.resetAt - right.resetAt || left.observedAt - right.observedAt, + ); + return compacted.slice(-Math.max(1, Math.floor(limit))); +} + +/** Sanitize untrusted persisted data without carrying through raw fields. */ +export function sanitizeCodexQuotaHistory( + value: unknown, + limit = CODEX_QUOTA_HISTORY_LIMIT, +): CodexQuotaObservation[] { + const raw = Array.isArray(value) ? value : []; + const normalized = raw + .map(normalizeObservation) + .filter((item): item is CodexQuotaObservation => item !== undefined); + return appendCodexQuotaHistory([], normalized, limit); +} + +export function weeklyQuotaObservationsFromCodexHistory( + history: readonly CodexQuotaObservation[], +): WeeklyQuotaObservation[] { + return history.map((item) => ({ + provider: 'codex' as const, + seriesKey: 'codex', + observedAt: item.observedAt, + resetAt: item.resetAt, + usedPercent: item.usedPercent, + })); +} + +export function weeklyQuotaObservationsFromCodexLimit( + snapshot: ProviderLimitSnapshot, + sourceKey?: string, +): WeeklyQuotaObservation[] { + return codexQuotaObservationsFromLimit(snapshot).map((item) => ({ + provider: 'codex' as const, + seriesKey: 'codex', + observedAt: item.observedAt, + resetAt: item.resetAt, + usedPercent: item.usedPercent, + ...(sourceKey ? { sourceKey } : {}), + })); +} diff --git a/src/providers/codex/codexUsage.ts b/src/providers/codex/codexUsage.ts index d84c1a96..c31024c2 100644 --- a/src/providers/codex/codexUsage.ts +++ b/src/providers/codex/codexUsage.ts @@ -9,7 +9,12 @@ import { CodexTodayCoverage, } from './codexIndex'; import { formatHourLabel, rollingDayKeysFromDayKey } from '../../dateKeys'; -import { CODEX_ROLLING_HOURLY_DAYS } from './codexPeriodIndex'; +import { + CODEX_ROLLING_HOURLY_DAYS, + CodexDailySlice, + CODEX_PERIOD_LINEAGE_VERSION, + codexPeriodFitsAggregate, +} from './codexPeriodIndex'; import { freshInputPlusOutput, processedTokens, @@ -314,6 +319,39 @@ function apiEquivalentForBuckets( ); } +function fallbackPeriodSlice(total: ProviderTokenCounts): CodexDailySlice { + return { + total, + byModel: {}, + byEffort: {}, + structural: zeroStructural(), + }; +} + +/** + * Period sidecars are a configured-zone, model-aware projection. A legacy or + * currently rebuilding sidecar must not be allowed to inflate a range beyond + * the verified per-file all-time aggregate. The UTC-keyed all-time day map is + * a conservative temporary fallback until the zone-aware sidecar is rebuilt; + * it preserves totals without pretending to know model/day attribution. + */ +function periodSlicesForFile( + file: CodexFileAggregate, + timeZone: string, +): Array<[string, CodexDailySlice]> { + if ( + file.period?.timeZone === timeZone && + (file.period.lineageVersion === undefined || + file.period.lineageVersion === CODEX_PERIOD_LINEAGE_VERSION) && + codexPeriodFitsAggregate(file.period, file.total) + ) { + return Object.entries(file.period.days); + } + return Object.entries(file.byDay) + .filter(([day]) => /^\d{4}-\d{2}-\d{2}$/.test(day)) + .map(([day, total]) => [day, fallbackPeriodSlice(total)]); +} + function ratio(numerator: number, denominator: number): number { return denominator > 0 ? numerator / denominator : 0; } @@ -400,10 +438,7 @@ function scopeFromPeriodDays( let approvalReviewerThreads = 0; for (const file of files) { - if (file.period?.timeZone !== timeZone) { - continue; - } - const slices = Object.entries(file.period.days) + const slices = periodSlicesForFile(file, timeZone) .filter(([day]) => selectedKeys.has(day)) .map(([, slice]) => slice); if (slices.length === 0) { @@ -508,10 +543,7 @@ function dailyRows( } >(); for (const file of files) { - if (file.period?.timeZone !== timeZone) { - continue; - } - for (const [day, slice] of Object.entries(file.period.days)) { + for (const [day, slice] of periodSlicesForFile(file, timeZone)) { const row = days.get(day) ?? { tokens: zeroTokens(), models: new Map(), @@ -562,10 +594,7 @@ function monthlyRows( } >(); for (const file of files) { - if (file.period?.timeZone !== timeZone) { - continue; - } - for (const [day, slice] of Object.entries(file.period.days)) { + for (const [day, slice] of periodSlicesForFile(file, timeZone)) { const period = day.slice(0, 7); if (!/^\d{4}-\d{2}$/.test(period)) { continue; @@ -719,10 +748,7 @@ function periodMembership( if (context.recentSessionKeys.has(sessionIdentityKey(file))) { result.push('recent'); } - if (file.period?.timeZone !== context.timeZone) { - return result; - } - const days = Object.keys(file.period.days); + const days = periodSlicesForFile(file, context.timeZone).map(([day]) => day); if (days.some((day) => context.last7DayKeys.has(day))) { result.push('7d'); } @@ -739,9 +765,9 @@ function dayMembership( file: CodexFileAggregate, context: CodexThreadPeriodContext, ): string[] { - return file.period?.timeZone === context.timeZone - ? Object.keys(file.period.days).sort() - : []; + return periodSlicesForFile(file, context.timeZone) + .map(([day]) => day) + .sort(); } function recentThreadRows( diff --git a/src/test/codexIndex.test.ts b/src/test/codexIndex.test.ts index 96f74cad..36f47f70 100644 --- a/src/test/codexIndex.test.ts +++ b/src/test/codexIndex.test.ts @@ -746,6 +746,39 @@ function tokenLine(input: number, output: number, timestamp: string): string { }); } +function tokenLineWithWeeklyLimit( + input: number, + output: number, + timestamp: string, + resetAt: string, + usedPercent: number, +): string { + return JSON.stringify({ + timestamp, + type: 'event_msg', + payload: { + type: 'token_count', + info: { + total_token_usage: { + input_tokens: input, + cached_input_tokens: Math.floor(input / 2), + output_tokens: output, + reasoning_output_tokens: Math.floor(output / 2), + total_tokens: input + output, + }, + }, + rate_limits: { + limit_id: 'codex', + secondary: { + used_percent: usedPercent, + window_minutes: 7 * 24 * 60, + resets_at: resetAt, + }, + }, + }, + }); +} + function structuralLine( timestamp: string, type: 'event_msg' | 'response_item', @@ -798,6 +831,153 @@ function chunkingIo(chunkBytes: number): CodexIndexIo { }; } +test('weekly quota history retains separate resets after the source file is removed', async () => { + const root = await mkdtemp(path.join(os.tmpdir(), 'ccu-codex-index-quota-history-')); + try { + const indexPath = path.join(root, 'codex-index.json'); + const sessions = path.join(root, 'sessions'); + await mkdir(sessions, { recursive: true }); + await writeFile( + path.join(sessions, 'quota-history.jsonl'), + [ + sessionLine('quota-history'), + contextLine(), + tokenLineWithWeeklyLimit( + 100, + 20, + '2026-08-28T08:00:00.000Z', + '2026-08-28T11:00:00.000Z', + 18, + ), + tokenLineWithWeeklyLimit( + 200, + 40, + '2026-08-30T04:00:00.000Z', + '2026-08-30T07:00:00.000Z', + 42, + ), + '', + ].join('\n'), + 'utf8', + ); + const manifest = await scanCodexManifest(root, SALT); + const now = Date.parse('2026-08-31T12:00:00.000Z'); + const cold = await updateCodexIndex(createEmptyCodexIndex('UTC'), manifest, { + salt: SALT, + timeZone: 'UTC', + now: () => now, + }); + + assert.deepEqual( + cold.index.quotaHistory?.map(({ resetAt, usedPercent }) => ({ resetAt, usedPercent })), + [ + { resetAt: Date.parse('2026-08-28T11:00:00.000Z'), usedPercent: 18 }, + { resetAt: Date.parse('2026-08-30T07:00:00.000Z'), usedPercent: 42 }, + ], + ); + + await saveCodexIndexAtomic(indexPath, { + ...cold.index, + quotaHistory: cold.index.quotaHistory?.map((observation) => ({ + ...observation, + account: 'must-not-persist', + })) as any, + }); + const reloaded = await loadCodexIndex(indexPath, 'UTC'); + assert.deepEqual( + reloaded.quotaHistory?.map(({ resetAt, usedPercent }) => ({ resetAt, usedPercent })), + [ + { resetAt: Date.parse('2026-08-28T11:00:00.000Z'), usedPercent: 18 }, + { resetAt: Date.parse('2026-08-30T07:00:00.000Z'), usedPercent: 42 }, + ], + ); + assert.doesNotMatch(await readFile(indexPath, 'utf8'), /must-not-persist/); + + const removed = await updateCodexIndex( + reloaded, + persistableManifest([]), + { salt: SALT, timeZone: 'UTC', now: () => now }, + ); + assert.equal(removed.bodyReads, 0); + assert.deepEqual( + removed.index.quotaHistory?.map(({ resetAt, usedPercent }) => ({ resetAt, usedPercent })), + [ + { resetAt: Date.parse('2026-08-28T11:00:00.000Z'), usedPercent: 18 }, + { resetAt: Date.parse('2026-08-30T07:00:00.000Z'), usedPercent: 42 }, + ], + ); + } finally { + await rm(root, { recursive: true, force: true }); + } +}); + +test('legacy indexed quota projections seed reset history once without rereading bodies', async () => { + const root = await mkdtemp(path.join(os.tmpdir(), 'ccu-codex-index-quota-seed-')); + try { + const sessions = path.join(root, 'sessions'); + await mkdir(sessions, { recursive: true }); + await writeFile( + path.join(sessions, 'quota-seed.jsonl'), + [ + sessionLine('quota-seed'), + contextLine(), + tokenLineWithWeeklyLimit( + 100, + 20, + '2026-08-30T04:00:00.000Z', + '2026-08-30T07:00:00.000Z', + 42, + ), + '', + ].join('\n'), + 'utf8', + ); + const manifest = await scanCodexManifest(root, SALT); + const now = Date.parse('2026-08-31T12:00:00.000Z'); + const cold = await updateCodexIndex(createEmptyCodexIndex('UTC'), manifest, { + salt: SALT, + timeZone: 'UTC', + now: () => now, + }); + const legacy = structuredClone(cold.index); + delete legacy.quotaHistory; + for (const contribution of Object.values(legacy.files)) { + // Model an older complete schema-3 index: it has the last observed + // limit projection, but predates the neutral reset-history cache. + delete contribution.quotaHistory; + } + + const io = trackingIo(); + const seeded = await updateCodexIndex(legacy, manifest, { + salt: SALT, + timeZone: 'UTC', + now: () => now, + io, + }); + + assert.equal(seeded.bodyReads, 0); + assert.equal(io.bodyReads.size, 0); + assert.deepEqual( + seeded.index.quotaHistory?.map(({ resetAt, usedPercent }) => ({ resetAt, usedPercent })), + [{ resetAt: Date.parse('2026-08-30T07:00:00.000Z'), usedPercent: 42 }], + ); + assert.equal(seeded.indexChanged, true); + + const warmIo = trackingIo(); + const warm = await updateCodexIndex(seeded.index, manifest, { + salt: SALT, + timeZone: 'UTC', + now: () => now, + io: warmIo, + }); + assert.equal(warm.indexChanged, false); + assert.equal(warm.bodyReads, 0); + assert.equal(warmIo.bodyReads.size, 0); + } finally { + await rm(root, { recursive: true, force: true }); + } +}); + test('fully indexed unchanged corpus returns a warm no-op without body reads', async () => { const root = await mkdtemp(path.join(os.tmpdir(), 'ccu-codex-index-noop-')); try { diff --git a/src/test/codexProvider.test.ts b/src/test/codexProvider.test.ts index 71006515..e22e5d86 100644 --- a/src/test/codexProvider.test.ts +++ b/src/test/codexProvider.test.ts @@ -520,6 +520,98 @@ test('provider snapshots promote exact period slices for scoped consumers', asyn } }); +test('an invalid period sidecar cannot create a false weekly priced estimate', async () => { + const root = await mkdtemp(path.join(os.tmpdir(), 'ccu-codex-provider-period-guard-')); + try { + await mkdir(path.join(root, 'sessions'), { recursive: true }); + const index = duplicateIndex(false); + const file = index.files['active-key']; + file.aggregate.period = { + timeZone: 'UTC', + indexedThrough: file.offset, + days: { + '2026-07-20': { + total: { inputTotal: 10_000_000_000, outputTotal: 5_000_000_000 }, + byModel: { + 'gpt-5.6-sol': { + inputTotal: 10_000_000_000, + outputTotal: 5_000_000_000, + }, + }, + byEffort: {}, + structural: file.aggregate.structural, + }, + }, + }; + + const provider = new CodexProvider( + { + enabled: true, + codexHome: root, + indexPath: 'index', + salt: 'salt', + timeZone: 'UTC', + }, + () => new FakeClient([{ ...workerResult(index), failedFiles: 0 }]), + ); + + const refreshed = await provider.refresh(); + + assert.equal(refreshed.snapshot.total.inputTotal, 80); + assert.deepEqual(refreshed.snapshot.weeklyValueInputs?.usage, []); + } finally { + await rm(root, { recursive: true, force: true }); + } +}); + +test('a period with a mismatched lineage marker stays hidden during rebuild', async () => { + const root = await mkdtemp(path.join(os.tmpdir(), 'ccu-codex-provider-period-marker-')); + try { + await mkdir(path.join(root, 'sessions'), { recursive: true }); + const index = duplicateIndex(false); + const file = index.files['active-key']; + file.lineage = { + fingerprintBlocks: [], + pendingFingerprints: [], + tokenEvents: 0, + desiredPrefixEvents: 0, + appliedPrefixEvents: 0, + }; + file.aggregate.period = { + timeZone: 'UTC', + indexedThrough: file.offset, + lineageVersion: 1, + lineagePrefixEvents: 1, + days: { + '2026-07-20': { + total: { ...file.aggregate.total }, + byModel: { ...file.aggregate.byModel }, + byEffort: { ...file.aggregate.byEffort }, + structural: file.aggregate.structural, + }, + }, + }; + + const provider = new CodexProvider( + { + enabled: true, + codexHome: root, + indexPath: 'index', + salt: 'salt', + timeZone: 'UTC', + }, + () => new FakeClient([{ ...workerResult(index), failedFiles: 0 }]), + ); + + const refreshed = await provider.refresh(); + + assert.equal(refreshed.snapshot.files[0].period, undefined); + assert.deepEqual(refreshed.snapshot.weeklyValueInputs?.usage, []); + } finally { + await rm(root, { recursive: true, force: true }); + } +}); + test('a worker failure retains the last verified provider snapshot', async () => { const root = await mkdtemp(path.join(os.tmpdir(), 'ccu-codex-provider-stale-')); try { diff --git a/src/test/codexQuotaHistory.test.ts b/src/test/codexQuotaHistory.test.ts new file mode 100644 index 00000000..ca03218e --- /dev/null +++ b/src/test/codexQuotaHistory.test.ts @@ -0,0 +1,127 @@ +import { test } from 'node:test'; +import * as assert from 'node:assert/strict'; + +import { + appendCodexQuotaHistory, + CODEX_QUOTA_HISTORY_LIMIT, + codexQuotaObservationsFromLimit, + weeklyQuotaObservationsFromCodexHistory, +} from '../providers/codex/codexQuotaHistory'; +import { ProviderLimitSnapshot } from '../providers/providerTypes'; + +function limit( + observedAt: number, + resetAt: number, + usedPercent: number, + options: { + limitId?: string; + windowMinutes?: number; + } = {}, +): ProviderLimitSnapshot { + return { + provider: 'codex', + ...(options.limitId ? { limitId: options.limitId } : {}), + observedAt, + source: 'local-log', + confidence: 'last-observed', + windows: [{ + label: 'secondary', + usedPercent, + windowMinutes: options.windowMinutes ?? 7 * 24 * 60, + resetsAt: resetAt, + }], + }; +} + +test('only account-wide weekly Codex windows become neutral reset evidence', () => { + const resetAt = Date.parse('2026-08-30T00:00:00.000Z'); + const observedAt = resetAt - 60 * 60 * 1000; + + assert.deepEqual( + codexQuotaObservationsFromLimit(limit(observedAt, resetAt, 42)), + [{ + provider: 'codex', + seriesKey: 'codex', + observedAt, + resetAt, + usedPercent: 42, + }], + ); + assert.deepEqual( + codexQuotaObservationsFromLimit( + limit(observedAt, resetAt, 42, { windowMinutes: 300 }), + ), + [], + ); + assert.deepEqual( + codexQuotaObservationsFromLimit( + limit(observedAt, resetAt, 42, { limitId: 'codex-spark' }), + ), + [], + ); +}); + +test('irregular reset boundaries survive polling compaction', () => { + const firstReset = Date.parse('2026-08-28T11:00:00.000Z'); + const secondReset = Date.parse('2026-08-30T07:00:00.000Z'); + const history = appendCodexQuotaHistory([], [ + { + provider: 'codex', + seriesKey: 'codex', + observedAt: firstReset - 3 * 60 * 60 * 1000, + resetAt: firstReset, + usedPercent: 12, + }, + { + provider: 'codex', + seriesKey: 'codex', + observedAt: firstReset - 30 * 60 * 1000, + resetAt: firstReset, + usedPercent: 38, + }, + { + provider: 'codex', + seriesKey: 'codex', + observedAt: secondReset - 2 * 60 * 60 * 1000, + resetAt: secondReset, + usedPercent: 7, + }, + ]); + + assert.deepEqual(history, [ + { + provider: 'codex', + seriesKey: 'codex', + observedAt: firstReset - 30 * 60 * 1000, + resetAt: firstReset, + usedPercent: 38, + }, + { + provider: 'codex', + seriesKey: 'codex', + observedAt: secondReset - 2 * 60 * 60 * 1000, + resetAt: secondReset, + usedPercent: 7, + }, + ]); + + const weekly = weeklyQuotaObservationsFromCodexHistory(history); + assert.equal(weekly.length, 2); + assert.equal(weekly.every((item) => item.sourceKey === undefined), true); + assert.equal(Object.prototype.hasOwnProperty.call(weekly[0], 'account'), false); + assert.equal(Object.prototype.hasOwnProperty.call(weekly[0], 'sourceKey'), false); +}); + +test('quota history remains bounded even when resets are numerous', () => { + const additions = Array.from({ length: CODEX_QUOTA_HISTORY_LIMIT + 17 }, (_, index) => ({ + provider: 'codex' as const, + seriesKey: 'codex' as const, + observedAt: 1_700_000_000_000 + index * 7 * 24 * 60 * 60 * 1000, + resetAt: 1_700_000_100_000 + index * 7 * 24 * 60 * 60 * 1000, + usedPercent: index % 101, + })); + + const history = appendCodexQuotaHistory([], additions); + assert.equal(history.length, CODEX_QUOTA_HISTORY_LIMIT); + assert.equal(history[0].resetAt < history[history.length - 1].resetAt, true); +}); diff --git a/src/test/codexUsage.test.ts b/src/test/codexUsage.test.ts index 2c3ad3ef..39ddeb35 100644 --- a/src/test/codexUsage.test.ts +++ b/src/test/codexUsage.test.ts @@ -84,6 +84,49 @@ test('partial period coverage disables unreliable session membership ranges', () }); }); +test('an inflated period sidecar falls back to the verified daily aggregate', () => { + const snapshot = snapshotFixture(); + const file = snapshot.files[0]; + const safeDay = Object.keys(file.byDay)[0]; + const safeDailyTotal = file.byDay[safeDay]; + file.period = { + ...file.period!, + days: { + [safeDay]: { + ...file.period!.days[safeDay], + total: { + inputTotal: 10_000_000_000, + outputTotal: 5_000_000_000, + }, + }, + }, + }; + snapshot.files = [file]; + snapshot.total = { ...file.total }; + + const view = buildCodexUsageView(snapshot, NOW); + const daily = view.daily.find((row) => row.day === safeDay); + + assert.equal( + daily?.total.processed, + safeDailyTotal.inputTotal + safeDailyTotal.outputTotal, + ); + assert.equal( + view.allTime.total.processed, + file.total.inputTotal + file.total.outputTotal, + ); + assert.equal(view.last30Days.total.processed <= view.allTime.total.processed, true); + assert.deepEqual(daily?.apiEquivalent, { + equivalentUsd: 0, + freshInputUsd: 0, + cachedInputUsd: 0, + outputUsd: 0, + pricedTokens: 0, + totalTokens: safeDailyTotal.inputTotal + safeDailyTotal.outputTotal, + pricingCoverage: 0, + }); +}); + test('index backfill quality warning follows index convergence', () => { const snapshot = snapshotFixture(); const incomplete = buildCodexUsageView(snapshot, NOW); @@ -415,6 +458,18 @@ test('daily and monthly Codex rows retain exact-model API-equivalent cost and co }, }, }; + // The period projection is now required to be a subset of the verified + // file aggregate. Keep this synthetic cost fixture internally consistent so + // it exercises pricing rather than the corrupted-period fallback. + file.total = { + inputTotal: 5_000_000, + // Both period days are part of this file's verified aggregate. Keep the + // cache bucket consistent as well so the test exercises pricing rather + // than the corrupted-period fallback. + cachedInput: 2_000_000, + outputTotal: 2_500_000, + reasoningOutput: 1_750_000, + }; snapshot.files = [file]; const view = buildCodexUsageView(snapshot, NOW); diff --git a/src/test/repositoryPolicy.test.ts b/src/test/repositoryPolicy.test.ts index 382e04a3..de2d913d 100644 --- a/src/test/repositoryPolicy.test.ts +++ b/src/test/repositoryPolicy.test.ts @@ -202,12 +202,14 @@ function functionDeclarations(file: ts.SourceFile): ReadonlyMap(); + const trustedSanitizerBindings = new Set(); for (const statement of file.statements) { if ( !ts.isImportDeclaration(statement) || !ts.isStringLiteral(statement.moduleSpecifier) || - statement.moduleSpecifier.text !== './codexMetadataLabel' || + !['./codexMetadataLabel', './codexQuotaHistory'].includes( + statement.moduleSpecifier.text, + ) || !statement.importClause || statement.importClause.isTypeOnly || !statement.importClause.namedBindings || @@ -217,8 +219,14 @@ function sanitizedDtoViolations(file: ts.SourceFile): string[] { } for (const specifier of statement.importClause.namedBindings.elements) { const exportedName = specifier.propertyName?.text ?? specifier.name.text; - if (!specifier.isTypeOnly && exportedName === 'sanitizeCodexMetadataLabel') { - trustedLabelSanitizerBindings.add(specifier.name.text); + if ( + !specifier.isTypeOnly && + ( + exportedName === 'sanitizeCodexMetadataLabel' || + exportedName === 'sanitizeCodexQuotaHistory' + ) + ) { + trustedSanitizerBindings.add(specifier.name.text); } } } @@ -250,7 +258,7 @@ function sanitizedDtoViolations(file: ts.SourceFile): string[] { if (ts.isIdentifier(call.expression)) { return call.expression.text === 'resolveTimeZone' || call.expression.text === 'dayKeyInZone' || - (trustedLabelSanitizerBindings.has(call.expression.text) && + (trustedSanitizerBindings.has(call.expression.text) && !shadowedBindings.has(call.expression.text)); } if (!ts.isPropertyAccessExpression(call.expression)) { From 7fd03f3183240940d0a2d1cfe76f60cb80f0a6ae Mon Sep 17 00:00:00 2001 From: Carl <74130028+Carl723000@users.noreply.github.com> Date: Mon, 31 Aug 2026 18:23:50 +0800 Subject: [PATCH 18/72] fix: estimate Codex weekly allowance from observed usage Treat account-wide Codex quota observations as usable evidence across local log files, while lowering confidence for reset and boundary uncertainty. Keep current unused value withheld, preserve bounded reset history, and render Codex monthly history chronologically. Co-authored-by: OpenAI Codex <215057067+openai-codex[bot]@users.noreply.github.com> --- ARCHITECTURE-zh-CN.md | 11 +++-- ARCHITECTURE.md | 17 ++++--- CHANGELOG.md | 37 +++++++++------ README-en.md | 4 +- README-id.md | 4 +- README-ja.md | 4 +- README-ko.md | 4 +- README-zh-CN.md | 4 +- README-zh-TW.md | 4 +- README.md | 31 ++++++------ src/i18n.ts | 16 +++---- src/providers/codex/codexUsage.ts | 4 +- src/test/codexUsage.test.ts | 9 ++-- src/test/weeklyValue.test.ts | 76 +++++++++++++++++++++++------- src/webview.ts | 4 +- src/weeklyValue.ts | 78 ++++++++++++++++++++++--------- tests/ui/codex-visual.spec.mjs | 2 +- 17 files changed, 207 insertions(+), 102 deletions(-) diff --git a/ARCHITECTURE-zh-CN.md b/ARCHITECTURE-zh-CN.md index a89f822a..4a90a2de 100644 --- a/ARCHITECTURE-zh-CN.md +++ b/ARCHITECTURE-zh-CN.md @@ -236,10 +236,13 @@ watcher、worker、network request 与 backfill 的创建者,只在真实 stop 为降低首次可用延迟,有界的首次索引可在失焦后继续;但 extension dispose、provider 关闭或显式取消 仍对其终止负责。 -每周 API 等效价值历史只从已经聚合的 Token 用量生成。有真实每周重置观测时,七天窗口按该重置 -对齐;没有时,仅已用历史按 UTC 周一至周一的自然周分组。Token 日志能够证明已用价值,但不能证明 -历史订阅总额度,因此总额度与未用额度仅在存在真实额度用量比例观测时生成。Codex 的仅已用历史可能 -合并同一 home 中的多个登录,而额度推算行仍绑定其实际观测的重置序列。 +每周 API 等效价值历史只从已经聚合的 Token 用量生成。最新有效的重置观测用于锚定互不重叠的七天 +展示窗口;没有观测时,仅已用历史按 UTC 周一至周一的自然周分组。真实的额度用量比例观测可以支持 +历史窗口的总额度近似,而不只支持当前窗口。Codex 的 account-wide `codex` series 会纳入同一 home +中所有符合条件的本地文件,因为文件键不是账号身份。重置时间如果偏离七天网格,则按观测时间映射到 +对应展示窗口;来源不确定、重置漂移或日切片跨边界时降低可信度并标注为近似。真正不同的 quota +series 仍仅显示已用值;当前周期不显示未用值,历史周期只有在同时存在总额度近似时才显示未用值。 +每条用量仍只进入一个窗口。 v2.3.0 不推断 20、100 或 200 美元的订阅档位。本地 Codex 日志没有可靠的账号与套餐身份, 后续比较必须采用用户明确选择的 opt-in 账号映射,不能猜测后绑定价格。 diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index 8660b367..0a82a2fd 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -310,12 +310,17 @@ first-use latency, but extension disposal, provider disable, or explicit cancellation still owns its termination. Weekly API-equivalent history is derived from already-aggregated token usage. -Observed weekly resets align seven-day buckets; otherwise usage-only history -uses Monday-to-Monday UTC calendar weeks. A token log can prove used value, but -not an historical subscription capacity: full and unused estimates are emitted -only where a real quota-utilization sample exists. Codex usage-only history may -combine multiple sign-ins in one home, while quota-derived rows remain bound to -their observed reset series. +The newest valid reset observation anchors non-overlapping seven-day display +buckets; without one, usage-only history uses Monday-to-Monday UTC calendar +weeks. A real utilization sample can support a full-window estimate, including +for historical buckets. For Codex, the account-wide `codex` series is allowed to +use all eligible local files in the same home because file keys are not account +identities. A reset that drifts from the seven-day grid is mapped by observation +time to the corresponding display bucket; source uncertainty, reset drift, and +daily slices crossing a boundary lower confidence and are rendered as an +approximation. A genuinely different quota series remains usage-only. Current +unused value is withheld, while historical unused value is shown only alongside +a full estimate. Each usage row still contributes to exactly one bucket. v2.3.0 does not infer a $20, $100, or $200 subscription tier. Local Codex logs do not expose a reliable account-and-plan identity, so a future comparison must diff --git a/CHANGELOG.md b/CHANGELOG.md index dc29032f..e214712f 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -50,8 +50,12 @@ upstream release: 1.0.8). Format follows [Keep a Changelog](https://keepachangel The existing index pass also keeps a small, account-neutral history of account-wide weekly reset observations, preserving irregular reset boundaries across file replacement or removal without polling, credentials, or a second - scanner. Ambiguous history remains used-value-only rather than producing a - guessed allowance. + scanner. The weekly-value timeline now treats Codex file keys as local-source + evidence rather than account identities: the same account-wide `codex` series + can estimate historical and current full values from all eligible files. Reset + drift and daily boundary crossings are mapped conservatively and lower + confidence; genuinely different quota series remain used-value-only. Codex + monthly charts and tables now list months oldest-first. ### Privacy and packaging - Disabled or unconsented advice adds no timer, watcher, worker, network request, @@ -76,16 +80,19 @@ upstream release: 1.0.8). Format follows [Keep a Changelog](https://keepachangel - **Weekly API-equivalent periods no longer overlap or double-count usage** — the newest valid official reset observation anchors one sequence of unique `[start, reset)` weekly buckets, so each local usage event contributes to - exactly one period. Any overlapping, non-aligned future reset is treated as a - conflict even when its series name differs; it cannot create an additional + exactly one period. A genuinely different quota series with an overlapping, + non-aligned future reset is treated as a conflict; same-series observations + remain one low-confidence series rather than creating an additional "current" row. Codex usage is persisted in daily slices, so a slice that crosses an official intraday reset remains counted once but marks the affected - period as a boundary approximation and suppresses total / unused allowance - inference. This display-only correction does not change the index schema or - trigger a rebuild. Historical Codex periods are always used-value-only; only - the newest current period may infer a total when its reset is unambiguous and - its indexed usage can be attributed to one observation source. Current unused - value remains withheld, and ambiguous multi-sign-in usage stays used-only. + period as a boundary approximation and lowers confidence on any full + allowance estimate. Drifted reset observations are mapped by observation time + to the corresponding display bucket. This display-only correction does not + change the index schema or trigger a rebuild. Same-series Codex observations + can estimate historical as well as current full values from all eligible local + files; current unused value remains withheld, historical unused value requires + a full estimate, and genuinely different quota series or missing evidence stays + used-only. - **Claude changed-file refreshes no longer reread the full corpus (#87)** — the production refresh path now keeps an exact in-memory per-file index, reads only a verified append tail, and rebuilds only affected files for @@ -148,10 +155,12 @@ upstream release: 1.0.8). Format follows [Keep a Changelog](https://keepachangel views now calculate historical used API-equivalent value directly from local token logs. One valid observed reset anchors unique, non-overlapping weekly buckets; without one, usage-only rows use Monday-to-Monday UTC calendar weeks. - Codex historical periods, boundary-approximate daily slices, conflicting reset - sequences, and usage that cannot be assigned to one observed sign-in remain - used-value-only. Only an unambiguous newest current period with single-source - attribution may infer a total allowance; its unused value is still withheld. + Codex account-wide observations can decorate historical and current buckets; + reset drift is mapped by observation time, while source ambiguity and boundary + slices lower confidence. Genuinely conflicting quota series and periods with + no usable observation remain used-value-only. Current unused value is still + withheld, while historical unused value is shown only when a full estimate is + available. Current official API prices are applied consistently and each period includes model-price coverage. This remains an estimate, not a bill or an official subscription price. The panel is enabled by default and can be hidden with diff --git a/README-en.md b/README-en.md index 6250bbf2..3b9de533 100644 --- a/README-en.md +++ b/README-en.md @@ -51,9 +51,9 @@ The package version remains unchanged; this is a local validation candidate, not - Codex usage records are discovered only from `sessions/**/*.jsonl` and `archived_sessions/**/*.jsonl`; credential, database, and unknown files remain excluded. Separately, the extension streams exactly `$CODEX_HOME/session_index.jsonl` to map `id` to `thread_name` for truthful thread titles. Absolute paths are redacted and titles remain memory-only. Usage-record JSONL lines are streamed and temporarily parsed only to extract allowlisted usage and structural metadata; prompt, response, command, and tool-argument fields are not inspected or used for analysis, and are never retained or persisted. - **Processed** means input + output, **uncached usage** means uncached input + output, **cached input** remains a subset of input, and reasoning remains a subset of output. The overview also shows input cache hit rate as cached input / input. Codex billing cost is not shown. Its first summary card is a clearly labelled API-equivalent cost estimate for the selected scope, and the All-time view shows the weekly trend on the same pricing basis; Claude / Codex / Compare keep each provider's accounting separate. -- Codex **Today** means the current calendar day in the configured timezone. It adds exact hourly API-equivalent cost beside a separate token-composition view; daily and monthly primary charts also default to API-equivalent cost while token composition stays separately visible. Only exact known-model prices contribute, so unknown models remain unpriced and pricing coverage stays visible. The additive, schema-3-compatible hourly sidecar keeps sparse buckets for the rolling last 30 days, is checkpointed and resumable, evicts day 31, and serves date expansion without reading JSONL on click. +- Codex **Today** means the current calendar day in the configured timezone. It adds exact hourly API-equivalent cost beside a separate token-composition view; daily and monthly primary charts also default to API-equivalent cost while token composition stays separately visible. Only exact known-model prices contribute, so unknown models remain unpriced and pricing coverage stays visible. The additive, schema-3-compatible hourly sidecar keeps sparse buckets for the rolling last 30 days, is checkpointed and resumable, evicts day 31, and serves date expansion without reading JSONL on click. Codex monthly charts and tables list months oldest-first. - Request-level attribution prefers valid `last_token_usage` components; its `total_tokens` is active-context size, not request usage. A full numeric total-plus-last signature suppresses only proven replay from the same pseudonymous rate-limit source or an immediately adjacent duplicate. Missing last snapshots fall back to cumulative lineage high-water. Upgrading triggers one automatic reindex, while the indexed subtotal remains visible throughout the pass. -- Claude and Codex All-time / Compare views calculate historical used equivalents directly from local token logs. The newest valid official reset observation anchors one sequence of unique, non-overlapping weekly periods, so each usage event is counted once; without a usable observation, usage-only rows fall back to Monday-to-Monday UTC calendar weeks. Any overlapping, non-aligned future reset is a conflict even when its series name differs; it cannot create another current period, and period ranges are shown separately from reset times. Codex usage is stored in daily slices: when one crosses an official intraday reset, its tokens are still counted once, the affected period is labelled a boundary approximation, and only used equivalent is shown. This display rule does not change the index schema or trigger a rebuild. Codex historical periods are always used-value-only. Only the newest current period may infer a total when the reset is unambiguous and indexed usage can be attributed to one observation source; current unused value remains withheld. Multi-sign-in usage that cannot be attributed reliably also stays used-only, without inventing an account split. Current official API rates are applied consistently across history. This is a proxy, not a bill or an official subscription price. The panel is enabled by default and can be hidden in Settings with `showWeeklyEquivalentValue`. Claude records profile-scoped quota observations from this release onward. +- Claude and Codex All-time / Compare views calculate historical used equivalents directly from local token logs. The newest valid official reset observation anchors one sequence of unique, non-overlapping weekly periods, so each usage event is counted once; without a usable observation, usage-only rows fall back to Monday-to-Monday UTC calendar weeks. A genuinely different quota series with an overlapping, non-aligned future reset is a conflict; same-series observations remain one series and may be shown as a low-confidence approximation. It cannot create another current period, and period ranges are shown separately from reset times. Codex usage is stored in daily slices, and an account-wide `codex` observation may decorate both historical and current buckets. If an observed reset drifts from the seven-day grid, the sample is mapped to the display period containing its observation time; file-source uncertainty, reset drift, and daily boundary crossings lower confidence and are labelled approximate. File keys are not account identities, so eligible local files in one home are included together. A genuinely different quota series, or a period without a usable observation, stays used-only rather than inventing an account split. Current unused value remains withheld; historical unused value appears only when a full estimate exists. This display rule does not change the index schema or trigger a rebuild. Current official API rates are applied consistently across history. This is a proxy, not a bill or an official subscription price. The panel is enabled by default and can be hidden in Settings with `showWeeklyEquivalentValue`. Claude records profile-scoped quota observations from this release onward. - Switching to Codex keeps the established Today / Month / All time / Sessions / Projects / Content / Settings structure, relabelled where Codex semantics differ. Both providers use the same render functions, HTML classes, charts, tables, spacing, and responsive rules; their time-series charts stay width-aligned while dense content scrolls inside its own region. Codex recommendations use indexed 30-day structural evidence. - Root tasks use the latest real thread title after path redaction. Child rows prefer their own real thread title; when it is missing, they use the reported nickname and display the parent/root title; if those are also missing, they receive a localized neutral fallback. Project names use the Git repository name, or the directory basename outside Git. The extension does not invent Branches or Workflows that cannot be measured reliably. - Rolling 7-day and 30-day values use exact event-day slices in the configured timezone. During an incomplete migration or rebuild, every Codex card, table, project, session, recommendation, and status value is visibly an **indexed subtotal**; unverified legacy totals are excluded. Once a Codex home is detected, its provider tab appears immediately; the page shows exact indexed-file, percentage, and byte progress during the first build, while Compare waits until both providers have real data. After the first atomic checkpoint, the complete subtotal dashboard stays usable while indexing continues; progress never replaces its cards or tables. A selected Codex home is account-agnostic, so logs left by multiple sign-ins in that same home are combined. Local logs expose no reliable account identity, so limit cards remain **last-observed** and are never summed. diff --git a/README-id.md b/README-id.md index 9141f1f5..3838259f 100644 --- a/README-id.md +++ b/README-id.md @@ -51,9 +51,9 @@ Versi paket tetap tidak berubah; ini kandidat validasi lokal, bukan rilis yang d - Catatan penggunaan Codex hanya ditemukan dari `sessions/**/*.jsonl` dan `archived_sessions/**/*.jsonl`; file kredensial, database, dan file tak dikenal tetap dikecualikan. Secara terpisah, ekstensi hanya melakukan streaming terhadap `$CODEX_HOME/session_index.jsonl` untuk memetakan `id` ke `thread_name` bagi judul thread yang sebenarnya. Jalur absolut dalam judul disamarkan dan judul hanya disimpan di memori. Setiap baris JSONL penggunaan di-stream dan diparse sementara hanya untuk mengambil metadata penggunaan dan struktur dalam daftar izin; field prompt, respons, perintah, dan argumen alat tidak diperiksa atau dipakai untuk analisis, serta tidak pernah disimpan atau dipersistenkan. - **Diproses** = input + output, **Penggunaan tanpa cache** = Input tanpa cache + output, **cached input** tetap bagian dari input, dan reasoning bagian dari output. Ringkasan juga menampilkan tingkat hit cache input sebagai cached input / input. Biaya tagihan Codex tidak ditampilkan. Kartu ringkasan pertama menampilkan estimasi biaya ekuivalen API untuk cakupan terpilih dengan label yang jelas, sedangkan tampilan Sepanjang Waktu menunjukkan tren mingguan dengan dasar harga yang sama. Claude / Codex / Compare tetap memisahkan makna tiap penyedia. -- **Hari Ini** pada Codex berarti hari kalender saat ini dalam zona waktu yang dikonfigurasi. Tampilan ini menambahkan biaya ekuivalen API per jam yang tepat di samping tampilan komposisi Token yang terpisah. Grafik utama harian dan bulanan juga memakai biaya ekuivalen API sebagai default, sementara komposisi Token tetap tersedia secara terpisah. Hanya harga model yang dikenal dan cocok tepat yang dihitung; model tak dikenal tetap tanpa harga dan cakupan harga selalu terlihat. Sidecar per jam yang bersifat tambahan dan kompatibel dengan schema 3 menyimpan bucket tanggal / jam yang jarang untuk 30 hari bergulir, mendukung checkpoint dan resume, membuang hari ke-31, dan membuka tanggal tanpa membaca JSONL. +- **Hari Ini** pada Codex berarti hari kalender saat ini dalam zona waktu yang dikonfigurasi. Tampilan ini menambahkan biaya ekuivalen API per jam yang tepat di samping tampilan komposisi Token yang terpisah. Grafik utama harian dan bulanan juga memakai biaya ekuivalen API sebagai default, sementara komposisi Token tetap tersedia secara terpisah. Hanya harga model yang dikenal dan cocok tepat yang dihitung; model tak dikenal tetap tanpa harga dan cakupan harga selalu terlihat. Sidecar per jam yang bersifat tambahan dan kompatibel dengan schema 3 menyimpan bucket tanggal / jam yang jarang untuk 30 hari bergulir, mendukung checkpoint dan resume, membuang hari ke-31, dan membuka tanggal tanpa membaca JSONL. Grafik dan tabel bulanan Codex menampilkan bulan dari yang paling lama ke yang terbaru. - Atribusi Token per permintaan memprioritaskan component `last_token_usage` yang valid; `total_tokens` di dalamnya adalah ukuran konteks aktif, bukan penggunaan permintaan. Signature numerik lengkap total-plus-last hanya menghapus replay yang terbukti berasal dari sumber rate-limit pseudonim yang sama atau record yang tepat bersebelahan. Jika last snapshot tidak ada, parser kembali ke cumulative lineage high-water. Setelah upgrade, indeks dibangun ulang otomatis satu kali dan subtotal terindeks tetap terlihat selama prosesnya. -- Tampilan Sepanjang Waktu / Perbandingan Claude dan Codex menghitung ekuivalen terpakai historis langsung dari log Token lokal. Pengamatan reset resmi terbaru yang valid menjadi satu jangkar bagi rangkaian periode mingguan yang unik dan tidak tumpang tindih, sehingga setiap peristiwa penggunaan hanya dihitung dalam satu periode. Jika tidak ada pengamatan yang dapat dipakai, riwayat khusus pemakaian kembali ke minggu kalender UTC Senin-ke-Senin. Reset masa depan yang tumpang tindih tetapi tidak selaras dianggap konflik meskipun nama series berbeda; reset itu tidak dapat membuat periode berjalan kedua. Rentang periode dan waktu reset ditampilkan terpisah. Penggunaan Codex disimpan sebagai irisan harian. Bila suatu irisan melewati reset resmi di tengah hari, Token tetap dihitung satu kali, periode terdampak diberi label 「perkiraan batas」, dan hanya ekuivalen terpakai yang ditampilkan tanpa menyimpulkan total batas atau sisa. Aturan tampilan ini tidak mengubah schema indeks dan tidak memicu pembangunan ulang. Periode historis Codex selalu hanya menampilkan nilai terpakai. Hanya periode berjalan terbaru yang boleh memperkirakan total batas bila reset tidak berkonflik dan penggunaan terindeks dapat diatribusikan dengan andal ke satu sumber pengamatan; nilai sisa periode berjalan tetap tidak ditampilkan sebagai kesimpulan. Penggunaan dari beberapa login yang tidak dapat diatribusikan juga tetap hanya menampilkan nilai terpakai tanpa mengarang pemisahan akun. Harga API resmi saat ini diterapkan konsisten pada riwayat. Ini proksi, bukan tagihan atau harga langganan resmi. Panel ini aktif secara default dan dapat disembunyikan di Settings dengan `showWeeklyEquivalentValue`. Claude mulai merekam pengamatan batas per profil sejak rilis ini. +- Tampilan Sepanjang Waktu / Perbandingan Claude dan Codex menghitung ekuivalen terpakai historis langsung dari log Token lokal. Pengamatan reset resmi terbaru yang valid menjadi jangkar bagi rangkaian periode mingguan yang unik dan tidak tumpang tindih, sehingga setiap peristiwa penggunaan hanya dihitung dalam satu periode. Jika tidak ada pengamatan yang dapat dipakai, riwayat khusus pemakaian kembali ke minggu kalender UTC Senin-ke-Senin. Reset masa depan yang tumpang tindih tetapi tidak selaras dianggap konflik hanya jika berasal dari series kuota yang benar-benar berbeda; pengamatan dari series yang sama tetap menjadi satu series dan dapat ditampilkan sebagai perkiraan berkeyakinan rendah. Reset itu tidak dapat membuat periode berjalan kedua. Rentang periode dan waktu reset ditampilkan terpisah. Pengamatan account-wide `codex` Codex dapat menghias periode historis dan berjalan. Jika waktu reset bergeser dari grid tujuh hari, sampel dipetakan ke periode tampilan yang memuat waktu pengamatan. Ketidakpastian sumber file, pergeseran reset, atau irisan harian yang melintasi batas tetap dapat menghasilkan perkiraan, tetapi menurunkan keyakinan dan diberi label pendekatan. Kunci file bukan identitas akun, sehingga file lokal yang memenuhi syarat dalam satu home dipakai bersama. Series kuota yang benar-benar berbeda atau periode tanpa pengamatan yang dapat dipakai tetap hanya menampilkan ekuivalen terpakai tanpa mengarang pemisahan akun. Nilai sisa saat ini disembunyikan; nilai sisa historis hanya ditampilkan bila ada estimasi total. Penggunaan Codex disimpan sebagai irisan harian dan setiap Token tetap dihitung satu kali. Aturan tampilan ini tidak mengubah schema indeks dan tidak memicu pembangunan ulang. Harga API resmi saat ini diterapkan konsisten pada riwayat. Ini proksi, bukan tagihan atau harga langganan resmi. Panel ini aktif secara default dan dapat disembunyikan di Settings dengan `showWeeklyEquivalentValue`. Claude mulai merekam pengamatan batas per profil sejak rilis ini. - Beralih ke Codex tetap memakai struktur Hari Ini / Bulan Ini / Sepanjang Waktu / Sesi / Proyek / Konten / Settings yang sudah ada, dengan label yang disesuaikan pada makna Codex. Kedua penyedia memakai render function, HTML class, grafik, tabel, jarak, dan aturan responsif yang sama; grafik deret waktunya tetap sejajar lebarnya, sedangkan konten padat hanya bergulir di areanya sendiri. Rekomendasi Codex memakai bukti struktural 30 hari yang telah diindeks. - Tugas utama memakai judul thread nyata terbaru setelah jalur disamarkan. Baris tugas anak memprioritaskan judul thread nyatanya sendiri; jika tidak ada, baris memakai nama panggilan yang dilaporkan sambil menampilkan judul induk / tugas utama. Jika informasi itu juga tidak ada, baris memakai nama pengganti netral yang dilokalkan. Nama proyek memakai nama repositori Git, atau nama folder di luar Git. Ekstensi tidak mengarang Branches atau Workflows yang tidak dapat dihitung secara andal. - Nilai 7 dan 30 hari memakai irisan tepat berdasarkan hari peristiwa di zona waktu yang dikonfigurasi. Saat migrasi atau pembangunan ulang belum selesai, setiap kartu, tabel, proyek, sesi, rekomendasi, dan nilai status Codex ditandai sebagai **subtotal terindeks**, sementara total lama yang belum diverifikasi dikecualikan. Setelah Codex home terdeteksi, tab penyedianya langsung ditampilkan; selama indeks pertama dibuat, jumlah file terindeks yang tepat, persentase, dan progres kapasitas tampil langsung di halaman, sedangkan Perbandingan menunggu hingga kedua penyedia memiliki data nyata. Setelah checkpoint atomik pertama dibuat, seluruh dasbor subtotal tetap dapat digunakan saat pengindeksan berlanjut; indikator progres tidak menggantikan kartu atau tabel. Codex home yang dipilih tidak membedakan akun, jadi log dari beberapa login di home yang sama digabung. Log lokal tidak memiliki identitas akun yang andal; kartu batas tetap **terakhir diamati** dan tidak dijumlahkan. diff --git a/README-ja.md b/README-ja.md index 6b5d3e60..3891bd75 100644 --- a/README-ja.md +++ b/README-ja.md @@ -51,9 +51,9 @@ - Codex の使用量レコードは `sessions/**/*.jsonl` と `archived_sessions/**/*.jsonl` からだけ検出し、認証情報、DB、未知のファイルは対象外です。これとは別に、`$CODEX_HOME/session_index.jsonl` だけをストリームで読み込み、実際のスレッド名に使う `id` → `thread_name` の対応を取得します。名前に含まれる絶対パスは伏せ、名前はメモリ内だけに保持します。使用量 JSONL の各行は、許可リスト内の使用量・構造メタデータを抽出するためだけにストリーム処理で一時解析します。プロンプト、応答、コマンド、ツール引数の各フィールドは参照も分析利用もせず、保持・永続化もしません。 - **処理済み** = 入力 + 出力、**非キャッシュ使用量** = 非キャッシュ入力 + 出力、**キャッシュ入力**は入力の部分集合、reasoning は出力の部分集合です。概要には「キャッシュ入力 / 入力」による入力キャッシュヒット率も表示します。Codex の請求コストは表示しません。先頭のサマリーカードには選択範囲の API 等価コスト推定を明示し、全期間ビューには同じ単価基準の週間推移を表示します。Claude / Codex / Compare は各プロバイダーの集計方法を分けて扱います。 -- Codex の**今日**は、設定したタイムゾーンの現在の暦日です。正確な時間別 API 等価コストを表示し、Token 構成は独立したビューとして残します。日別・月別のメインチャートも API 等価コストを既定表示にし、Token 構成は別に確認できます。正確に一致する既知モデルだけを価格計算に含めるため、未知モデルは価格未設定のままで、価格カバレッジは常に表示されます。schema 3 と互換性のある追加型の時間別 sidecar は、直近 30 日の疎な日付 / 時間バケットを保持し、checkpoint 保存と再開、31 日目の削除に対応します。日付を展開しても JSONL は読みません。 +- Codex の**今日**は、設定したタイムゾーンの現在の暦日です。正確な時間別 API 等価コストを表示し、Token 構成は独立したビューとして残します。日別・月別のメインチャートも API 等価コストを既定表示にし、Token 構成は別に確認できます。正確に一致する既知モデルだけを価格計算に含めるため、未知モデルは価格未設定のままで、価格カバレッジは常に表示されます。schema 3 と互換性のある追加型の時間別 sidecar は、直近 30 日の疎な日付 / 時間バケットを保持し、checkpoint 保存と再開、31 日目の削除に対応します。日付を展開しても JSONL は読みません。Codex の月別チャートと表は古い月から順に表示します。 - リクエスト単位の Token 帰属では、有効な `last_token_usage` の各 component を優先します。その `total_tokens` はアクティブなコンテキスト量であり、リクエスト使用量ではありません。total と last の完全な数値署名により、同じ仮名化 rate-limit ソースまたは直前の隣接レコードの再生を証明できる場合だけ除外します。last snapshot がない場合は cumulative lineage high-water にフォールバックします。アップグレード後はインデックスを一度自動再構築し、その間もインデックス済み小計を表示します。 -- Claude と Codex の全期間 / 比較ビューは、ローカル Token ログから過去の使用済み等価値を直接計算します。最新の有効な公式リセット観測を基準に、一意で重複しない週間期間の列を作り、各使用イベントは一つの期間にだけ数えます。利用できる観測がなければ、使用分のみの履歴を UTC の月曜から月曜の暦週にフォールバックします。系列名が異なっていても、重複して整合しない将来リセットは競合として扱い、二つ目の「現在の期間」を作りません。期間範囲とリセット時刻は分けて表示します。Codex の使用量は日単位のスライスで保存されます。スライスが公式の日中リセットをまたぐ場合も Token は一度だけ数え、影響する期間を「境界近似」と表示し、使用済み等価値だけを示して総上限や未使用分を逆算しません。この表示規則はインデックス schema を変更せず、再構築も発生させません。Codex の過去期間は常に使用済み値だけを表示します。最新の現在期間に限り、リセット競合がなく、インデックス済み使用量を単一の観測元へ確実に帰属できる場合だけ総上限を推定でき、現在の未使用値は結論として示しません。帰属できない複数ログインの使用量も使用済み値だけとし、架空のアカウント分割は行いません。履歴には現在の公式 API 単価を一貫して適用します。これは請求額や公式のサブスクリプション価格ではなく代理推定です。このパネルは既定で有効で、設定の `showWeeklyEquivalentValue` から非表示にできます。Claude はこのリリースからプロファイル別に上限観測を記録します。 +- Claude と Codex の全期間 / 比較ビューは、ローカル Token ログから過去の使用済み等価値を直接計算します。最新の有効な公式リセット観測を基準に、一意で重複しない週間期間の列を作り、各使用イベントは一つの期間にだけ数えます。利用できる観測がなければ、使用分のみの履歴を UTC の月曜から月曜の暦週にフォールバックします。重複して整合しない将来リセットは、別の quota series である場合に競合として扱います。同じ series の観測は一つの series のまま、低信頼度の近似として表示されることがあります。二つ目の「現在の期間」は作りません。期間範囲とリセット時刻は分けて表示します。Codex の account-wide `codex` 観測は過去と現在の期間の両方を装飾できます。リセット時刻が七日グリッドからずれる場合は、観測時刻を含む表示期間へ割り当てます。ファイルソース、リセットのずれ、日次スライスの境界越えは推定を可能にする一方で信頼度を下げ、近似として表示します。ファイルキーはアカウント ID ではないため、同じ home の対象ファイルをまとめて扱います。別の quota series、または観測のない期間は使用済み等価値だけを表示し、アカウント分割を作りません。現在の未使用分は表示せず、過去の未使用分は総上限推定がある場合だけ表示します。Codex の使用量は日単位のスライスで保存され、Token は一つの期間にだけ数えます。この表示規則はインデックス schema を変更せず、再構築も発生させません。履歴には現在の公式 API 単価を一貫して適用します。これは請求額や公式のサブスクリプション価格ではなく代理推定です。このパネルは既定で有効で、設定の `showWeeklyEquivalentValue` から非表示にできます。Claude はこのリリースからプロファイル別に上限観測を記録します。 - Codex に切り替えても、既存の今日 / 今月 / 全期間 / セッション / プロジェクト / コンテンツ / 設定の構成を維持し、Codex の意味に合わせてラベルだけを調整します。両プロバイダーは同じ render function、HTML class、チャート、表、余白、レスポンシブルールを使います。時系列チャートの幅は揃い、密な内容だけが各領域内でスクロールします。Codex の提案はインデックス済み 30 日間の構造的根拠を使います。 - ルートタスクには、絶対パスを伏せた最新の実際のスレッド名を使います。子タスクは自身の実際のスレッド名を優先し、ない場合は報告されたニックネームを使って親 / ルートタスク名も表示します。それらもない場合はローカライズ済みの中立的な代替名を表示します。プロジェクト名は Git リポジトリ名、Git 外ではフォルダー名です。信頼できる集計ができない Branches や Workflows は作り上げません。 - 過去 7 日 / 30 日は、設定したタイムゾーンでイベント発生日を正確に区切ります。移行または再構築中は、Codex のカード、表、プロジェクト、セッション、提案、ステータス値を**インデックス済み小計**として明示し、未検証の旧集計値を除外します。Codex home を検出するとプロバイダータブを直ちに表示し、初回インデックス中はページ内に正確なインデックス済みファイル数、割合、容量の進捗をリアルタイム表示します。「比較」は両プロバイダーに実データが揃ってから表示します。最初のアトミックチェックポイント生成後は、インデックス処理が続いていても小計ダッシュボード全体を利用でき、進捗表示がカードや表に置き換わることはありません。選択した Codex home はアカウントを区別しないため、同じ home に複数のログインが残したログは合算されます。ローカルログには信頼できるアカウント識別子がないため、上限カードは**最終観測**のままで合算しません。 diff --git a/README-ko.md b/README-ko.md index 40573acc..2ab43d27 100644 --- a/README-ko.md +++ b/README-ko.md @@ -51,9 +51,9 @@ - Codex 사용량 레코드는 `sessions/**/*.jsonl`과 `archived_sessions/**/*.jsonl`에서만 찾으며 자격 증명, DB, 알 수 없는 파일은 제외합니다. 별도로 `$CODEX_HOME/session_index.jsonl` 하나만 정확히 스트리밍해 실제 스레드 제목에 사용할 `id` → `thread_name` 매핑을 가져옵니다. 제목 속 절대 경로는 가리고 제목은 메모리에만 유지합니다. 사용량 JSONL 각 줄은 허용 목록의 사용량·구조 메타데이터를 추출하기 위해서만 스트리밍 방식으로 일시 파싱합니다. 프롬프트, 응답, 명령, 도구 인수 필드는 검사하거나 분석에 사용하지 않으며 보관·영구 저장하지 않습니다. - **처리된** 값 = 입력 + 출력, **캐시되지 않은 사용량** = 캐시되지 않은 입력 + 출력, **캐시 입력**은 입력의 부분집합, reasoning은 출력의 부분집합입니다. 개요에는 캐시 입력 / 입력으로 계산한 입력 캐시 적중률도 표시합니다. Codex 청구 비용은 표시하지 않습니다. 첫 요약 카드에는 선택 범위의 API 등가 비용 추정치를 명확히 표시하고, 전체 기간 보기에는 같은 가격 기준의 주간 추이를 표시합니다. Claude / Codex / Compare는 공급자별 집계 의미를 분리합니다. -- Codex의 **오늘**은 설정한 시간대의 현재 달력 날짜를 뜻합니다. 정확한 시간별 API 등가 비용을 표시하고 Token 구성은 별도 보기로 유지합니다. 일별·월별 기본 차트도 API 등가 비용을 기본값으로 사용하며 Token 구성은 따로 볼 수 있습니다. 정확히 일치하는 알려진 모델만 가격 계산에 포함하므로 알 수 없는 모델은 가격을 매기지 않고 가격 적용 범위는 계속 표시합니다. schema 3과 호환되는 추가형 시간별 sidecar는 최근 30일의 희소한 날짜 / 시간 버킷을 보관하고 checkpoint 저장과 재개, 31일째 제거를 지원하며 날짜를 펼칠 때 JSONL을 읽지 않습니다. +- Codex의 **오늘**은 설정한 시간대의 현재 달력 날짜를 뜻합니다. 정확한 시간별 API 등가 비용을 표시하고 Token 구성은 별도 보기로 유지합니다. 일별·월별 기본 차트도 API 등가 비용을 기본값으로 사용하며 Token 구성은 따로 볼 수 있습니다. 정확히 일치하는 알려진 모델만 가격 계산에 포함하므로 알 수 없는 모델은 가격을 매기지 않고 가격 적용 범위는 계속 표시합니다. schema 3과 호환되는 추가형 시간별 sidecar는 최근 30일의 희소한 날짜 / 시간 버킷을 보관하고 checkpoint 저장과 재개, 31일째 제거를 지원하며 날짜를 펼칠 때 JSONL을 읽지 않습니다. Codex 월별 차트와 표는 오래된 달부터 표시합니다. - 요청 단위 Token 귀속은 유효한 `last_token_usage` component를 우선 사용합니다. 그 안의 `total_tokens`는 활성 컨텍스트 크기이며 요청 사용량이 아닙니다. total과 last의 전체 숫자 서명이 동일한 가명 rate-limit 소스 또는 바로 인접한 레코드의 재생을 증명할 때만 중복을 제거합니다. last snapshot이 없으면 cumulative lineage high-water로 되돌아갑니다. 업그레이드 후 인덱스를 한 번 자동 재구축하며 그동안에도 인덱싱된 소계를 계속 표시합니다. -- Claude와 Codex의 전체 기간 / 비교 보기는 로컬 Token 로그에서 과거 사용 등가치를 직접 계산합니다. 가장 최근의 유효한 공식 재설정 관측 하나만 고유하고 겹치지 않는 주간 기간열의 기준이 되므로 각 사용 이벤트는 한 기간에만 집계됩니다. 사용할 수 있는 관측이 없으면 사용분 전용 기록만 UTC 월요일부터 월요일까지의 달력 주로 되돌아갑니다. series 이름이 달라도 겹치면서 정렬되지 않은 미래 재설정은 충돌로 처리하며 두 번째 「현재 기간」을 만들 수 없습니다. 기간 범위와 재설정 시각은 따로 표시합니다. Codex 사용량은 일별 슬라이스로 저장됩니다. 슬라이스가 공식 일중 재설정을 가로지르더라도 Token은 한 번만 집계하고, 영향을 받은 기간을 「경계 근사치」로 표시하며 사용된 등가치만 보여 주고 총한도나 미사용분을 역산하지 않습니다. 이 표시 규칙은 인덱스 schema를 변경하거나 재구축을 유발하지 않습니다. Codex 과거 기간은 항상 사용된 값만 표시합니다. 최신 현재 기간에 한해 재설정 충돌이 없고 인덱싱된 사용량을 단일 관측 출처에 안정적으로 귀속할 수 있을 때만 총한도를 추정할 수 있으며, 현재 미사용 값은 결론으로 제시하지 않습니다. 귀속할 수 없는 여러 로그인 사용량도 사용된 값만 표시하며 계정 분리를 만들어 내지 않습니다. 현재 공식 API 단가를 전체 기록에 일관되게 적용합니다. 이는 청구액이나 공식 구독 가격이 아닌 대리 추정치입니다. 이 패널은 기본으로 켜지며 설정의 `showWeeklyEquivalentValue`에서 끌 수 있습니다. Claude는 이 릴리스부터 프로필별 한도 관측을 기록합니다. +- Claude와 Codex의 전체 기간 / 비교 보기는 로컬 Token 로그에서 과거 사용 등가치를 직접 계산합니다. 가장 최근의 유효한 공식 재설정 관측을 기준으로 고유하고 겹치지 않는 주간 기간열을 만들며 각 사용 이벤트는 한 기간에만 집계됩니다. 사용할 수 있는 관측이 없으면 사용분 전용 기록만 UTC 월요일부터 월요일까지의 달력 주로 되돌아갑니다. 겹치면서 정렬되지 않은 미래 재설정은 실제로 다른 quota series인 경우에만 충돌로 처리합니다. 같은 series의 관측은 하나의 series로 유지하며 낮은 신뢰도의 근사치로 표시할 수 있습니다. 두 번째 「현재 기간」을 만들지 않습니다. 기간 범위와 재설정 시각은 따로 표시합니다. Codex의 account-wide `codex` 관측은 과거와 현재 기간 모두를 장식할 수 있으며, 재설정 시각이 7일 격자에서 벗어나면 관측 시각이 포함된 표시 기간에 매핑합니다. 파일 출처, 재설정 이동 또는 일별 슬라이스의 경계 통과가 있으면 추정은 가능하지만 신뢰도를 낮추고 근사치로 표시합니다. 파일 키는 계정 ID가 아니므로 같은 home의 대상 파일을 함께 사용합니다. 다른 quota series이거나 관측이 없는 기간은 사용된 등가치만 표시하고 계정 분리를 만들어 내지 않습니다. 현재 미사용 값은 표시하지 않으며 과거 미사용 값은 총한도 추정이 있을 때만 표시합니다. Codex 사용량은 일별 슬라이스로 저장되고 Token은 한 기간에만 집계됩니다. 이 표시 규칙은 인덱스 schema를 변경하거나 재구축을 유발하지 않습니다. 현재 공식 API 단가를 전체 기록에 일관되게 적용합니다. 이는 청구액이나 공식 구독 가격이 아닌 대리 추정치입니다. 이 패널은 기본으로 켜지며 설정의 `showWeeklyEquivalentValue`에서 끌 수 있습니다. Claude는 이 릴리스부터 프로필별 한도 관측을 기록합니다. - Codex로 전환해도 기존 오늘 / 이번 달 / 전체 기간 / 세션 / 프로젝트 / 콘텐츠 / 설정 구조를 유지하고 Codex 의미에 맞게 레이블만 조정합니다. 두 공급자는 같은 render function, HTML class, 차트, 표, 간격 및 반응형 규칙을 사용합니다. 시계열 차트는 너비가 정렬되고 밀집된 콘텐츠만 각 영역 안에서 스크롤됩니다. Codex 제안은 인덱싱된 30일 구조 근거를 사용합니다. - 루트 작업은 절대 경로를 가린 최신 실제 스레드 제목을 사용합니다. 자식 작업은 자신의 실제 스레드 제목을 우선 사용하고, 제목이 없으면 보고된 닉네임을 사용하면서 부모 / 루트 작업 제목도 표시합니다. 이 정보도 없으면 현지화된 중립 대체 이름을 표시합니다. 프로젝트 이름은 Git 저장소 이름을, Git 외부에서는 폴더 이름을 사용합니다. 안정적으로 집계할 수 없는 Branches나 Workflows를 만들어 내지 않습니다. - 최근 7일 / 30일 값은 설정한 시간대에서 이벤트 발생일을 정확히 나눠 계산합니다. 마이그레이션이나 재구축이 진행 중이면 Codex 카드, 표, 프로젝트, 세션, 권장 사항 및 상태 값을 **인덱싱된 소계**로 명시하고 검증되지 않은 이전 합계를 제외합니다. Codex home이 감지되면 공급자 탭을 즉시 표시하고, 최초 인덱싱 중에는 페이지 안에 정확한 인덱싱 파일 수, 백분율, 용량 진행 상황을 실시간으로 표시합니다. 비교 탭은 두 공급자 모두 실제 데이터가 생긴 뒤에만 표시됩니다. 첫 원자적 체크포인트가 생성되면 인덱싱이 계속되는 동안에도 전체 소계 대시보드를 사용할 수 있으며 진행 표시가 카드나 표를 대체하지 않습니다. 선택한 Codex home은 계정을 구분하지 않으므로 같은 home에 여러 로그인이 남긴 로그는 합산됩니다. 로컬 로그에는 신뢰할 수 있는 계정 식별자가 없으므로 한도 카드는 **마지막 관측**만 유지하며 합산하지 않습니다. diff --git a/README-zh-CN.md b/README-zh-CN.md index 4666db87..4f8e3d1c 100644 --- a/README-zh-CN.md +++ b/README-zh-CN.md @@ -70,9 +70,9 @@ AI 个性化仍为可选项。自备 `advice.apiKey` 后,默认请求只含聚 - Codex 用量记录只会从 `sessions/**/*.jsonl` 与 `archived_sessions/**/*.jsonl` 中发现;凭据、数据库和未知文件仍明确排除。此外,插件会精确流式读取 `$CODEX_HOME/session_index.jsonl`,仅将 `id` 映射为 `thread_name`,用于真实线程标题。标题中的绝对路径会先脱敏,标题只保留在内存。用量 JSONL 会逐行流式、临时解析,以提取白名单内的用量与结构元数据;提示词、回复、命令和工具参数字段不会被检查或用于分析,也不会被保留或持久化。 - **已处理** = 输入 + 输出;**未缓存用量** = 未缓存输入 + 输出;**缓存输入**仍是输入的子集,reasoning 仍是输出的子集。概览还会按「缓存输入 / 输入」显示输入缓存命中率。Codex 不显示账单成本;首张汇总卡会针对当前范围显示明确标注的 API 等效成本估算,「全部时间」则按同一定价口径展示每周趋势。Claude / Codex / Compare 仍保持各供应商口径独立。 -- Codex 的**今天**指配置时区中的当前自然日。页面会显示精确的小时 API 等效成本,并把 Token 构成作为独立视图;每日与每月主图也默认显示 API 等效成本,Token 构成仍单独保留。只有精确匹配的已知模型才参与计价,未知模型保持未定价,定价覆盖率始终可见。这个兼容 schema 3 的增量小时 sidecar 保存滚动最近 30 天的稀疏日期 / 小时桶,支持检查点与续传、淘汰第 31 天,并在展开日期时不读取 JSONL。 +- Codex 的**今天**指配置时区中的当前自然日。页面会显示精确的小时 API 等效成本,并把 Token 构成作为独立视图;每日与每月主图也默认显示 API 等效成本,Token 构成仍单独保留。只有精确匹配的已知模型才参与计价,未知模型保持未定价,定价覆盖率始终可见。这个兼容 schema 3 的增量小时 sidecar 保存滚动最近 30 天的稀疏日期 / 小时桶,支持检查点与续传、淘汰第 31 天,并在展开日期时不读取 JSONL。Codex 按月图表和表格按月份从旧到新显示。 - 单次请求 Token 归因优先采用有效的 `last_token_usage` component;其中 `total_tokens` 表示活跃上下文大小,不是该请求的用量。只有完整的 total-plus-last 数字签名能证明同一伪名 rate-limit 来源或紧邻记录发生重放时才去重;缺少 last snapshot 时回退为 cumulative lineage high-water。升级后会自动重建一次索引,期间持续显示已索引小计。 -- Claude 与 Codex 的「全部时间」/「对比」会直接根据本地 Token 日志计算历史已用等价值。最新且有效的官方重置观测只会锚定一组唯一、互不重叠的每周周期,每条用量只计入一个周期;没有可用观测时,仅已用历史才回退到 UTC 周一至周一的自然周。任何重叠但未对齐的未来重置都会视为冲突,即使 series 名称不同,也不能再生成第二个「当前周期」;周期范围与重置时刻会分开表达。Codex 用量按日切片保存:若某个切片跨过官方日内重置,其 Token 仍只计一次,受影响周期会标为「边界近似」,并且只显示已用等价值,不反推总额度或未用额度。该显示规则不改变索引 schema,也不会触发重建。Codex 历史周期一律仅显示已用值;只有最新当前周期在重置无冲突、且已索引用量可可靠归属到单一观测来源时,才允许反推总额度,当前未用值仍不会作为结论。无法归属的多登录用量同样只显示已用值,不会虚构账号拆分。历史统一套用当前官方 API 单价;这是代理估算,不是账单或官方订阅价格。该面板默认开启,可在设置中通过 `showWeeklyEquivalentValue` 关闭。Claude 从此版本开始按本地 profile 记录额度观测。 +- Claude 与 Codex 的「全部时间」/「对比」会直接根据本地 Token 日志计算历史已用等价值。最新且有效的官方重置观测会锚定一组唯一、互不重叠的每周周期,每条用量只计入一个周期;没有可用观测时,仅已用历史才回退到 UTC 周一至周一的自然周。只有真正不同 quota series 且其未来重置重叠、未对齐时才视为冲突;同一 series 的观测仍保持为一个 series,并可按低可信度近似显示,不会再生成第二个「当前周期」。周期范围与重置时刻会分开表达。Codex 的 account-wide `codex` 观测可以同时装饰历史和当前周期;如果重置时间偏离七天网格,就按观测时间映射到对应展示周期。文件来源、重置漂移或日切片跨边界时仍可给出总额度近似,但会降低可信度并标注为近似。文件键不是账号身份,因此同一 home 中符合条件的本地文件会一起参与;真正不同的 quota series 或没有可用观测的周期仍只显示已用值,不会虚构账号拆分。当前未用值仍不显示,历史未用值只有在存在总额度近似时才显示。Codex 用量按日切片保存,Token 仍只计入一个周期;该显示规则不改变索引 schema,也不会触发重建。历史统一套用当前官方 API 单价;这是代理估算,不是账单或官方订阅价格。该面板默认开启,可在设置中通过 `showWeeklyEquivalentValue` 关闭。Claude 从此版本开始按本地 profile 记录额度观测。 - 切换到 Codex 后仍沿用既有的今天 / 本月 / 全部时间 / 会话 / 项目 / 内容 / 设置结构,并按 Codex 语义调整标签。两个供应商复用同一组渲染函数、HTML class、图表、表格、间距和响应式规则;时间序列图保持宽度对齐,密集内容只在各自区域内滚动。Codex 建议使用已索引的 30 天结构化证据。 - 根任务使用经过路径脱敏的最新真实线程标题;子任务优先使用自己的真实线程标题,缺失时使用其上报的昵称并显示父级 / 根任务标题;这些信息仍缺失时采用本地化的中性回退名称。项目名称使用 Git 仓库名,非 Git 目录则使用文件夹的末级名称。插件不会虚构无法可靠统计的 Branches 或 Workflows。 - 最近 7 天与 30 天按配置时区中的事件发生日精确切片。迁移或重建未完成时,Codex 的卡片、表格、项目、会话、建议与状态栏都会明确标为**已索引小计**,并排除未经验证的旧版总量。检测到 Codex home 后会立即显示供应商标签;首次建立索引期间在页面内实时显示精确的已索引文件数、百分比与容量进度,「对比」则等到两个供应商都有真实数据后才出现。生成首个原子检查点后,完整的小计仪表盘会在索引继续期间保持可用;进度提示不会替代卡片和表格。所选 Codex home 不区分账号,因此同一目录中多个登录账号留下的日志会合并统计;本地日志没有可靠的账号身份字段,额度卡只能保留**最后观测**,不会相加。 diff --git a/README-zh-TW.md b/README-zh-TW.md index 1667a00f..a12d1623 100644 --- a/README-zh-TW.md +++ b/README-zh-TW.md @@ -51,9 +51,9 @@ - Codex 用量記錄只會從 `sessions/**/*.jsonl` 與 `archived_sessions/**/*.jsonl` 中發現;憑證、資料庫與未知檔案仍明確排除。此外,擴充功能會精確串流讀取 `$CODEX_HOME/session_index.jsonl`,僅將 `id` 對應到 `thread_name`,用於真實執行緒標題。標題中的絕對路徑會先去識別化,標題只保留在記憶體。用量 JSONL 會逐行串流並暫時解析,以擷取白名單內的用量與結構中繼資料;提示、回覆、命令和工具參數欄位不會被檢查或用於分析,也不會被保留或持久化。 - **已處理** = 輸入 + 輸出;**未快取用量** = 未快取輸入 + 輸出;**快取輸入**仍是輸入的子集,reasoning 仍是輸出的子集。概覽也會依「快取輸入 / 輸入」顯示輸入快取命中率。Codex 不顯示帳單成本;首張摘要卡會針對目前範圍顯示明確標示的 API 等效成本估算,「全部時間」則依同一定價口徑顯示每週趨勢。Claude / Codex / Compare 仍維持各供應商口徑獨立。 -- Codex 的**今天**指設定時區中的目前自然日。頁面會顯示精確的每小時 API 等效成本,並將 Token 構成保留為獨立檢視;每日與每月主圖也預設顯示 API 等效成本,Token 構成仍分開呈現。只有精確符合的已知模型才參與計價,未知模型維持未定價,定價涵蓋率始終可見。這個相容 schema 3 的增量小時 sidecar 會保留最近 30 天的稀疏日期 / 小時分桶,支援檢查點與續傳、淘汰第 31 天,並在展開日期時不讀取 JSONL。 +- Codex 的**今天**指設定時區中的目前自然日。頁面會顯示精確的每小時 API 等效成本,並將 Token 構成保留為獨立檢視;每日與每月主圖也預設顯示 API 等效成本,Token 構成仍分開呈現。只有精確符合的已知模型才參與計價,未知模型維持未定價,定價涵蓋率始終可見。這個相容 schema 3 的增量小時 sidecar 會保留最近 30 天的稀疏日期 / 小時分桶,支援檢查點與續傳、淘汰第 31 天,並在展開日期時不讀取 JSONL。Codex 按月圖表與表格會按月份由舊到新顯示。 - 單次請求 Token 歸因優先採用有效的 `last_token_usage` component;其中 `total_tokens` 表示目前上下文大小,不是該次請求的用量。只有完整的 total-plus-last 數字簽名能證明同一假名化 rate-limit 來源或相鄰記錄發生重播時才去重;缺少 last snapshot 時回退為 cumulative lineage high-water。升級後會自動重建一次索引,期間持續顯示已索引小計。 -- Claude 與 Codex 的「全部時間」/「比較」會直接依本機 Token 日誌計算歷史已用等價值。最新且有效的官方重設觀測只會錨定一組唯一、互不重疊的每週週期,每筆用量只計入一個週期;沒有可用觀測時,僅已用歷史才回退到 UTC 週一至週一的自然週。任何重疊但未對齊的未來重設都會視為衝突,即使 series 名稱不同,也不能再建立第二個「目前週期」;週期範圍與重設時刻會分開呈現。Codex 用量按日切片保存:若某個切片跨過官方日內重設,其 Token 仍只計一次,受影響週期會標為「邊界近似」,並且只顯示已用等價值,不反推總額度或未用額度。此顯示規則不改變索引 schema,也不會觸發重建。Codex 歷史週期一律只顯示已用值;只有最新目前週期在重設無衝突,且已索引用量可可靠歸屬到單一觀測來源時,才允許反推總額度,目前未用值仍不作為結論。無法歸屬的多登入用量同樣只顯示已用值,不會虛構帳號拆分。歷史統一套用目前官方 API 單價;這是代理估算,不是帳單或官方訂閱價格。此面板預設開啟,可在設定中透過 `showWeeklyEquivalentValue` 關閉。Claude 從此版本開始依本機 profile 記錄額度觀測。 +- Claude 與 Codex 的「全部時間」/「比較」會直接依本機 Token 日誌計算歷史已用等價值。最新且有效的官方重設觀測會錨定一組唯一、互不重疊的每週週期,每筆用量只計入一個週期;沒有可用觀測時,僅已用歷史才回退到 UTC 週一至週一的自然週。只有真正不同的 quota series 且其未來重設重疊、未對齊時才視為衝突;同一 series 的觀測仍維持為一個 series,並可按低可信度近似顯示,不會再建立第二個「目前週期」。週期範圍與重設時刻會分開呈現。Codex 的 account-wide `codex` 觀測可同時裝飾歷史與目前週期;若重設時間偏離七天網格,就依觀測時間映射到對應展示週期。檔案來源、重設漂移或日切片跨邊界時仍可給出總額度近似,但會降低可信度並標示為近似。檔案鍵不是帳號身分,因此同一個 home 中符合條件的本機檔案會一起參與;真正不同的 quota series 或沒有可用觀測的週期仍只顯示已用值,不會虛構帳號拆分。目前未用值仍不顯示,歷史未用值只有在存在總額度近似時才顯示。Codex 用量按日切片保存,Token 仍只計入一個週期;此顯示規則不改變索引 schema,也不會觸發重建。歷史統一套用目前官方 API 單價;這是代理估算,不是帳單或官方訂閱價格。此面板預設開啟,可在設定中透過 `showWeeklyEquivalentValue` 關閉。Claude 從此版本開始依本機 profile 記錄額度觀測。 - 切換到 Codex 後仍沿用既有的今天 / 本月 / 全部時間 / 工作階段 / 專案 / 內容 / 設定結構,並依 Codex 語義調整標籤。兩個供應商共用同一組 render function、HTML class、圖表、表格、間距與響應式規則;時間序列圖維持寬度對齊,密集內容只在各自區域內捲動。Codex 建議使用已索引的 30 天結構化證據。 - 根任務使用經過路徑去識別化的最新真實執行緒標題;子任務優先使用自己的真實執行緒標題,缺少時使用其回報的暱稱並顯示父級 / 根任務標題;這些資訊仍缺少時採用本地化的中性回退名稱。專案名稱使用 Git 儲存庫名稱,非 Git 目錄則使用資料夾的末級名稱。擴充功能不會虛構無法可靠統計的 Branches 或 Workflows。 - 最近 7 天與 30 天依設定時區中的事件發生日精確切片。遷移或重建未完成時,Codex 的卡片、表格、專案、工作階段、建議與狀態列都會明確標為**已索引小計**,並排除未驗證的舊版總量。偵測到 Codex home 後會立即顯示供應商分頁;首次建立索引期間會在頁面內即時顯示精確的已索引檔案數、百分比與容量進度,「比較」則等到兩個供應商都有真實資料後才出現。產生第一個原子檢查點後,完整的小計儀表盤會在索引繼續期間保持可用;進度提示不會取代卡片和表格。所選 Codex home 不區分帳戶,因此同一目錄中多個登入帳戶留下的記錄會合併統計;本機記錄沒有可靠的帳戶身分欄位,額度卡只能保留**最後觀測**,不會相加。 diff --git a/README.md b/README.md index e49a484f..9ca9ea0a 100644 --- a/README.md +++ b/README.md @@ -171,7 +171,7 @@ release. row keeps pricing coverage visible. The schema-3-compatible hourly sidecar keeps sparse buckets for the rolling last 30 days, is checkpointed and resumable, evicts day 31, and serves date expansion with zero JSONL reads on - click. + click. Codex monthly charts and tables list months oldest-first. - **Request-level token attribution** — valid `last_token_usage` components are preferred, while its `total_tokens` remains an active-context measurement, not request usage. A full numeric total-plus-last signature suppresses only @@ -183,20 +183,25 @@ release. newest valid official reset observation anchors one sequence of unique, non-overlapping weekly periods, and each usage event belongs to exactly one period; without a usable observation, usage-only rows fall back to - Monday-to-Monday UTC calendar weeks. Any overlapping, non-aligned future reset - is a conflict even if its series name differs; it cannot create a second current - period, and period ranges are shown separately from reset times. Codex usage is + Monday-to-Monday UTC calendar weeks. A genuinely different quota series with + an overlapping, non-aligned future reset is a conflict; same-series observations + remain one series and may be shown as a low-confidence approximation. It cannot + create a second current period, and period ranges are shown separately from reset times. Codex usage is stored in daily slices: when one crosses an official intraday reset, its tokens are still counted once, the affected period is labelled a boundary - approximation, and only used equivalent is shown. This display rule neither - changes the index schema nor triggers a rebuild. Codex historical periods are - always used-value-only. Only the newest current period may infer a total when - the reset is unambiguous and indexed usage can be attributed to one observation - source; current unused value is still withheld. Multi-sign-in usage that cannot - be attributed reliably also remains used-only, without inventing an account - split. Current official API rates are applied consistently across history. This - is a proxy, not a bill or an official subscription price. The panel is enabled - by default and can be hidden in Settings with `showWeeklyEquivalentValue`. + approximation. Codex's account-wide `codex` observations can decorate both + historical and current buckets. If an observed reset drifts from the seven-day + grid, the sample is mapped to the display period containing its observation + time; file-source uncertainty, reset drift, and daily boundary crossings lower + confidence and are labelled as approximate. File keys are not account + identities, so eligible local files in one home are included together. A + genuinely different quota series, or a period without a usable observation, + remains usage-only rather than inventing an account split. Current unused value + is withheld; historical unused value appears only when a full estimate exists. + This display rule neither changes the index schema nor triggers a rebuild. + Current official API rates are applied consistently across history. This is a + proxy, not a bill or an official subscription price. The panel is enabled by + default and can be hidden in Settings with `showWeeklyEquivalentValue`. - **One dashboard render stack** — switching to Codex keeps the established Today / Month / All time / Sessions / Projects / Content / Settings structure, relabelled where Codex semantics differ. The same render functions, HTML diff --git a/src/i18n.ts b/src/i18n.ts index 54f64be4..5309b14b 100644 --- a/src/i18n.ts +++ b/src/i18n.ts @@ -1048,7 +1048,7 @@ const WEEKLY_VALUE_COPY: Record = { historyFromLogs: 'Historical used equivalents come directly from local token logs. Full and unused estimates appear only for windows with a real quota-utilization observation.', calendarFallback: 'No historical weekly reset was observed; usage-only history is grouped into Monday-to-Monday UTC calendar weeks.', multiAccount: 'Codex usage-only history combines all sign-ins in this Codex home. When usage cannot be reliably attributed to a single quota observation, only the used equivalent is shown; no account split or allowance estimate is invented.', - boundaryApproximation: 'Codex usage is aggregated by day. If an official reset falls within a recorded day, the affected period keeps only its used equivalent; full and unused values are not inferred.', + boundaryApproximation: 'Codex usage is aggregated by day. If an official reset falls within a recorded day, the affected period may show a low-confidence full-allowance approximation; current unused value remains withheld.', boundaryApproximate: 'Boundary approx.', indexedSubtotal: 'Indexing is incomplete; weekly values are conservative subtotals.', }, @@ -1063,7 +1063,7 @@ const WEEKLY_VALUE_COPY: Record = { historyFromLogs: '历史“已用等价值”直接由本地 Token 日志计算;只有某个窗口存在真实额度用量观测时,才显示总额度和未用额度估算。', calendarFallback: '未观测到可用于对齐历史的每周重置时间;仅已用历史按 UTC 周一至周一的自然周分组。', multiAccount: 'Codex 的仅已用历史会合并此 Codex home 中的全部登录。用量无法可靠归属到单一额度观测时,只显示已用等价值;不会虚构账号拆分或额度估算。', - boundaryApproximation: 'Codex 用量按日汇总。如果官方重置发生在某个已记录日期内,受影响周期只保留已用等价值,不反推总额度或未用额度。', + boundaryApproximation: 'Codex 用量按日汇总。如果官方重置发生在某个已记录日期内,受影响周期仍可显示低可信度的总额度近似值;当前周期仍不显示未用估算。', boundaryApproximate: '边界近似', indexedSubtotal: '索引尚未完成;每周价值目前是保守小计。', }, @@ -1078,7 +1078,7 @@ const WEEKLY_VALUE_COPY: Record = { historyFromLogs: '歷史「已用等價值」直接由本機 Token 日誌計算;只有視窗存在真實額度用量觀測時,才顯示總額度與未用額度估算。', calendarFallback: '未觀測到可用於對齊歷史的每週重設時間;僅已用歷史依 UTC 週一至週一的自然週分組。', multiAccount: 'Codex 的僅已用歷史會合併此 Codex home 中的所有登入。用量無法可靠歸屬到單一額度觀測時,只顯示已用等價值;不會虛構帳號拆分或額度估算。', - boundaryApproximation: 'Codex 用量按日彙總。如果官方重設發生在某個已記錄日期內,受影響週期只保留已用等價值,不反推總額度或未用額度。', + boundaryApproximation: 'Codex 用量按日彙總。如果官方重設發生在某個已記錄日期內,受影響週期仍可顯示低可信度的總額度近似值;目前週期仍不顯示未用估算。', boundaryApproximate: '邊界近似', indexedSubtotal: '索引尚未完成;每週價值目前是保守小計。', }, @@ -1093,7 +1093,7 @@ const WEEKLY_VALUE_COPY: Record = { historyFromLogs: '過去の使用済み等価値はローカルの Token ログから直接計算します。総上限と未使用分は、実際の上限利用率が観測された枠だけで推定します。', calendarFallback: '履歴を揃える週間リセットが観測されていないため、使用分のみの履歴は UTC の月曜から月曜の暦週で集計します。', multiAccount: 'Codex の使用分のみの履歴は、この Codex home の全ログインを合算します。使用量を単一の上限観測に確実に帰属できない場合は使用済み等価値だけを表示し、アカウント分割や上限推定を作りません。', - boundaryApproximation: 'Codex の使用量は日単位で集計されます。公式リセットが記録日の途中にある場合、影響する期間は使用済み等価値だけを保持し、総上限や未使用分を推定しません。', + boundaryApproximation: 'Codex の使用量は日単位で集計されます。公式リセットが記録日の途中にある場合、影響する期間には低信頼度の総上限近似を表示できます。現在の未使用分は表示しません。', boundaryApproximate: '境界近似', indexedSubtotal: '索引作成中のため、週間価値は保守的な小計です。', }, @@ -1108,7 +1108,7 @@ const WEEKLY_VALUE_COPY: Record = { historyFromLogs: '과거 사용 등가치는 로컬 Token 로그에서 직접 계산합니다. 실제 한도 사용률 관측이 있는 창에서만 총한도와 미사용분을 추정합니다.', calendarFallback: '과거를 정렬할 주간 재설정이 관측되지 않아 사용분 전용 기록은 UTC 월요일부터 월요일까지의 달력 주로 묶습니다.', multiAccount: 'Codex 사용분 전용 기록은 이 Codex home의 모든 로그인을 합산합니다. 사용량을 단일 할당량 관측에 안정적으로 귀속할 수 없으면 사용 등가치만 표시하며 계정 분리나 한도 추정을 만들어 내지 않습니다.', - boundaryApproximation: 'Codex 사용량은 일별로 집계됩니다. 공식 재설정이 기록된 하루 중간에 발생하면 영향을 받는 기간에는 사용 등가치만 유지하고 총한도나 미사용분을 추정하지 않습니다.', + boundaryApproximation: 'Codex 사용량은 일별로 집계됩니다. 공식 재설정이 기록된 하루 중간에 발생하면 영향을 받는 기간에 낮은 신뢰도의 총한도 근사를 표시할 수 있으며, 현재 미사용 값은 표시하지 않습니다.', boundaryApproximate: '경계 근사', indexedSubtotal: '인덱싱이 끝나지 않아 주간 가치는 보수적인 소계입니다.', }, @@ -1123,7 +1123,7 @@ const WEEKLY_VALUE_COPY: Record = { historyFromLogs: 'Os equivalentes usados no histórico vêm diretamente dos logs locais de Token. O limite total e o não usado só são estimados quando há uma observação real da utilização da cota.', calendarFallback: 'Nenhuma redefinição semanal histórica foi observada; o histórico somente de uso é agrupado em semanas UTC de segunda a segunda.', multiAccount: 'O histórico somente de uso do Codex combina todos os logins deste Codex home. Quando o uso não pode ser atribuído com segurança a uma única observação de cota, só o equivalente usado é mostrado; nenhuma divisão por conta ou estimativa de limite é inventada.', - boundaryApproximation: 'O uso do Codex é agregado por dia. Se uma redefinição oficial ocorrer dentro de um dia registrado, o período afetado mantém apenas o equivalente usado; o limite total e o não usado não são inferidos.', + boundaryApproximation: 'O uso do Codex é agregado por dia. Se uma redefinição oficial ocorrer dentro de um dia registrado, o período afetado pode mostrar uma aproximação de baixa confiança do limite total; o não usado atual continua oculto.', boundaryApproximate: 'Fronteira aprox.', indexedSubtotal: 'A indexação está incompleta; os valores semanais são subtotais conservadores.', }, @@ -1138,7 +1138,7 @@ const WEEKLY_VALUE_COPY: Record = { historyFromLogs: 'Historische genutzte Gegenwerte stammen direkt aus lokalen Token-Logs. Gesamtlimit und ungenutzter Anteil werden nur bei real beobachteter Quotenauslastung geschätzt.', calendarFallback: 'Kein historischer Wochen-Reset wurde beobachtet; reine Nutzungsverläufe werden in UTC-Kalenderwochen von Montag bis Montag gruppiert.', multiAccount: 'Der reine Codex-Nutzungsverlauf kombiniert alle Anmeldungen in diesem Codex home. Lässt sich die Nutzung nicht zuverlässig einer einzelnen Kontingentbeobachtung zuordnen, wird nur der genutzte Gegenwert angezeigt; weder Kontotrennung noch Limitschätzung werden erfunden.', - boundaryApproximation: 'Die Codex-Nutzung wird tageweise aggregiert. Fällt ein offizieller Reset in einen erfassten Tag, behält der betroffene Zeitraum nur den genutzten Gegenwert; Gesamtlimit und ungenutzter Anteil werden nicht geschätzt.', + boundaryApproximation: 'Die Codex-Nutzung wird tageweise aggregiert. Fällt ein offizieller Reset in einen erfassten Tag, kann der betroffene Zeitraum eine Schätzung des Gesamtlimits mit niedriger Sicherheit zeigen; der aktuelle ungenutzte Wert bleibt verborgen.', boundaryApproximate: 'Grenznäherung', indexedSubtotal: 'Die Indizierung ist unvollständig; Wochenwerte sind konservative Zwischensummen.', }, @@ -1153,7 +1153,7 @@ const WEEKLY_VALUE_COPY: Record = { historyFromLogs: 'Ekuivalen terpakai historis dihitung langsung dari log Token lokal. Total batas dan sisa hanya diestimasi jika ada pengamatan nyata atas persentase kuota.', calendarFallback: 'Tidak ada reset mingguan historis yang teramati; riwayat khusus pemakaian dikelompokkan dalam minggu UTC Senin-ke-Senin.', multiAccount: 'Riwayat khusus pemakaian Codex menggabungkan semua login di Codex home ini. Jika penggunaan tidak dapat dikaitkan secara andal ke satu pengamatan kuota, hanya ekuivalen terpakai yang ditampilkan; pemisahan akun atau perkiraan batas tidak direka.', - boundaryApproximation: 'Penggunaan Codex diagregasi per hari. Jika reset resmi terjadi di tengah hari yang tercatat, periode yang terdampak hanya mempertahankan ekuivalen terpakai; total batas dan sisa tidak diperkirakan.', + boundaryApproximation: 'Penggunaan Codex diagregasi per hari. Jika reset resmi terjadi di tengah hari yang tercatat, periode terdampak dapat menampilkan perkiraan batas total dengan keyakinan rendah; sisa saat ini tetap disembunyikan.', boundaryApproximate: 'Perkiraan batas', indexedSubtotal: 'Pengindeksan belum selesai; nilai mingguan masih berupa subtotal konservatif.', }, diff --git a/src/providers/codex/codexUsage.ts b/src/providers/codex/codexUsage.ts index c31024c2..08954ad3 100644 --- a/src/providers/codex/codexUsage.ts +++ b/src/providers/codex/codexUsage.ts @@ -611,7 +611,9 @@ function monthlyRows( } } return [...months.entries()] - .sort(([left], [right]) => right.localeCompare(left)) + // Monthly history is a time series: keep the oldest month first so the + // chart, table, and screen-reader order all read chronologically. + .sort(([left], [right]) => left.localeCompare(right)) .map(([period, row]) => { const total = metrics(row.tokens); return { diff --git a/src/test/codexUsage.test.ts b/src/test/codexUsage.test.ts index 39ddeb35..9e057217 100644 --- a/src/test/codexUsage.test.ts +++ b/src/test/codexUsage.test.ts @@ -803,9 +803,12 @@ test('all-time, monthly, and behavior views stay provider-native', () => { assert.equal(view.allTime.total.processed, 1_560); assert.equal(view.allTime.total.fresh, 640); - assert.equal(view.monthly[0].period, '2026-07'); - assert.equal(view.monthly[0].total.processed, 1_440); - assert.equal(view.monthly[0].threads, 3); + assert.deepEqual( + view.monthly.map((row) => row.period), + ['2026-06', '2026-07'], + ); + assert.equal(view.monthly[1].total.processed, 1_440); + assert.equal(view.monthly[1].threads, 3); assert.equal(view.last7DaysDaily.length, 7); assert.equal(view.last7DaysDaily[0].day, '2026-07-14'); assert.equal(view.last7DaysDaily[6].day, '2026-07-20'); diff --git a/src/test/weeklyValue.test.ts b/src/test/weeklyValue.test.ts index dd40266b..6af3132e 100644 --- a/src/test/weeklyValue.test.ts +++ b/src/test/weeklyValue.test.ts @@ -323,7 +323,7 @@ test('conflicting stale Codex reset schedules do not choose an arbitrary alignme assert.equal(points[0]?.fullEquivalentUsd, null); }); -test('a fresh Codex reset starts a new current period instead of carrying the old week forward', () => { +test('a fresh Codex reset starts a new current period and estimates both observed windows', () => { const now = RESET + HOUR; const nextReset = RESET + 7 * DAY; const points = buildWeeklyValueTimeline('codex', { @@ -342,8 +342,8 @@ test('a fresh Codex reset starts a new current period instead of carrying the ol }), ], usage: [ - usage(40, RESET - 30 * 60 * 1000, 'one-file'), - usage(7, RESET + 45 * 60 * 1000, 'one-file'), + usage(40, RESET - 2 * HOUR, 'one-file'), + usage(7, RESET + 15 * 60 * 1000, 'one-file'), ], }, { now }); @@ -352,10 +352,12 @@ test('a fresh Codex reset starts a new current period instead of carrying the ol assert.equal(current?.resetAt, nextReset); assert.equal(current?.usedEquivalentUsd, 7); assert.equal(current?.utilizationPercent, 10); - assert.equal(current?.fullEquivalentUsd, null); + assert.equal(points.find((point) => point.resetAt === RESET)?.fullEquivalentUsd, 50); + assert.equal(points.find((point) => point.resetAt === RESET)?.unusedEquivalentUsd, 10); + assert.equal(current?.fullEquivalentUsd, 70); }); -test('ambiguous overlapping Codex reset observations do not invent a combined allowance', () => { +test('same account-wide Codex series uses the newest coherent observation across stale reset forecasts', () => { const now = Date.parse('2026-08-27T04:00:00.000Z'); const points = buildWeeklyValueTimeline('codex', { observations: [ @@ -377,9 +379,9 @@ test('ambiguous overlapping Codex reset observations do not invent a combined al const current = points.find((point) => point.current); assert.equal(current?.utilizationPercent, 43); - assert.equal(current?.fullEquivalentUsd, null); + assert.equal(current?.fullEquivalentUsd, 40 / 0.43); assert.equal(current?.unusedEquivalentUsd, null); - assert.equal(current?.confidence, 'usage-only'); + assert.equal(current?.confidence, 'medium'); }); test('different anonymous Codex series with overlapping current resets are still ambiguous', () => { @@ -408,7 +410,7 @@ test('different anonymous Codex series with overlapping current resets are still assert.equal(current?.confidence, 'usage-only'); }); -test('Codex allowance inference is withheld when combined usage has multiple log sources', () => { +test('Codex allowance inference includes multiple local log sources but lowers confidence', () => { const now = RESET - HOUR; const points = buildWeeklyValueTimeline('codex', { observations: [observation({ @@ -424,11 +426,12 @@ test('Codex allowance inference is withheld when combined usage has multiple log assert.equal(points[0].usedEquivalentUsd, 50); assert.equal(points[0].utilizationPercent, 50); - assert.equal(points[0].fullEquivalentUsd, null); - assert.equal(points[0].confidence, 'usage-only'); + assert.equal(points[0].fullEquivalentUsd, 100); + assert.equal(points[0].unusedEquivalentUsd, null); + assert.equal(points[0].confidence, 'low'); }); -test('completed Codex periods remain usage-only because account attribution is unavailable', () => { +test('completed Codex periods estimate total and unused value from an observed window', () => { const points = buildWeeklyValueTimeline('codex', { observations: [observation({ sourceKey: 'one-file' })], usage: [usage(40, RESET - 2 * HOUR, 'one-file')], @@ -437,12 +440,12 @@ test('completed Codex periods remain usage-only because account attribution is u assert.equal(points[0].current, false); assert.equal(points[0].usedEquivalentUsd, 40); assert.equal(points[0].utilizationPercent, 75); - assert.equal(points[0].fullEquivalentUsd, null); - assert.equal(points[0].unusedEquivalentUsd, null); - assert.equal(points[0].confidence, 'usage-only'); + assert.equal(points[0].fullEquivalentUsd, 40 / 0.75); + assert.ok(Math.abs((points[0].unusedEquivalentUsd ?? 0) - 40 / 3) < 1e-9); + assert.equal(points[0].confidence, 'high'); }); -test('a daily Codex aggregate crossing a reset is marked approximate and cannot infer allowance', () => { +test('a daily Codex aggregate crossing a reset keeps a low-confidence allowance estimate', () => { const now = RESET + HOUR; const points = buildWeeklyValueTimeline('codex', { observations: [observation({ @@ -459,8 +462,47 @@ test('a daily Codex aggregate crossing a reset is marked approximate and cannot const affected = points.find((point) => point.resetAt === RESET + 7 * DAY); assert.equal(affected?.boundaryUncertain, true); - assert.equal(affected?.fullEquivalentUsd, null); - assert.equal(affected?.confidence, 'usage-only'); + assert.equal(affected?.fullEquivalentUsd, 80); + assert.equal(affected?.unusedEquivalentUsd, null); + assert.equal(affected?.confidence, 'low'); +}); + +test('an irregular Codex reset is mapped to the fixed period containing its observation', () => { + const now = Date.parse('2026-08-31T12:00:00.000Z'); + const currentReset = Date.parse('2026-09-07T00:00:00.000Z'); + const driftingReset = Date.parse('2026-08-31T02:00:00.000Z'); + const points = buildWeeklyValueTimeline('codex', { + observations: [ + observation({ + observedAt: Date.parse('2026-08-25T12:00:00.000Z'), + resetAt: driftingReset, + usedPercent: 40, + sourceKey: 'older-file', + }), + observation({ + observedAt: Date.parse('2026-08-31T11:00:00.000Z'), + resetAt: currentReset, + usedPercent: 20, + sourceKey: 'current-file', + }), + ], + usage: [ + usage(30, Date.parse('2026-08-25T10:00:00.000Z'), 'older-file'), + usage(40, Date.parse('2026-08-31T10:00:00.000Z'), 'current-file'), + ], + }, { now }); + + const previous = points.find((point) => point.resetAt === driftingReset - 2 * HOUR); + const current = points.find((point) => point.resetAt === currentReset); + assert.equal(previous?.usedEquivalentUsd, 30); + assert.equal(previous?.utilizationPercent, 40); + assert.equal(previous?.fullEquivalentUsd, 75); + assert.equal(previous?.unusedEquivalentUsd, 45); + assert.equal(previous?.confidence, 'low'); + assert.equal(current?.usedEquivalentUsd, 40); + assert.equal(current?.utilizationPercent, 20); + assert.equal(current?.fullEquivalentUsd, 200); + assert.equal(current?.unusedEquivalentUsd, null); }); test('quota observations at the reset boundary do not decorate the closed period', () => { diff --git a/src/webview.ts b/src/webview.ts index 0173a7db..8763de4c 100644 --- a/src/webview.ts +++ b/src/webview.ts @@ -3118,7 +3118,9 @@ export class UsageWebviewProvider { }; const confidenceLabel = (point: WeeklyValuePoint): string => { if (point.boundaryUncertain) { - return `${copy.usageOnly} · ${copy.boundaryApproximate}`; + return point.fullEquivalentUsd === null + ? `${copy.usageOnly} · ${copy.boundaryApproximate}` + : `${copy.low} · ${copy.boundaryApproximate}`; } switch (point.confidence) { case 'high': return copy.high; diff --git a/src/weeklyValue.ts b/src/weeklyValue.ts index 6cbe05ca..2c8cf913 100644 --- a/src/weeklyValue.ts +++ b/src/weeklyValue.ts @@ -551,10 +551,12 @@ function latestKnownCodexResetAnchor( * observations may then decorate their matching bucket with utilization and a * conservative allowance estimate; they never add a second usage row. * - * Codex logs do not expose a reliable account identity. Any overlapping, - * non-aligned future reset makes the current allowance ambiguous regardless of - * its label. Historical Codex periods and source-mixed current periods remain - * usage-only rather than manufacturing an account split. + * Codex logs do not expose a reliable account identity, but a file source key + * is not an account key either: one Codex home commonly contributes many + * files to the same account-wide quota series. Those rows may therefore be + * aggregated for an allowance estimate, with low confidence when the source + * or reset boundary is approximate. A genuinely different quota series still + * blocks the estimate rather than manufacturing an account split. */ export function buildWeeklyValueTimeline( provider: UsageProvider, @@ -639,23 +641,47 @@ export function buildWeeklyValueTimeline( .sort((left, right) => right.resetAt - left.resetAt); } - const alignedClusters = new Map(); + type MappedObservation = { + entry: typeof validClusters[number]; + resetAligned: boolean; + }; + const historyByReset = new Map(history.map((point) => [point.resetAt, point])); + const mappedObservations = new Map(); for (const entry of validClusters) { if (entry.cluster.seriesKey !== anchorEntry.cluster.seriesKey) { continue; } - const resetAt = alignedResetAt(entry.cluster.resetAt, anchorResetAt); - if (resetAt === null) { + const alignedReset = alignedResetAt(entry.cluster.resetAt, anchorResetAt); + const target = alignedReset === null + ? history.find((point) => + entry.latest.observedAt >= point.windowStart && + entry.latest.observedAt < point.resetAt, + ) + : historyByReset.get(alignedReset); + if (!target) { continue; } - const previous = alignedClusters.get(resetAt); - if (!previous || entry.latest.observedAt > previous.latest.observedAt) { - alignedClusters.set(resetAt, entry); + const candidate: MappedObservation = { + entry, + resetAligned: alignedReset !== null, + }; + const previous = mappedObservations.get(target.resetAt); + if ( + !previous || + entry.latest.observedAt > previous.entry.latest.observedAt || + ( + entry.latest.observedAt === previous.entry.latest.observedAt && + candidate.resetAligned && + !previous.resetAligned + ) + ) { + mappedObservations.set(target.resetAt, candidate); } } const ambiguousCurrentCodexReset = provider === 'codex' && validClusters.some((entry) => { if ( entry === anchorEntry || + entry.cluster.seriesKey === anchorEntry.cluster.seriesKey || entry.cluster.resetAt <= now || alignedResetAt(entry.cluster.resetAt, anchorResetAt) !== null ) { @@ -667,10 +693,11 @@ export function buildWeeklyValueTimeline( }); return history.map((point): WeeklyValuePoint => { - const entry = alignedClusters.get(point.resetAt); - if (!entry) { + const mapped = mappedObservations.get(point.resetAt); + if (!mapped) { return point; } + const { entry, resetAligned } = mapped; const latest = entry.latest; const sourceKeys = new Set( entry.cluster.observations.flatMap((item) => item.sourceKey ? [item.sourceKey] : []), @@ -681,11 +708,17 @@ export function buildWeeklyValueTimeline( row.timestamp >= point.windowStart && row.timestamp < point.resetAt, ); - const seriesRows = sourceKeys.size === 0 + // Codex's source key identifies a local log file, not a quota account. + // The provider reports the weekly window at the home/account level, so + // filter-by-file would systematically undercount the observed window. + // Claude observations remain profile-scoped and keep their source filter. + const seriesRows = provider === 'codex' ? bucketRows - : bucketRows.filter((row) => - row.sourceKey !== undefined && sourceKeys.has(row.sourceKey), - ); + : sourceKeys.size === 0 + ? bucketRows + : bucketRows.filter((row) => + row.sourceKey !== undefined && sourceKeys.has(row.sourceKey), + ); const bucketSourceKeys = new Set( bucketRows.flatMap((row) => row.sourceKey ? [row.sourceKey] : []), ); @@ -693,7 +726,7 @@ export function buildWeeklyValueTimeline( entry.cluster.observations.some((item) => !item.sourceKey); const hasMixedMissingUsageSource = bucketSourceKeys.size > 0 && bucketRows.some((row) => !row.sourceKey && finiteNonNegative(row.totalTokens) > 0); - const codexSourceAttributionSafe = provider !== 'codex' || ( + const codexSourceAttributionExact = provider !== 'codex' || ( (sourceKeys.size === 0 && bucketSourceKeys.size === 0) || ( !hasMixedMissingObservationSource && @@ -713,10 +746,11 @@ export function buildWeeklyValueTimeline( (point.current ? now : point.resetAt) - latest.observedAt, ); const withholdInference = provider === 'codex' && ( - !point.current || - Math.abs(point.resetAt - anchorResetAt) > RESET_CLUSTER_MS || - ambiguousCurrentCodexReset || - !codexSourceAttributionSafe || + (point.current && ambiguousCurrentCodexReset) + ); + const approximateInference = provider === 'codex' && ( + !resetAligned || + !codexSourceAttributionExact || point.boundaryUncertain === true ); let fullEquivalentUsd: number | null = null; @@ -749,7 +783,7 @@ export function buildWeeklyValueTimeline( observationGapMs, confidence: fullEquivalentUsd === null ? 'usage-only' - : observationOverrun + : observationOverrun || approximateInference ? 'low' : confidenceFor(point.current, observationGapMs, point.pricingCoverage), basis: 'quota-observation', diff --git a/tests/ui/codex-visual.spec.mjs b/tests/ui/codex-visual.spec.mjs index 3f033cca..37de3f97 100644 --- a/tests/ui/codex-visual.spec.mjs +++ b/tests/ui/codex-visual.spec.mjs @@ -298,7 +298,7 @@ test('weekly allowance value uses the shared all-time chart and exposes uncertai await expect(panel).toContainText('When usage cannot be reliably attributed to a single quota observation'); await expect(panel).toContainText('no account split or allowance estimate is invented'); await expect(panel).toContainText('If an official reset falls within a recorded day'); - await expect(panel.locator('tbody')).toContainText('Usage only · Boundary approx.'); + await expect(panel.locator('tbody')).toContainText('Low · Boundary approx.'); await expect(panel.locator('.hc-col')).toHaveCount(2); await expect(panel.locator('tbody tr')).toHaveCount(2); await expect(panel.locator('thead')).toContainText('Full allowance est.'); From f0af506d305b47b6c69b74e1a1c285c349e83497 Mon Sep 17 00:00:00 2001 From: Carl <74130028+Carl723000@users.noreply.github.com> Date: Tue, 1 Sep 2026 23:55:07 +0800 Subject: [PATCH 19/72] test: establish v2.3.1 G0 acceptance baseline Register the rolling-window, reindex, month-order, quota-reset, and effort fixtures in a tracked bilingual Goal ledger. Add a cross-view monotonicity and reconciliation assertion. Co-authored-by: OpenAI Codex <215057067+openai-codex[bot]@users.noreply.github.com> --- .../2026-09-01-v2.3.1-goal-ledger.en.md | 58 +++++++++++++++++++ .../2026-09-01-v2.3.1-goal-ledger.zh-CN.md | 58 +++++++++++++++++++ src/test/codexUsage.test.ts | 22 +++++++ 3 files changed, 138 insertions(+) create mode 100644 docs/releases/2026-09-01-v2.3.1-goal-ledger.en.md create mode 100644 docs/releases/2026-09-01-v2.3.1-goal-ledger.zh-CN.md diff --git a/docs/releases/2026-09-01-v2.3.1-goal-ledger.en.md b/docs/releases/2026-09-01-v2.3.1-goal-ledger.en.md new file mode 100644 index 00000000..96757d00 --- /dev/null +++ b/docs/releases/2026-09-01-v2.3.1-goal-ledger.en.md @@ -0,0 +1,58 @@ +# Claude Code Usage v2.3.1 Goal Ledger + +This tracked ledger records G0–G6 stage status, reproducible acceptance evidence, read-only Claude CLI reviews, and residual risk. It never stores accounts, credentials, real logs, prompts, thread titles, or local paths. + +## Execution boundary + +- Branch: `codex/v2.3.1-goal-g0-g6` +- Baseline: `7fd03f3183240940d0a2d1cfe76f60cb80f0a6ae` +- Workspace: isolated worktree; the user's primary checkout and pre-existing edits remain read-only. +- Allowed: local code/document changes, tests, builds, VSIX packaging, stage commits, and the specified read-only Claude CLI checkpoints. +- Forbidden: push, Marketplace publication, remote releases, real GitHub writes, or uploading user logs and identity data. + +## Stage status + +| Stage | Status | Local commit | Verification | Claude review | Residual risk | +|---|---|---|---|---|---| +| G0 baseline and fixtures | In progress | Pending | Node 795 passed / 1 skipped; UI 90 passed | N/A | Commit fixture registry and ledger | +| G1 data contract | Not started | — | — | Pending | — | +| G2 statistics/order/effort | Not started | — | — | N/A | — | +| G3 quota observations/estimates | Not started | — | — | Pending | — | +| G4A visual/interaction/accessibility | Not started | — | — | N/A | — | +| G4B combined heatmap/share | Not started | — | — | Pending | — | +| G5 README/screenshots | Not started | — | — | Pending | — | +| G6 release candidate | Not started | — | — | N/A | — | + +## G0 baseline evidence + +### Environment and tests + +- Node: `v24.19.0` +- npm: `11.17.0` +- `npm ci`: successful. +- `npm run test:node`: 796 tests, 795 passed, 0 failed, 1 skipped. Two loopback `EPERM` failures in the first restricted run were test-sandbox artifacts; both passed when the local test servers were allowed to bind. +- `npm run test:ui`: 90/90 passed. +- Baseline worktree: clean. + +### v2.3.1 minimum regression fixture registry + +| Risk | Fixed input/test | Acceptance fact | +|---|---|---| +| Last 30 days exceeds all time | `codexUsage.test.ts`: `an inflated period sidecar falls back to the verified daily aggregate`; `v2.3.1 acceptance fixture keeps non-negative ranges monotonic and reconciled` | `today ≤ last30Days ≤ allTime`; daily/monthly totals reconcile | +| Reindex double counting | `codexIndex.test.ts`: `fully indexed unchanged corpus returns a warm no-op without body reads`; `cold scan, appended tail, and persisted reload agree` | Repeated refresh is a stable no-op and reload preserves totals | +| Active/archive duplication | `codexIndex.test.ts`: `exact active and archive copies contribute usage only once` | Identical content contributes once and dedup coverage is reported | +| Month ordering | `codexUsage.test.ts`: `all-time, monthly, and behavior views stay provider-native` | Chart/table source rows are chronological | +| Consecutive/irregular quota resets | `codexQuotaHistory.test.ts`: `irregular reset boundaries survive polling compaction`; fresh/irregular reset fixtures in `weeklyValue.test.ts` | Distinct resets remain distinct and windows are not double-counted | +| Unknown reasoning effort | Current, legacy, missing, and unsafe context fixtures in `codexParser.test.ts`; `missing model and effort values are grouped as unknown` in `codexUsage.test.ts` | Parseable values survive; only missing or safely rejected values become unknown | + +### G0 exit gate + +- [x] Isolate the candidate and leave the user checkout untouched. +- [x] Record branch, HEAD, status, and baseline tests. +- [x] Register automated fixtures for every known regression family. +- [x] Add a cross-view non-negative, monotonic, daily/monthly reconciliation assertion. +- [ ] Commit the G0 ledger and acceptance assertion. + +## Update rule + +Each stage must finish its automated verification before this ledger records its commit, evidence, Claude decision, and risk. The G6 acceptance matrix must be derived from these records; screenshots or copy cannot substitute for a failed gate. diff --git a/docs/releases/2026-09-01-v2.3.1-goal-ledger.zh-CN.md b/docs/releases/2026-09-01-v2.3.1-goal-ledger.zh-CN.md new file mode 100644 index 00000000..68c6847a --- /dev/null +++ b/docs/releases/2026-09-01-v2.3.1-goal-ledger.zh-CN.md @@ -0,0 +1,58 @@ +# Claude Code Usage v2.3.1 Goal 执行台账 + +本台账记录本地候选从 G0 到 G6 的阶段状态、可复现验收证据、Claude CLI 只读审阅和剩余风险。它不记录账号、凭证、真实日志、提示词、线程标题或本地路径。 + +## 执行边界 + +- 分支:`codex/v2.3.1-goal-g0-g6` +- 基线:`7fd03f3183240940d0a2d1cfe76f60cb80f0a6ae` +- 工作方式:隔离 worktree;用户主 checkout 及其中已有修改保持只读。 +- 允许:本地代码/文档修改、测试、构建、VSIX、阶段性本地 commit、指定检查点的 Claude CLI 只读审阅。 +- 禁止:push、Marketplace 发布、远程 release、真实 GitHub 写入、上传用户日志或身份数据。 + +## 阶段状态 + +| 阶段 | 状态 | 本地 commit | 验证摘要 | Claude 审阅 | 剩余风险 | +|---|---|---|---|---|---| +| G0 基线与夹具 | 进行中 | 待记录 | Node 795 passed / 1 skipped;UI 90 passed | 不适用 | 完成统一 fixture 登记并提交 | +| G1 数据契约 | 未开始 | — | — | 待审阅 | — | +| G2 统计/排序/effort | 未开始 | — | — | 不适用 | — | +| G3 额度观测与估算 | 未开始 | — | — | 待审阅 | — | +| G4A 视觉/交互/可访问性 | 未开始 | — | — | 不适用 | — | +| G4B 综合热力图与分享 | 未开始 | — | — | 待审阅 | — | +| G5 README/截图 | 未开始 | — | — | 待审阅 | — | +| G6 发布候选 | 未开始 | — | — | 不适用 | — | + +## G0 基线证据 + +### 环境与测试 + +- Node:`v24.19.0` +- npm:`11.17.0` +- `npm ci`:成功。 +- `npm run test:node`:796 项,795 通过,0 失败,1 跳过。首次受限运行中的两个 loopback `EPERM` 由测试沙箱引起;允许本地测试服务器绑定后均通过。 +- `npm run test:ui`:90/90 通过。 +- 基线工作树:干净。 + +### v2.3.1 最小回归夹具登记 + +| 风险 | 固定输入/测试 | 验收事实 | +|---|---|---| +| 最近 30 天大于全部时间 | `codexUsage.test.ts`:`an inflated period sidecar falls back to the verified daily aggregate`;`v2.3.1 acceptance fixture keeps non-negative ranges monotonic and reconciled` | `today ≤ last30Days ≤ allTime`,日/月汇总与总计可核对 | +| 重复索引/重复累计 | `codexIndex.test.ts`:`fully indexed unchanged corpus returns a warm no-op without body reads`;`cold scan, appended tail, and persisted reload agree` | 连续刷新为稳定 no-op,重载后总计不变 | +| 活跃/归档重复文件 | `codexIndex.test.ts`:`exact active and archive copies contribute usage only once` | 同一内容只计一次并报告去重覆盖 | +| 月度顺序 | `codexUsage.test.ts`:`all-time, monthly, and behavior views stay provider-native` | 图表/表格源数据按月份正序 | +| 连续/不规则额度重置 | `codexQuotaHistory.test.ts`:`irregular reset boundaries survive polling compaction`;`weeklyValue.test.ts` 的 fresh reset/irregular reset fixtures | 独立 reset 被保留,窗口不重复计量 | +| unknown 推理强度 | `codexParser.test.ts` 的 current/legacy/missing/unsafe context fixtures;`codexUsage.test.ts`:`missing model and effort values are grouped as unknown` | 可解析值保留;确实缺失或被安全拒绝时进入 unknown | + +### G0 通过条件 + +- [x] 隔离候选,不触碰用户 checkout。 +- [x] 记录 branch、HEAD、状态和基线测试。 +- [x] 为五类已知回归建立可重复的自动化夹具索引。 +- [x] 加入跨视图非负、单调、日/月核对断言。 +- [ ] 提交 G0 台账与新增验收断言。 + +## 后续更新规则 + +每阶段必须先完成对应自动化验证,再更新本表中的 commit、测试证据、Claude 结论和风险。G6 的最终验收矩阵应由本台账中的证据生成;任何未通过项不得用截图或文案替代。 diff --git a/src/test/codexUsage.test.ts b/src/test/codexUsage.test.ts index 9e057217..f2076b64 100644 --- a/src/test/codexUsage.test.ts +++ b/src/test/codexUsage.test.ts @@ -831,6 +831,28 @@ test('all-time, monthly, and behavior views stay provider-native', () => { assert.equal(view.behaviorScopes.allTime.childFreshShare, view.behavior.childFreshShare); }); +test('v2.3.1 acceptance fixture keeps non-negative ranges monotonic and reconciled', () => { + const view = buildCodexUsageView(snapshotFixture(), NOW); + const monthProcessed = view.monthly.reduce( + (sum, row) => sum + row.total.processed, + 0, + ); + const allDailyProcessed = view.daily.reduce( + (sum, row) => sum + row.total.processed, + 0, + ); + + assert.equal(view.today.total.processed >= 0, true); + assert.equal(view.today.total.processed <= view.last30Days.total.processed, true); + assert.equal(view.last30Days.total.processed <= view.allTime.total.processed, true); + assert.equal(monthProcessed, view.allTime.total.processed); + assert.equal(allDailyProcessed, view.allTime.total.processed); + assert.deepEqual( + view.monthly.map((row) => row.period), + [...view.monthly.map((row) => row.period)].sort(), + ); +}); + test('behavior exposes patch and tool call proxies without file or command claims', () => { const snapshot = snapshotFixture(); snapshot.files[0].structural = { From 86d95dfd4062d8279db654c31155b273a8d955c2 Mon Sep 17 00:00:00 2001 From: Carl <74130028+Carl723000@users.noreply.github.com> Date: Wed, 2 Sep 2026 00:30:28 +0800 Subject: [PATCH 20/72] feat: secure v2.3.1 local data contract Co-authored-by: OpenAI Codex <215057067+openai-codex[bot]@users.noreply.github.com> --- ARCHITECTURE-zh-CN.md | 4 +- ARCHITECTURE.md | 5 +- .../2026-09-01-v2.3.1-goal-ledger.en.md | 30 +- .../2026-09-01-v2.3.1-goal-ledger.zh-CN.md | 30 +- .../2026-09-02-v2.3.1-local-data-contract.md | 143 +++++++++ ...-09-02-v2.3.1-local-data-contract.zh-CN.md | 143 +++++++++ package.json | 5 - src/extension.ts | 33 +- src/i18n.ts | 128 +++++--- src/settings.ts | 175 +++++++++- src/test/quotaI18n.test.ts | 27 ++ src/test/settingsPrivacy.test.ts | 303 ++++++++++++++++++ src/webview.ts | 9 +- 13 files changed, 958 insertions(+), 77 deletions(-) create mode 100644 docs/superpowers/specs/2026-09-02-v2.3.1-local-data-contract.md create mode 100644 docs/superpowers/specs/2026-09-02-v2.3.1-local-data-contract.zh-CN.md create mode 100644 src/test/settingsPrivacy.test.ts diff --git a/ARCHITECTURE-zh-CN.md b/ARCHITECTURE-zh-CN.md index 4a90a2de..f22462ce 100644 --- a/ARCHITECTURE-zh-CN.md +++ b/ARCHITECTURE-zh-CN.md @@ -3,6 +3,8 @@ > 本文简要说明扩展的 provider 边界、数据流与用量语义。模块职责或 > provider 行为变化时必须同步更新。英文版见 > [`ARCHITECTURE.md`](ARCHITECTURE.md)。 +> 持久数据清单、保留、迁移、清除和网络边界的规范来源是双语 +> [`v2.3.1 本地数据契约`](docs/superpowers/specs/2026-09-02-v2.3.1-local-data-contract.zh-CN.md)。 ## 产品边界 @@ -46,7 +48,7 @@ Opt-in GitHub 认证和跨设备聚合同步延后到 v2.4.x,届时单独做 | `providers/codex/codexUsage.ts` | Codex 自然日 Today/小时、7 天、30 天、月度、task 与项目 view model 聚合,以及精确模型 API 等效成本。 | | `providers/codex/codexInsights.ts` | 确定性的结构用量建议,不读 prompt/body。 | | `codexView.ts` / `codexViewComponents.ts` | Codex 本地化文案与默认 provider contract;不负责 HTML renderer、client script 或 CSS。 | -| `settings.ts` | 权威 `SETTINGS` catalog 和 `SettingsStore`;不得散落直接读取。 | +| `settings.ts` | 权威 `SETTINGS` catalog 和 `SettingsStore`;普通值进入 configuration/globalState,BYOK 凭证只进入 SecretStorage,且绝不进入 Webview snapshot。不得散落直接读取。 | | `statusBar.ts` / `codexStatus.ts` | Provider-specific 状态展示和通用 Claude 配额格式化。 | | `webview.ts` | Claude/Codex 唯一一套 provider-aware dashboard shell、共享 render function、共享 client 行为、provider tab 与 Compare 展示。 | | `i18n.ts` | 八个 UI locale 的全部用户可见文案。 | diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index 0a82a2fd..18a7cdb1 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -4,6 +4,9 @@ > semantics. Update it whenever module ownership or provider behavior changes. > A faithful Simplified-Chinese companion lives in > [`ARCHITECTURE-zh-CN.md`](ARCHITECTURE-zh-CN.md). +> The normative persistent-data inventory, retention, migration, clearing, and +> network boundaries live in the bilingual +> [`v2.3.1 local data contract`](docs/superpowers/specs/2026-09-02-v2.3.1-local-data-contract.md). ## Product boundary @@ -54,7 +57,7 @@ aggregate sync are deferred to v2.4.x after a separate privacy review. | `providers/codex/codexUsage.ts` | Codex calendar-Today/hourly, 7-day, 30-day, monthly, task, and project view-model aggregation with exact-model API-equivalent cost. | | `providers/codex/codexInsights.ts` | Deterministic structural usage guidance; no prompt/body inspection. | | `codexView.ts` / `codexViewComponents.ts` | Codex localized-copy and default-provider contracts; no HTML renderer, client script, or CSS ownership. | -| `settings.ts` | Canonical `SETTINGS` catalog and `SettingsStore`; do not scatter direct reads. | +| `settings.ts` | Canonical `SETTINGS` catalog and `SettingsStore`; ordinary values use configuration/globalState, while BYOK credentials use SecretStorage and never enter Webview snapshots. Do not scatter direct reads. | | `statusBar.ts` / `codexStatus.ts` | Provider-specific status presentation and generic Claude quota formatting. | | `webview.ts` | Single provider-aware Claude/Codex dashboard shell, shared render functions, shared client behavior, provider tabs, and Compare presentation. | | `i18n.ts` | All user-facing copy for all eight UI locales. | diff --git a/docs/releases/2026-09-01-v2.3.1-goal-ledger.en.md b/docs/releases/2026-09-01-v2.3.1-goal-ledger.en.md index 96757d00..ae5fe7de 100644 --- a/docs/releases/2026-09-01-v2.3.1-goal-ledger.en.md +++ b/docs/releases/2026-09-01-v2.3.1-goal-ledger.en.md @@ -14,8 +14,8 @@ This tracked ledger records G0–G6 stage status, reproducible acceptance eviden | Stage | Status | Local commit | Verification | Claude review | Residual risk | |---|---|---|---|---|---| -| G0 baseline and fixtures | In progress | Pending | Node 795 passed / 1 skipped; UI 90 passed | N/A | Commit fixture registry and ledger | -| G1 data contract | Not started | — | — | Pending | — | +| G0 baseline and fixtures | Complete | `f0af506` | Node 795 passed / 1 skipped; UI 90 passed; targeted 29 passed | N/A | No blocking risk | +| G1 data contract | Complete | This checkpoint | Focused 17 passed; Node 806 passed / 1 skipped; compile passed | Conditional GO, then revision GO | Blocked legacy workspace plaintext remains until explicit recovery; host SecretStorage backing varies; unified quota store is a G3 deliverable | | G2 statistics/order/effort | Not started | — | — | N/A | — | | G3 quota observations/estimates | Not started | — | — | Pending | — | | G4A visual/interaction/accessibility | Not started | — | — | N/A | — | @@ -51,7 +51,31 @@ This tracked ledger records G0–G6 stage status, reproducible acceptance eviden - [x] Record branch, HEAD, status, and baseline tests. - [x] Register automated fixtures for every known regression family. - [x] Add a cross-view non-negative, monotonic, daily/monthly reconciliation assertion. -- [ ] Commit the G0 ledger and acceptance assertion. +- [x] Commit the G0 ledger and acceptance assertion (`f0af506`). + +## G1 local-data and privacy evidence + +### Contract and implementation + +- Added the bilingual normative local-data contract covering raw sources, indexes, derived statistics, quota observations, reset events, UI/share preferences, schema migration, retention, clearing, and every remote boundary. +- Defined anonymous account fingerprints, atomic/deduplicated observation writes, per-account isolation, and the prohibition on credentials, complete identifiers, raw CLI output, and private source content. +- Moved the advice BYOK key from contributed configuration/globalState into VS Code SecretStorage. Webview snapshots expose only `configured`, and the password field never rehydrates a stored secret. +- Migration inspects Global, current Workspace, and every open Workspace Folder. It automatically migrates only one unambiguous machine-wide value, writes secure storage first, clears plaintext second, and fails closed with localized recovery copy on workspace-only, conflicting, or provider-failure cases. + +### Verification and review + +- Focused privacy/contract suite: 17 passed. +- Full Node suite on an isolated loopback port: 807 tests, 806 passed, 0 failed, 1 filesystem-capability skip. +- Claude CLI first review: conditional GO, requesting complete scope enumeration, concurrency/idempotency protection, fixed failure UX, and explicit Settings Sync evidence. +- Revision review: GO to G2 with no blocker. Non-blocking caveats about blocked workspace plaintext and host-dependent SecretStorage backing are now explicit in the contract. + +### G1 exit gate + +- [x] Track a bilingual normative data contract in the repository. +- [x] Make every persisted data class, retention rule, clear path, and remote interaction auditable. +- [x] Migrate the only credential to SecretStorage without exposing it to Webview state. +- [x] Fail closed and preserve recoverability for ambiguous or failed legacy migration. +- [x] Pass the G1 test gate and Claude revision review. ## Update rule diff --git a/docs/releases/2026-09-01-v2.3.1-goal-ledger.zh-CN.md b/docs/releases/2026-09-01-v2.3.1-goal-ledger.zh-CN.md index 68c6847a..38773d90 100644 --- a/docs/releases/2026-09-01-v2.3.1-goal-ledger.zh-CN.md +++ b/docs/releases/2026-09-01-v2.3.1-goal-ledger.zh-CN.md @@ -14,8 +14,8 @@ | 阶段 | 状态 | 本地 commit | 验证摘要 | Claude 审阅 | 剩余风险 | |---|---|---|---|---|---| -| G0 基线与夹具 | 进行中 | 待记录 | Node 795 passed / 1 skipped;UI 90 passed | 不适用 | 完成统一 fixture 登记并提交 | -| G1 数据契约 | 未开始 | — | — | 待审阅 | — | +| G0 基线与夹具 | 已完成 | `f0af506` | Node 795 passed / 1 skipped;UI 90 passed;定向 29 passed | 不适用 | 无阻塞风险 | +| G1 数据契约 | 已完成 | 本 checkpoint | 定向 17 passed;Node 806 passed / 1 skipped;compile passed | 首轮有条件 GO,修订复核 GO | 受阻 workspace 明文需用户显式恢复;SecretStorage 底层保护依赖宿主;统一额度存储属于 G3 | | G2 统计/排序/effort | 未开始 | — | — | 不适用 | — | | G3 额度观测与估算 | 未开始 | — | — | 待审阅 | — | | G4A 视觉/交互/可访问性 | 未开始 | — | — | 不适用 | — | @@ -51,7 +51,31 @@ - [x] 记录 branch、HEAD、状态和基线测试。 - [x] 为五类已知回归建立可重复的自动化夹具索引。 - [x] 加入跨视图非负、单调、日/月核对断言。 -- [ ] 提交 G0 台账与新增验收断言。 +- [x] 提交 G0 台账与新增验收断言(`f0af506`)。 + +## G1 本地数据与隐私证据 + +### 契约与实现 + +- 新增中英文规范性本地数据契约,覆盖原始来源、索引、派生统计、额度观测、重置事件、UI/分享偏好、schema 迁移、保留期、清除方式和所有远程边界。 +- 定义匿名账号 fingerprint、原子且去重的观测写入、账号隔离,并禁止保存凭证、完整账号标识、原始 CLI 输出及私密源内容。 +- 将 AI 建议 BYOK key 从 contributed configuration/globalState 迁入 VS Code SecretStorage;Webview snapshot 只暴露 `configured`,密码框绝不回填 secret。 +- 迁移检查 Global、当前 Workspace 和所有已打开 Workspace Folder;仅自动迁移唯一且无歧义的机器级值,先写安全存储、后清明文;workspace-only、冲突值或 provider 失败均以本地化固定提示 fail closed,并保留可恢复数据。 + +### 验证与审阅 + +- 隐私/契约定向测试:17 passed。 +- 独立 loopback 端口完整 Node 测试:807 项,806 passed,0 failed,1 项因文件系统能力 skipped。 +- Claude CLI 首轮审阅:有条件 GO,要求补齐 scope 枚举、并发/幂等保护、固定失败 UX 及 Settings Sync 证据。 +- 修订复核:无阻塞,可进入 G2;关于受阻 workspace 明文和 SecretStorage 宿主差异的非阻塞提示已写入契约。 + +### G1 通过条件 + +- [x] 在 repo 内跟踪中英文规范性数据契约。 +- [x] 使所有持久化数据类别、保留期、清除路径和远程交互可审计。 +- [x] 唯一凭证迁入 SecretStorage,且不暴露到 Webview 状态。 +- [x] 对歧义或失败的旧数据迁移 fail closed 并保留可恢复性。 +- [x] 通过 G1 测试门和 Claude 修订复核。 ## 后续更新规则 diff --git a/docs/superpowers/specs/2026-09-02-v2.3.1-local-data-contract.md b/docs/superpowers/specs/2026-09-02-v2.3.1-local-data-contract.md new file mode 100644 index 00000000..6386c8c5 --- /dev/null +++ b/docs/superpowers/specs/2026-09-02-v2.3.1-local-data-contract.md @@ -0,0 +1,143 @@ +# Claude Code Usage v2.3.1 Local Data Contract + +Status: normative v2.3.1 specification. Items marked **migration input** describe legacy storage that v2.3.1 must import or clear; they are not the desired final shape. + +This document is the source of truth for data read, retained, derived, displayed, exported, or sent by the extension. Any new persistent field must update this contract, its Chinese counterpart, migration and clearing logic, privacy tests, and the README privacy table in the same change. + +## 1. Non-negotiable rules + +1. Local Claude and Codex logs remain read-only source data. The extension never deletes or rewrites them. +2. Persistent caches contain bounded aggregates, cursors, pseudonymous relationships, and minimum quota observations—not copied conversations. +3. Account identity is represented only by a stable, machine-local HMAC fingerprint. Email, display name, subscription name, raw account ID, credentials path, and reversible identifiers are forbidden. +4. Different account fingerprints are never combined. If account continuity cannot be established, observations stay in an isolated unattributed epoch and must not support account-level unused-quota claims. +5. Observations are facts; cost, full-quota, unused-quota, confidence, and heatmap intensity are recomputable derivatives. +6. API keys use VS Code SecretStorage. OAuth tokens, cookies, API keys, authorization headers, and unfiltered CLI output never enter configuration, globalState, Webview state, logs, exports, or indexes. +7. Network writes require an explicit user action and an exact preview of the destination and payload class. No background GitHub publication is permitted. +8. Every persisted object has a schema version, bounded retention, an idempotent migration, corruption handling, and a user-visible clearing route. +9. API-equivalent cost is an estimate at the current known API price, never a bill or a promise about subscription allowance. + +## 2. Data inventory + +| ID | Class and source | Persistent location | Allowed persisted fields | Retention / clearing | Remote interaction | +|---|---|---|---|---|---| +| R1 | Claude local JSONL | Source owned by Claude Code; extension cache is memory-only | Runtime token, model, date, project/session derivations; bounded prompt text only when content analysis is enabled | Source retention is controlled by Claude Code. Disable Claude analysis or clear extension-derived state; never delete R1 | None by default. A separately consented advice request may include the exact previewed bounded sample | +| R2 | Codex `sessions/**/*.jsonl` and `archived_sessions/**/*.jsonl` | Source owned by Codex; read-only | Runtime token, model, effort, timestamps, structural and lineage evidence | Source retention is controlled by Codex. Never delete R2 | None | +| P1 | Codex incremental index derived from R2 | `globalStorageUri/codex-index-v1.json`, internal schema 3 at the v2.3.1 baseline | HMAC file keys, source area, size/mtime/inode, offsets, parser state, scan coverage, bounded numeric aggregates, sanitized relationship keys, normalized limits | Retained until rebuild, clear, uninstall, or schema replacement. `Clear local derived data` removes only this file and permits rebuild | None | +| P2 | Unified quota observations derived from official provider observations | `globalStorageUri/quota-observations-v2.json`, schema 2; atomically replaced | Only fields in section 3 | Maximum 180 days and 512 retained observations per provider/account/period type after boundary-preserving compaction. Clear by provider/account epoch or all | Claude observations originate from the Anthropic quota request; Codex observations originate from local structured log events | +| P3 | Latest Claude quota response **migration input** | `ccu.usageLimits.` in globalState | Legacy response plus timestamp | v2.3.1 reads once, normalizes into P2, then removes it. No raw response is a long-term source of truth | No additional request | +| P4 | Legacy Claude weekly history **migration input** | `ccu.weeklyQuotaHistory.v1.` in globalState | Legacy normalized observations | v2.3.1 imports, deduplicates into P2, then removes it | None | +| P5 | Legacy Codex quota history **migration input** | `quotaHistory` inside P1 | provider, observed/reset time, used percent | v2.3.1 normalizes into schema 2 without inventing an account. Rebuilding P1 must not erase already migrated P2 history | None | +| P6 | Extension preferences and migration flags | `ccu.setting.`, `ccu.lastSeenVersion`, `ccu.codex.backgroundWork.v1`, migration flags in globalState; language and data directories in VS Code configuration | Typed preferences, bounded worker status, schema/migration flags | Until reset, clear, or uninstall | Settings Sync may sync VS Code configuration according to the user's VS Code account; extension globalState is not intentionally uploaded by this extension | +| P7 | Machine-local pseudonymization material | `ccu.codex.machineSalt` plus versioned local fingerprint salt | Random secret salt and algorithm version only | Until all derived data is cleared. Clearing it invalidates old fingerprints rather than relinking them | Never | +| P8 | BYOK advice credential | `claudeCodeUsage.secret.advice.apiKey` in VS Code SecretStorage | Secret value only | Until the user clears the key or extension data. VS Code documents SecretStorage as encrypted and not synchronized across machines; the concrete protection mechanism remains host-dependent | Sent only as an authorization credential to the user-configured endpoint after a separate Send action; never included in request body or preview | +| P9 | Advice-effectiveness ledger | `ccu.adviceEffectiveness.localState`, schema 3; legacy feedback key may exist | Consent mode, feedback, suppression records, comparable-pair metrics and results; no prompt body/path/account | Bounded by implementation (feedback/suppression 500, pairs/results 200). Clear Advice Data removes current and known legacy keys | Advice payload is sent only after preview and explicit Send | +| P10 | Webview preferences | Webview state and localStorage keys `ccu.activeTab`, `ccu.sessionFilter`, `ccu.sessionRange`, `ccu.sessionModel` | Tab, sort/filter/range, expanded state, scroll and chart selection | Until Reset UI State, Webview storage clear, or uninstall | Never | +| P11 | Heatmap/share preferences | `showHeatmap`, share-card settings, `ccu.heatmapRepo`, `ccu.heatmapPath`; v2.3.1 may add title/range/privacy-preview preferences | Presentation preferences and optional GitHub `owner/repo/path` strings | Until Reset Sharing Preferences or all derived data clear | Local SVG/Markdown export has none. Any future direct publish requires an exact destination confirmation every time and must not cache GitHub credentials | +| M1 | Runtime-only Claude/Codex views | Extension-host memory | Records, prepared aggregates, charts, current session mappings, bounded content sample | Released on reload/exit | None unless included in a separately previewed request/export | +| M2 | Runtime credentials and network responses | Extension-host memory or provider-owned credential store | OAuth access/refresh token, transient headers, response/error object | Never retained by this extension. Provider refresh may update the provider-owned credential store | Anthropic OAuth/quota endpoint or user-configured advice endpoint only | + +The extension must never persist raw prompt/response text, tool arguments, command text, thread titles, project names, repository remotes, absolute source-log paths, full account identifiers, OAuth tokens, cookies, API keys outside SecretStorage, or unfiltered Claude CLI output. + +## 3. Quota observation schema + +P2 is an atomically written document: + +```ts +interface QuotaObservationStoreV2 { + schemaVersion: 2; + fingerprintAlgorithm: 'hmac-sha256-v1'; + observations: QuotaObservationV2[]; +} + +interface QuotaObservationV2 { + schemaVersion: 2; + provider: 'claude' | 'codex'; + accountFingerprint: string; // machine-local HMAC or isolated unattributed epoch + accountAttribution: 'verified-local-signal' | 'profile-continuity' | 'unattributed'; + observedAt: number; // Unix milliseconds when the observation was received + periodType: 'five-hour' | 'seven-day' | 'provider-scoped-seven-day'; + usedFraction: number; // finite 0..1 + remainingFraction: number; // finite 0..1, normalized with usedFraction + resetAt: number | null; // Unix milliseconds from structured evidence when available + source: 'claude-official-api' | 'codex-local-structured-event'; + windowId: string; // HMAC(provider, fingerprint, period, reset identity/epoch) + confidence: 'high' | 'medium' | 'low'; + captureReason: 'refresh' | 'reset-at-change' | 'window-id-change' | 'usage-drop' | 'profile-change' | 'migration'; + flags: Array<'approximate-boundary' | 'low-log-coverage' | 'low-price-coverage' | 'account-ambiguous' | 'clock-anomaly'>; +} +``` + +### 3.1 Fingerprints and account isolation + +- A fingerprint is `HMAC(machineSalt, provider + stableLocalIdentitySignal + algorithmVersion)`. The raw identity signal is discarded immediately. +- A filesystem path alone identifies a local profile, not a verified account. Such observations use `profile-continuity` and cannot be merged after a detected profile/login change. +- If Codex exposes no safe stable identity signal, generate a random unattributed epoch. Never label a Codex home hash as an account. Window changes, `resetAt` changes, implausible percentage rollback, or explicit profile changes close the epoch. +- Only identical fingerprints and compatible attribution may be robustly aggregated. `unattributed` epochs never merge with one another, even when reset times coincide. + +### 3.2 Validation, deduplication, and atomicity + +- Reject non-finite times/fractions, impossible period types, observations from the far future, and unknown free-form fields. +- Normalize fractions to 0..1 only when the source semantics are known; do not silently reinterpret arbitrary values. +- Deduplication key: `provider + accountFingerprint + periodType + windowId + observedAt + usedFraction + resetAt + source` after timestamp normalization. Exact duplicates are removed. +- Semantically unchanged polls within a short interval may compact, but each window retains first, last, pre-reset, post-reset, and anomaly observations. +- Write a sanitized temporary file in the same directory, flush and close it, then atomically rename it over the canonical file. A failed write leaves the prior valid file intact. +- Corrupt or future-schema files are quarantined with a bounded diagnostic name; no raw contents enter logs. Rebuild or migration never overwrites the only recoverable source before validation succeeds. + +### 3.3 Reset events and derived estimates + +A reset/window boundary is detected from, in priority order: a stable provider window identifier change, `resetAt` identity change, or a significant used-fraction rollback that exceeds the configured noise tolerance. Multiple boundaries on the same day are retained as separate events. Event timestamps split usage when available; day-only aggregates are explicitly approximate. + +For each valid observation in one stable fingerprint/window: + +```text +candidateFullEquivalent = pricedUsedEquivalentThroughObservedAt / usedFraction +``` + +Candidates require `0 < usedFraction <= 1`. They are robustly aggregated with weights for price coverage, log/index coverage, time-boundary quality, account attribution, and observation recency. The displayed total is never below final confirmed used equivalent. Unused value is `max(full - confirmedUsed, 0)` and is shown only when account continuity and window boundaries are adequate. Multi-account or unattributed evidence may show used equivalent and an explanation, but no account-level unused claim. + +Full/unused estimates are not persisted facts. They are recomputed when pricing, coverage, logs, or observation evidence changes. + +## 4. Aggregation and combined heatmap semantics + +All date keys use the configured data timezone. “Last 30 days” means today plus the previous 29 calendar dates in that timezone. Daily, monthly, weekly, model, effort, and grand totals must reconcile; repeated refresh/reindex must be idempotent. + +The v2.3.1 combined heatmap reuses provider daily aggregates and must not rescan source logs or create a second statistics cache. + +```text +Claude processed = input + cache_creation + cache_read + output +Codex processed = input_total + output_total +Combined activity volume = Claude processed + Codex processed +``` + +Codex cached input is already included in `input_total`; reasoning is already included in `output_total`. Neither is added twice. The combined number is a local activity volume only—not productivity, billable usage, model capability, or provider equivalence. The tooltip always exposes both provider components. Cost may be an optional view only with price-coverage disclosure. + +## 5. Exports, sharing, and network boundaries + +- Deterministic local SVG and Markdown snippets may contain only selected date range, title, normalized daily aggregates, provider labels, totals, and explicit caveats. +- Export/share payloads must pass privacy-canary tests proving absence of account IDs, fingerprints, project/thread names, repository names, local paths, prompt/log text, and credentials. +- Optional GitHub owner/repo/path preferences are destinations, not credentials. GitHub authentication remains transient in the VS Code authentication provider. +- v2.3.1 release validation uses mocked GitHub publication only. No Goal step may write to a real repository. +- AI advice requires exact request preview and a separate Send action. Quota tracking sends no local logs. Pricing refresh sends no user usage. + +## 6. User controls + +By G6, the Settings/Data surface and command palette must expose: + +1. View local data inventory: category, location class (not a sensitive absolute path), schema, approximate size/count, oldest/newest timestamp, and possible network interaction. +2. Clear Codex derived index and rebuild it without touching source logs. +3. Clear quota observation history by provider/account epoch or all; this is irreversible but does not alter provider quotas. +4. Clear advice ledger/current and known legacy keys without clearing external accounts. +5. Reset UI state and sharing preferences independently. +6. Clear the BYOK secret from SecretStorage. +7. Clear all extension-derived data only after listing exact targets; source Claude/Codex logs and provider-owned credentials are always excluded. + +Uninstall behavior is disclosed rather than assumed: VS Code may retain extension global storage until it is removed by the host/user, so the explicit clear command is the reliable deletion route. + +## 7. Migration and release gates + +- Migration order: load and validate current schema; migrate legacy quota/config values into a new temporary document; validate invariants; atomically install it; only then delete legacy plaintext/duplicate state. +- Secret migration writes SecretStorage first, then removes all explicit Global, current Workspace, every open Workspace Folder, and old globalState value. Every closed workspace is inspected when it is next opened. A workspace-only or conflicting key is never silently promoted into the one global secret slot: activation stops with fixed, localized recovery instructions and leaves plaintext recoverable. Until the user removes or reconciles that blocked legacy workspace value, it remains ordinary configuration and may remain eligible for Settings Sync. Repeat/concurrent machine-wide migration is idempotent because only identical global candidates are automatic. Other failures likewise preserve the legacy value and stop activation rather than pretending migration succeeded. +- [VS Code's extension storage documentation](https://code.visualstudio.com/api/extension-capabilities/common-capabilities#data-storage) states that SecretStorage is encrypted and is not synchronized across machines. The extension relies on the active VS Code-compatible host's SecretStorage implementation, whose concrete backing protection can vary by host. Ordinary VS Code configuration can still participate in Settings Sync according to the user's VS Code settings. +- Unknown reasoning effort is retained only when the field is missing or unrecognized. Non-zero unknown is visible and explained; zero unknown is omitted. Effort components reconcile with provider totals. +- Tests must cover old/current/future schema, duplicate observations, corruption, consecutive and same-day resets, rollback noise, account/profile changes, low price/log coverage, timezone boundaries, deterministic/private exports, and clear-data exclusion of source logs. +- G1, G3, G4B, and G5 require the specified read-only Claude CLI checkpoint. A passing review does not replace local tests or the main agent's acceptance decision. diff --git a/docs/superpowers/specs/2026-09-02-v2.3.1-local-data-contract.zh-CN.md b/docs/superpowers/specs/2026-09-02-v2.3.1-local-data-contract.zh-CN.md new file mode 100644 index 00000000..7577e80d --- /dev/null +++ b/docs/superpowers/specs/2026-09-02-v2.3.1-local-data-contract.zh-CN.md @@ -0,0 +1,143 @@ +# Claude Code Usage v2.3.1 本地数据契约 + +状态:v2.3.1 规范性说明。标记为 **迁移输入** 的项目描述旧版存储,只能用于迁移或清除,不是最终目标结构。 + +本文件是插件读取、保留、派生、展示、导出或发送数据的事实来源。任何新增持久字段,都必须在同一变更中更新本文件、英文版、迁移与清除逻辑、隐私测试和 README 隐私表。 + +## 1. 不可放宽的原则 + +1. 本机 Claude/Codex 日志始终是只读源数据;插件绝不删除或改写它们。 +2. 持久缓存只保存有界聚合、游标、匿名关系和最小额度观测,不复制完整会话。 +3. 账号只能表示为本机稳定的 HMAC 匿名 fingerprint。禁止保存邮箱、显示名、订阅名、原始账号 ID、凭证路径或可逆身份标识。 +4. 不同 fingerprint 的观测绝不合并。无法证明账号连续性时,观测进入隔离的未归属 epoch,不得据此声称账号级“未用额度”。 +5. 官方百分比和重置时间是观测事实;成本、总周额度、未用额度、可信度和热力图强度均为可重算派生值。 +6. API key 只能进入 VS Code SecretStorage。OAuth token、cookie、API key、Authorization header 和未经筛选的 CLI 输出不得进入 configuration、globalState、Webview 状态、日志、导出或索引。 +7. 远程写入必须由用户明确触发,并预览精确目标和载荷类别;禁止后台发布 GitHub 内容。 +8. 每个持久对象都要有 schema 版本、有界保留期、幂等迁移、损坏处理和用户可见的清除入口。 +9. API 等效成本只是按当前已知单价计算的估算,不是账单,也不是官方订阅额度承诺。 + +## 2. 本地数据清单 + +| ID | 数据类别与来源 | 持久位置 | 允许持久化的字段 | 保留与清除 | 远程交互 | +|---|---|---|---|---|---| +| R1 | Claude 本机 JSONL | Claude Code 自有源文件;插件聚合主要在内存 | 运行期 token、模型、日期、项目/会话派生信息;仅内容分析开启时使用有界 prompt 文本 | 源文件保留由 Claude Code 控制;插件绝不删除 R1 | 默认无。只有用户单独同意并确认 AI 建议请求时,才可能发送精确预览中的有界样本 | +| R2 | Codex `sessions/**/*.jsonl` 与 `archived_sessions/**/*.jsonl` | Codex 自有源文件,只读 | 运行期 token、模型、effort、时间、结构和 lineage 证据 | 源文件保留由 Codex 控制;插件绝不删除 R2 | 无 | +| P1 | R2 派生的 Codex 增量索引 | `globalStorageUri/codex-index-v1.json`;v2.3.1 基线内部 schema 3 | HMAC fileKey、来源区、size/mtime/inode、offset、解析器状态、扫描覆盖、有界数值聚合、清洗后的关系键和规范化 limit | 保留到重建、显式清除、卸载或 schema 替换;“清除本地派生数据”只删除此文件并允许重建 | 无 | +| P2 | 官方观测派生的统一额度历史 | `globalStorageUri/quota-observations-v2.json`,schema 2,原子替换 | 仅第 3 节定义字段 | 每个 provider/account/周期类型最多保留 180 天、512 条;压缩时保留窗口边界。可按 provider/account epoch 或全部清除 | Claude 来源为 Anthropic 额度请求;Codex 来源为本机结构化日志事件 | +| P3 | 最新 Claude 额度 response **迁移输入** | globalState `ccu.usageLimits.` | 旧版 response 与时间戳 | v2.3.1 读取一次并规范化到 P2,成功后删除;完整 response 不再作为长期事实 | 不新增请求 | +| P4 | Claude 周额度历史 **迁移输入** | globalState `ccu.weeklyQuotaHistory.v1.` | 旧版规范化观测 | 导入 P2、去重后删除 | 无 | +| P5 | Codex 旧额度历史 **迁移输入** | P1 内 `quotaHistory` | provider、观测/重置时间、已用百分比 | 迁移为 schema 2,不虚构账号;重建 P1 不得抹掉已迁移 P2 历史 | 无 | +| P6 | 插件设置与迁移标记 | globalState 中 `ccu.setting.`、`ccu.lastSeenVersion`、`ccu.codex.backgroundWork.v1` 及迁移 flag;语言和数据目录在 VS Code configuration | 类型化偏好、有界后台任务状态、schema/迁移标记 | 直到重置、清除或卸载 | VS Code configuration 可能按用户的 VS Code 账号设置参与 Settings Sync;插件自身不主动上传 globalState | +| P7 | 本机匿名化材料 | `ccu.codex.machineSalt` 与带版本的 fingerprint salt | 随机盐和算法版本 | 保留到清除全部派生数据;清除后旧 fingerprint 失效,不跨盐重连 | 永不外发 | +| P8 | AI 建议 BYOK 凭证 | VS Code SecretStorage 的 `claudeCodeUsage.secret.advice.apiKey` | secret 值本身 | 用户清除 key 或清除插件数据前保留;VS Code 官方说明 SecretStorage 会加密且不跨设备同步,具体保护机制仍取决于宿主实现 | 只有用户预览后再次点击 Send,才作为授权凭证发送给用户配置的 endpoint;不进入请求正文或预览 | +| P9 | 建议效果账本 | `ccu.adviceEffectiveness.localState` schema 3;可能仍有旧 feedback key | consent 模式、反馈、抑制记录、可比较任务指标和结果;无 prompt/path/account | 实现上限:反馈/抑制约 500,对与结果约 200;“清除建议数据”同时删除当前键和已知旧键 | 请求只有在预览后显式 Send 才外发 | +| P10 | Webview 偏好 | Webview state 与 localStorage:`ccu.activeTab`、`ccu.sessionFilter`、`ccu.sessionRange`、`ccu.sessionModel` | tab、排序/筛选/范围、展开状态、滚动和图表选择 | 直到重置 UI、清除 Webview 存储或卸载 | 无 | +| P11 | 热力图/分享偏好 | `showHeatmap`、share-card 设置、`ccu.heatmapRepo`、`ccu.heatmapPath`;v2.3.1 可新增标题、时间范围和隐私预览偏好 | 展示偏好及可选 GitHub `owner/repo/path` 字符串 | 直到重置分享偏好或清除派生数据 | 本地 SVG/Markdown 导出不联网。任何未来直接发布都必须逐次确认精确目标,且不得缓存 GitHub 凭证 | +| M1 | Claude/Codex 运行期视图 | Extension Host 内存 | records、prepared aggregate、图表、当前会话映射和有界内容样本 | reload/退出即释放 | 除非进入单独预览并确认的请求/导出,否则无 | +| M2 | 运行期凭证和网络 response | Extension Host 内存或 provider 自有凭证存储 | OAuth access/refresh token、临时 header、response/error object | 插件绝不长期保留;provider 刷新流程可能更新其自有凭证存储 | 仅 Anthropic OAuth/额度端点或用户配置的建议端点 | + +插件绝不能持久化原始 prompt/response、tool arguments、命令文本、线程标题、项目名、repo remote、源日志绝对路径、完整账号标识、OAuth token、cookie、SecretStorage 之外的 API key,或未过滤的 Claude CLI 输出。 + +## 3. 额度观测 schema + +P2 是原子写入的文档: + +```ts +interface QuotaObservationStoreV2 { + schemaVersion: 2; + fingerprintAlgorithm: 'hmac-sha256-v1'; + observations: QuotaObservationV2[]; +} + +interface QuotaObservationV2 { + schemaVersion: 2; + provider: 'claude' | 'codex'; + accountFingerprint: string; // 本机 HMAC 或隔离的未归属 epoch + accountAttribution: 'verified-local-signal' | 'profile-continuity' | 'unattributed'; + observedAt: number; // Unix 毫秒 + periodType: 'five-hour' | 'seven-day' | 'provider-scoped-seven-day'; + usedFraction: number; // 有限数,0..1 + remainingFraction: number; // 有限数,0..1,与 usedFraction 规范化一致 + resetAt: number | null; + source: 'claude-official-api' | 'codex-local-structured-event'; + windowId: string; // HMAC(provider, fingerprint, period, reset identity/epoch) + confidence: 'high' | 'medium' | 'low'; + captureReason: 'refresh' | 'reset-at-change' | 'window-id-change' | 'usage-drop' | 'profile-change' | 'migration'; + flags: Array<'approximate-boundary' | 'low-log-coverage' | 'low-price-coverage' | 'account-ambiguous' | 'clock-anomaly'>; +} +``` + +### 3.1 Fingerprint 与账号隔离 + +- fingerprint 采用 `HMAC(machineSalt, provider + stableLocalIdentitySignal + algorithmVersion)`;原始 identity signal 立即丢弃。 +- 文件路径只能说明本机 profile,不能证明真实账号。此类观测标为 `profile-continuity`,检测到登录/profile 改变后不得继续合并。 +- Codex 若没有安全稳定的账号信号,就生成随机 `unattributed` epoch,绝不能把 Codex home hash 冒充账号。窗口变化、`resetAt` 变化、明显百分比回落或显式 profile 变化都会关闭 epoch。 +- 只有 fingerprint 相同且 attribution 兼容的观测才能稳健聚合。不同 `unattributed` epoch 即使 reset 时间相同也绝不合并。 + +### 3.2 校验、去重和原子写入 + +- 拒绝非有限时间/比例、非法周期、明显未来时间和未知自由字段。 +- 只有确认来源语义时才把比例规范化为 0..1,不能静默猜测任意数值。 +- 去重键:规范化后的 `provider + accountFingerprint + periodType + windowId + observedAt + usedFraction + resetAt + source`。 +- 短时间内语义不变的轮询可以压缩,但每个窗口必须保留首条、末条、reset 前后及异常观测。 +- 在同一目录写入清洗后的临时文件,flush/close 后原子 rename 覆盖 canonical 文件;失败时旧文件保持有效。 +- 损坏或未来 schema 文件使用有界诊断名隔离,原文不得进入日志;迁移验证成功前不得覆盖唯一可恢复版本。 + +### 3.3 重置事件与派生估算 + +重置/窗口边界按以下优先级判断:稳定 window ID 改变、`resetAt` identity 改变、超过噪声阈值的 used fraction 回落。同一天多次重置必须分别保留。有事件时间戳时按事件切分;只有日汇总时明确标记近似。 + +同一稳定 fingerprint/window 内,每个有效观测产生候选: + +```text +candidateFullEquivalent = pricedUsedEquivalentThroughObservedAt / usedFraction +``` + +只接受 `0 < usedFraction <= 1`。多个候选按定价覆盖、日志/索引覆盖、边界质量、账号归属和时效做稳健聚合。展示的总额度不得低于最终确认的已用等价值;未用值为 `max(full - confirmedUsed, 0)`,仅在账号连续性与窗口边界足够可靠时显示。多账号或未归属证据可以显示已用值及原因,但不得声称账号级未用额度。 + +总额/未用估算不作为永久事实落盘;单价、覆盖、日志或观测变化后重新计算。 + +## 4. 聚合与综合热力图语义 + +所有日期键都使用用户配置的数据时区。“最近 30 天”定义为该时区下今天及此前 29 个自然日期。日/月/周/模型/effort/总计必须守恒;重复刷新或重建索引必须幂等。 + +v2.3.1 综合热力图复用 provider 日汇总,不重复扫描源日志,也不新增第二套统计缓存。 + +```text +Claude processed = input + cache_creation + cache_read + output +Codex processed = input_total + output_total +综合活动量 = Claude processed + Codex processed +``` + +Codex cached input 已包含在 `input_total`,reasoning 已包含在 `output_total`,都不得重复相加。综合值只表示本机活动量,不代表生产率、账单、模型能力或 provider 等价。tooltip 必须分别展示两端构成。成本只能是可选视图,并披露定价覆盖率。 + +## 5. 导出、分享和网络边界 + +- 确定性本地 SVG 与 Markdown 片段只能包含所选日期范围、标题、规范化日聚合、provider 标签、总计和明确限制说明。 +- 导出/分享必须通过隐私 canary 测试,确保不存在账号 ID/fingerprint、项目/线程名、repo 名、本地路径、prompt/log 文本或凭证。 +- GitHub owner/repo/path 是目的地偏好,不是凭证;GitHub 登录只临时使用 VS Code authentication provider。 +- v2.3.1 的发布验证只允许 mock GitHub publish;Goal 全程不得写入真实仓库。 +- AI 建议必须先给出精确请求预览,再单独点击 Send;额度跟踪不发送本地日志;价格刷新不发送用户用量。 + +## 6. 用户数据控制 + +G6 前,设置/Data 页面和命令面板应提供: + +1. 查看本地数据清单:类别、非敏感位置类别、schema、近似大小/条数、最早/最新时间和可能的网络交互。 +2. 清除 Codex 派生索引并重建,不触碰源日志。 +3. 按 provider/account epoch 或全部清除额度历史;该操作不可恢复,但不会改变官方额度。 +4. 清除当前及已知旧版建议账本,不影响外部账号。 +5. 分别重置 UI 状态和分享偏好。 +6. 从 SecretStorage 清除 BYOK key。 +7. “清除全部插件派生数据”执行前列出精确目标;Claude/Codex 源日志和 provider 自有凭证永远排除。 + +不能假定卸载一定删除 VS Code global storage;README 应披露这一点,显式清除命令才是可靠删除路径。 + +## 7. 迁移与发版门槛 + +- 迁移顺序:加载并校验当前 schema;将旧额度/config 写入新临时文档;验证不变量;原子安装;最后删除旧明文或重复状态。 +- Secret 迁移先写 SecretStorage,再清除 Global、当前 Workspace、所有已打开 Workspace Folder 和旧 globalState 中的明文值;关闭的 workspace 会在下次打开时检查。workspace-only 或相互冲突的 key 不会静默提升为一个全局 secret:插件用固定、已本地化的恢复提示停止激活,并保留可恢复明文。在用户删除或协调该受阻的旧 workspace 值之前,它仍属于普通 configuration,因而仍可能参与 Settings Sync。机器级自动迁移只接受相同候选,因此重复/并发激活保持幂等。其他失败同样保留旧值并停止,不得假装迁移成功。 +- [VS Code 扩展存储官方说明](https://code.visualstudio.com/api/extension-capabilities/common-capabilities#data-storage)指出 SecretStorage 会加密且不跨设备同步;插件依赖当前 VS Code 兼容宿主的 SecretStorage 实现,而其具体底层保护方式可能因宿主而异。普通 VS Code configuration 仍可能按用户的 Settings Sync 设置同步。 +- `unknown` effort 只在字段缺失或无法识别时保留;非零时展示并解释,零时隐藏;各 effort 分量必须与 provider 总计守恒。 +- 测试覆盖旧/当前/未来 schema、重复观测、损坏、连续及同日重置、回落噪声、账号/profile 改变、低定价/日志覆盖、时区边界、确定性/隐私导出及清除数据不触碰源日志。 +- G1、G3、G4B、G5 执行指定的只读 Claude CLI 检查点。Claude 通过不替代本地测试和主代理的验收判断。 diff --git a/package.json b/package.json index bbc0d0bd..45145fa1 100644 --- a/package.json +++ b/package.json @@ -69,11 +69,6 @@ ], "default": "auto", "description": "Display language (auto-detection or manual selection). All other settings now live in the dashboard's ⚙ Settings tab (Show Usage Details)." - }, - "claudeCodeUsage.advice.apiKey": { - "type": "string", - "default": "", - "description": "Bring-your-own API key for AI advice and the Usage Optimizer. Requests are sent only after you preview and explicitly send them; Claude Code subscription credentials are never used. Other advice options live in the dashboard's ⚙ Settings tab." } } }, diff --git a/src/extension.ts b/src/extension.ts index 7457abad..1697abc7 100644 --- a/src/extension.ts +++ b/src/extension.ts @@ -24,7 +24,7 @@ import { buildOptimizerSystemPrompt, } from './advisor'; import { ClaudeApiUsageResponse, ContentAnalysis, ExtensionConfig } from './types'; -import { SettingsStore } from './settings'; +import { SettingsSecretMigrationError, SettingsStore } from './settings'; import { normalizeQuotaWindows } from './quotaWindows'; import { appendWeeklyQuotaObservations, @@ -229,12 +229,15 @@ export class ClaudeCodeUsageExtension { private disposed = false; private disposal: Promise | null = null; - constructor(private context: vscode.ExtensionContext) { + constructor( + private context: vscode.ExtensionContext, + settings?: SettingsStore, + ) { console.log('Claude Code Usage Extension: Constructor called'); this.outputChannel = vscode.window.createOutputChannel('Claude Code Usage'); context.subscriptions.push(this.outputChannel); this.statusBar = new StatusBarManager(); - this.settings = new SettingsStore(context); + this.settings = settings ?? new SettingsStore(context); const backgroundRestore = restoreBackgroundWorkState( context.globalState.get( ClaudeCodeUsageExtension.CODEX_BACKGROUND_WORK_STATE_KEY, @@ -831,8 +834,8 @@ export class ClaudeCodeUsageExtension { } private getConfiguration(): ExtensionConfig { - // All settings now flow through SettingsStore: the core trio (language, - // dataDirectory, advice.apiKey) still lives in VS Code config; the rest in + // All settings flow through SettingsStore: language and dataDirectory live + // in VS Code config, BYOK secrets live in SecretStorage, and the rest use // the dashboard-managed store. Defaults come from the settings catalog. const s = this.settings; return { @@ -2924,10 +2927,26 @@ export class ClaudeCodeUsageExtension { let activeExtension: ClaudeCodeUsageExtension | null = null; -export function activate(context: vscode.ExtensionContext) { +export async function activate(context: vscode.ExtensionContext): Promise { console.log('Claude Code Usage extension is now active'); - const extension = new ClaudeCodeUsageExtension(context); + const settings = new SettingsStore(context); + I18n.setLanguage(settings.get('language') as any); + try { + await settings.initializeSecrets(); + } catch (error) { + const needsManualWorkspaceMigration = + error instanceof SettingsSecretMigrationError && + (error.code === 'legacy-secret-conflict' || + error.code === 'workspace-secret-requires-manual-migration'); + await vscode.window.showErrorMessage( + needsManualWorkspaceMigration + ? I18n.t.popup.secretMigrationWorkspace + : I18n.t.popup.secretMigrationFailed, + ); + throw error; + } + const extension = new ClaudeCodeUsageExtension(context, settings); activeExtension = extension; context.subscriptions.push({ dispose: () => { void extension.dispose(); } diff --git a/src/i18n.ts b/src/i18n.ts index 5309b14b..fba8a26b 100644 --- a/src/i18n.ts +++ b/src/i18n.ts @@ -123,6 +123,8 @@ export interface Translations { settings: string; settingsTab: string; settingsIntro: string; + secretMigrationFailed: string; + secretMigrationWorkspace: string; settingsResetAll: string; settingsGroupGeneral: string; settingsGroupProviders: string; @@ -1186,7 +1188,11 @@ const translations: Record = { settings: 'Settings', settingsTab: 'Settings', settingsIntro: - 'Settings live here now. Only language, data directory and API key remain in VS Code Settings (so they sync). Changes apply immediately.', + 'Settings live here now. Language and data directories remain in VS Code Settings; API keys use VS Code SecretStorage and never sync. Changes apply immediately.', + secretMigrationFailed: + 'Claude Code Usage could not move the saved advice API key into encrypted SecretStorage. The extension stopped before deleting the old value. Fix the legacy setting and reload the window.', + secretMigrationWorkspace: + 'A workspace-specific advice API key cannot be migrated safely into one global SecretStorage entry. Copy it, remove it from workspace settings, reload, then enter it under Dashboard Settings → Advice.', settingsResetAll: 'Reset all to defaults', settingsGroupGeneral: 'General', settingsGroupProviders: 'Providers', @@ -1435,10 +1441,10 @@ const translations: Record = { '> feature produces. It is **not** based on your actual Claude Code\n' + '> usage data — nothing was sent to any API to generate this.\n\n' + '### To get real, personalised advice based on YOUR usage:\n\n' + - '1. Open Settings (`Ctrl+,` / `Cmd+,`)\n' + - '2. Search for **`claudeCodeUsage.advice.apiKey`**\n' + - '3. Paste an OpenAI-compatible API key — DeepSeek works out of the box\n' + - ' ([deepseek.com](https://platform.deepseek.com))\n' + + '1. Run **`Claude Code Usage: Show Usage Details`**\n' + + '2. Open the dashboard **Settings → Advice** section\n' + + '3. Paste an OpenAI-compatible API key — it is kept in VS Code SecretStorage\n' + + ' DeepSeek: [deepseek.com](https://platform.deepseek.com)\n' + '4. Re-run **`Claude Code Usage: Get AI Usage Advice`**', costComposition: 'Cost Composition', date: 'Date', @@ -1481,7 +1487,11 @@ const translations: Record = { settings: "Einstellungen", settingsTab: "Einstellungen", settingsIntro: - "Die Einstellungen sind jetzt hier. Nur Sprache, Datenverzeichnis und API-Schlüssel bleiben in den VS-Code-Einstellungen (damit sie synchronisiert werden). Änderungen wirken sofort.", + "Die Einstellungen sind jetzt hier. Sprache und Datenverzeichnisse bleiben in den VS-Code-Einstellungen; API-Schlüssel liegen in VS Code SecretStorage und werden nie synchronisiert. Änderungen wirken sofort.", + secretMigrationFailed: + 'Der gespeicherte API-Schlüssel konnte nicht in den verschlüsselten SecretStorage verschoben werden. Die Erweiterung wurde beendet, bevor der alte Wert gelöscht wurde. Korrigieren Sie die alte Einstellung und laden Sie das Fenster neu.', + secretMigrationWorkspace: + 'Ein arbeitsbereichsspezifischer API-Schlüssel kann nicht sicher in einen globalen SecretStorage-Eintrag migriert werden. Kopieren und entfernen Sie ihn aus den Arbeitsbereichseinstellungen, laden Sie neu und geben Sie ihn unter Dashboard-Einstellungen → Beratung ein.', settingsResetAll: "Alle zurücksetzen", settingsGroupGeneral: "Allgemein", settingsGroupProviders: "Anbieter", @@ -1731,10 +1741,10 @@ const translations: Record = { '> Ihren tatsächlichen Nutzungsdaten — es wurde nichts an eine API\n' + '> gesendet, um diesen Text zu generieren.\n\n' + '### Für echten, personalisierten Rat basierend auf IHRER Nutzung:\n\n' + - '1. Einstellungen öffnen (`Ctrl+,` / `Cmd+,`)\n' + - '2. Nach **`claudeCodeUsage.advice.apiKey`** suchen\n' + - '3. Einen OpenAI-kompatiblen API-Key einfügen — DeepSeek funktioniert\n' + - ' sofort ([deepseek.com](https://platform.deepseek.com))\n' + + '1. **`Claude Code Usage: Show Usage Details`** ausführen\n' + + '2. Im Dashboard **Einstellungen → Beratung** öffnen\n' + + '3. Einen OpenAI-kompatiblen API-Key einfügen — er bleibt in VS Code SecretStorage\n' + + ' DeepSeek: [deepseek.com](https://platform.deepseek.com)\n' + '4. **`Claude Code Usage: Get AI Usage Advice`** erneut ausführen', costComposition: "Kostenzusammensetzung", date: "Datum", @@ -1779,7 +1789,11 @@ const translations: Record = { settings: '設定', settingsTab: '設定', settingsIntro: - '設定現在都在這裡。只有語言、資料目錄與 API 金鑰仍留在 VS Code 設定中(以便同步)。變更會立即生效。', + '設定現在都在這裡。語言與資料目錄仍使用 VS Code 設定;API 金鑰存於 VS Code SecretStorage,絕不同步。變更會立即生效。', + secretMigrationFailed: + '無法將已儲存的建議 API 金鑰移入加密 SecretStorage。擴充套件已在刪除舊值前停止。請修正舊設定並重新載入視窗。', + secretMigrationWorkspace: + '工作區專用 API 金鑰無法安全遷移至單一全域 SecretStorage。請先複製並從工作區設定移除,重新載入後在「儀表板設定 → 建議」輸入。', settingsResetAll: '全部還原為預設', settingsGroupGeneral: '一般', settingsGroupProviders: '供應商', @@ -2024,10 +2038,10 @@ const translations: Record = { '> 它**不是**基於你實際的 Claude Code 用量資料 ——\n' + '> 沒有任何資料被送往 API 來產生本內容。\n\n' + '### 要取得基於你實際用量的個人化建議:\n\n' + - '1. 開啟設定(`Ctrl+,` / `Cmd+,`)\n' + - '2. 搜尋 **`claudeCodeUsage.advice.apiKey`**\n' + - '3. 貼入 OpenAI-相容 API key —— DeepSeek 開箱即用\n' + - ' ([deepseek.com](https://platform.deepseek.com))\n' + + '1. 執行 **`Claude Code Usage: Show Usage Details`**\n' + + '2. 開啟儀表板的 **設定 → 建議** 區段\n' + + '3. 貼入 OpenAI 相容 API key —— 金鑰只會存於 VS Code SecretStorage\n' + + ' DeepSeek: [deepseek.com](https://platform.deepseek.com)\n' + '4. 重新執行 **`Claude Code Usage: Get AI Usage Advice`**', costComposition: '成本構成', date: '日期', @@ -2070,7 +2084,11 @@ const translations: Record = { settings: '设置', settingsTab: '设置', settingsIntro: - '设置现在都在这里。只有语言、数据目录和 API key 仍留在 VS Code 设置中(便于同步)。更改即时生效。', + '设置现在都在这里。语言和数据目录仍使用 VS Code 设置;API 密钥存入 VS Code SecretStorage,绝不同步。更改即时生效。', + secretMigrationFailed: + '无法把已保存的建议 API 密钥迁入加密 SecretStorage。插件已在删除旧值前停止。请修正旧设置并重新加载窗口。', + secretMigrationWorkspace: + '工作区专用 API 密钥无法安全迁移到单一全局 SecretStorage。请先复制并从工作区设置中移除,重新加载后在“仪表板设置 → 建议”中输入。', settingsResetAll: '全部恢复默认', settingsGroupGeneral: '常规', settingsGroupProviders: '供应商', @@ -2315,10 +2333,10 @@ const translations: Record = { '> 它**不是**基于你实际的 Claude Code 用量数据 ——\n' + '> 没有任何数据被发往 API 来生成本内容。\n\n' + '### 要获得基于你实际用量的个性化建议:\n\n' + - '1. 打开设置(`Ctrl+,` / `Cmd+,`)\n' + - '2. 搜索 **`claudeCodeUsage.advice.apiKey`**\n' + - '3. 填入 OpenAI-兼容 API key —— DeepSeek 开箱即用\n' + - ' ([deepseek.com](https://platform.deepseek.com))\n' + + '1. 运行 **`Claude Code Usage: Show Usage Details`**\n' + + '2. 打开仪表板的 **设置 → 建议** 区域\n' + + '3. 填入 OpenAI 兼容 API key —— 密钥只存入 VS Code SecretStorage\n' + + ' DeepSeek: [deepseek.com](https://platform.deepseek.com)\n' + '4. 重新运行 **`Claude Code Usage: Get AI Usage Advice`**', costComposition: '成本构成', date: '日期', @@ -2361,7 +2379,11 @@ const translations: Record = { settings: '設定', settingsTab: '設定', settingsIntro: - '設定はここにまとまりました。言語・データディレクトリ・API キーのみ VS Code 設定に残ります(同期のため)。変更は即時反映されます。', + '設定はここにまとまりました。言語とデータディレクトリは VS Code 設定を使い、API キーは同期されない VS Code SecretStorage に保存されます。変更は即時反映されます。', + secretMigrationFailed: + '保存済みのアドバイス API キーを暗号化された SecretStorage に移動できませんでした。旧値を削除する前に拡張機能を停止しました。旧設定を修正してウィンドウを再読み込みしてください。', + secretMigrationWorkspace: + 'ワークスペース固有の API キーを単一のグローバル SecretStorage に安全に移行できません。キーをコピーしてワークスペース設定から削除し、再読み込み後に「ダッシュボード設定 → アドバイス」で入力してください。', settingsResetAll: 'すべて既定値に戻す', settingsGroupGeneral: '一般', settingsGroupProviders: 'プロバイダー', @@ -2611,10 +2633,10 @@ const translations: Record = { '> に基づくものでは**ありません** —— この内容を生成するために\n' + '> API にデータは送信されていません。\n\n' + '### あなたの実際の使用量に基づくパーソナライズされたアドバイスを取得するには:\n\n' + - '1. 設定を開く(`Ctrl+,` / `Cmd+,`)\n' + - '2. **`claudeCodeUsage.advice.apiKey`** を検索\n' + - '3. OpenAI 互換 API キーを貼り付け —— DeepSeek はすぐに使えます\n' + - ' ([deepseek.com](https://platform.deepseek.com))\n' + + '1. **`Claude Code Usage: Show Usage Details`** を実行\n' + + '2. ダッシュボードの **設定 → アドバイス** を開く\n' + + '3. OpenAI 互換 API キーを貼り付け —— VS Code SecretStorage のみに保存されます\n' + + ' DeepSeek: [deepseek.com](https://platform.deepseek.com)\n' + '4. **`Claude Code Usage: Get AI Usage Advice`** を再実行', costComposition: 'コスト構成', date: '日付', @@ -2657,7 +2679,11 @@ const translations: Record = { settings: '설정', settingsTab: '설정', settingsIntro: - '설정이 이제 여기로 모였습니다. 언어, 데이터 디렉터리, API 키만 VS Code 설정에 남습니다(동기화를 위해). 변경은 즉시 적용됩니다.', + '설정이 이제 여기로 모였습니다. 언어와 데이터 디렉터리는 VS Code 설정을 사용하고, API 키는 동기화되지 않는 VS Code SecretStorage에 저장됩니다. 변경은 즉시 적용됩니다.', + secretMigrationFailed: + '저장된 조언 API 키를 암호화된 SecretStorage로 옮기지 못했습니다. 이전 값을 삭제하기 전에 확장이 중지되었습니다. 기존 설정을 수정하고 창을 다시 로드하세요.', + secretMigrationWorkspace: + '작업 영역별 API 키는 하나의 전역 SecretStorage 항목으로 안전하게 이전할 수 없습니다. 키를 복사한 뒤 작업 영역 설정에서 제거하고, 다시 로드한 후 대시보드 설정 → 조언에 입력하세요.', settingsResetAll: '모두 기본값으로', settingsGroupGeneral: '일반', settingsGroupProviders: '공급자', @@ -2907,10 +2933,10 @@ const translations: Record = { '> 기반하지 **않으며**, 이 내용을 생성하기 위해 API에 데이터가\n' + '> 전송된 적이 없습니다.\n\n' + '### 실제 사용량 기반의 맞춤형 조언을 받으려면:\n\n' + - '1. 설정 열기 (`Ctrl+,` / `Cmd+,`)\n' + - '2. **`claudeCodeUsage.advice.apiKey`** 검색\n' + - '3. OpenAI 호환 API 키 붙여넣기 — DeepSeek 즉시 사용 가능\n' + - ' ([deepseek.com](https://platform.deepseek.com))\n' + + '1. **`Claude Code Usage: Show Usage Details`** 실행\n' + + '2. 대시보드의 **설정 → 조언** 섹션 열기\n' + + '3. OpenAI 호환 API 키 붙여넣기 — VS Code SecretStorage에만 저장됩니다\n' + + ' DeepSeek: [deepseek.com](https://platform.deepseek.com)\n' + '4. **`Claude Code Usage: Get AI Usage Advice`** 다시 실행', costComposition: '비용 구성', date: '날짜', @@ -2953,7 +2979,11 @@ const translations: Record = { settings: 'Configurações', settingsTab: 'Configurações', settingsIntro: - 'As configurações agora ficam aqui. Apenas idioma, diretório de dados e chave de API permanecem nas Configurações do VS Code (para sincronizar). As alterações são aplicadas imediatamente.', + 'As configurações agora ficam aqui. Idioma e diretórios de dados continuam nas Configurações do VS Code; chaves de API ficam no VS Code SecretStorage e nunca são sincronizadas. As alterações são aplicadas imediatamente.', + secretMigrationFailed: + 'Não foi possível mover a chave de API de conselho salva para o SecretStorage criptografado. A extensão parou antes de excluir o valor antigo. Corrija a configuração legada e recarregue a janela.', + secretMigrationWorkspace: + 'Uma chave de API específica do espaço de trabalho não pode ser migrada com segurança para uma única entrada global do SecretStorage. Copie-a, remova-a das configurações do espaço de trabalho, recarregue e informe-a em Configurações do painel → Conselho.', settingsResetAll: 'Restaurar tudo para os padrões', settingsGroupGeneral: 'Geral', settingsGroupProviders: 'Provedores', @@ -3202,10 +3232,10 @@ const translations: Record = { '> que o recurso produz. Ele **não** é baseado nos seus dados reais de uso\n' + '> do Claude Code — nada foi enviado a nenhuma API para gerar isso.\n\n' + '### Para obter um conselho real e personalizado com base no SEU uso:\n\n' + - '1. Abra as Configurações (`Ctrl+,` / `Cmd+,`)\n' + - '2. Pesquise por **`claudeCodeUsage.advice.apiKey`**\n' + - '3. Cole uma chave de API compatível com OpenAI — DeepSeek funciona direto\n' + - ' ([deepseek.com](https://platform.deepseek.com))\n' + + '1. Execute **`Claude Code Usage: Show Usage Details`**\n' + + '2. Abra **Configurações → Conselho** no painel\n' + + '3. Cole uma chave compatível com OpenAI — ela fica apenas no VS Code SecretStorage\n' + + ' DeepSeek: [deepseek.com](https://platform.deepseek.com)\n' + '4. Execute novamente **`Claude Code Usage: Get AI Usage Advice`**', costComposition: 'Composição de custos', date: 'Data', @@ -3248,7 +3278,11 @@ const translations: Record = { settings: 'Pengaturan', settingsTab: 'Pengaturan', settingsIntro: - 'Pengaturan sekarang ada di sini. Hanya bahasa, direktori data, dan API key yang tetap berada di Pengaturan VS Code (agar dapat disinkronkan). Perubahan langsung diterapkan.', + 'Pengaturan sekarang ada di sini. Bahasa dan direktori data tetap memakai Pengaturan VS Code; API key disimpan di VS Code SecretStorage dan tidak pernah disinkronkan. Perubahan langsung diterapkan.', + secretMigrationFailed: + 'API key saran yang tersimpan tidak dapat dipindahkan ke SecretStorage terenkripsi. Ekstensi dihentikan sebelum nilai lama dihapus. Perbaiki pengaturan lama lalu muat ulang jendela.', + secretMigrationWorkspace: + 'API key khusus workspace tidak dapat dimigrasikan dengan aman ke satu entri SecretStorage global. Salin lalu hapus dari pengaturan workspace, muat ulang, kemudian masukkan di Pengaturan Dashboard → Saran.', settingsResetAll: 'Kembalikan semua ke default', settingsGroupGeneral: 'Umum', settingsGroupProviders: 'Penyedia', @@ -3498,10 +3532,10 @@ const translations: Record = { '> yang dihasilkan fitur ini. Ini **bukan** berdasarkan data penggunaan\n' + '> Claude Code Anda yang sebenarnya — tidak ada yang dikirim ke API mana pun untuk menghasilkan ini.\n\n' + '### Untuk mendapatkan saran nyata dan personal berdasarkan penggunaan ANDA:\n\n' + - '1. Buka Pengaturan (`Ctrl+,` / `Cmd+,`)\n' + - '2. Cari **`claudeCodeUsage.advice.apiKey`**\n' + - '3. Tempel API key yang kompatibel dengan OpenAI — DeepSeek langsung berfungsi\n' + - ' ([deepseek.com](https://platform.deepseek.com))\n' + + '1. Jalankan **`Claude Code Usage: Show Usage Details`**\n' + + '2. Buka bagian **Pengaturan → Saran** di dashboard\n' + + '3. Tempel API key yang kompatibel dengan OpenAI — hanya disimpan di VS Code SecretStorage\n' + + ' DeepSeek: [deepseek.com](https://platform.deepseek.com)\n' + '4. Jalankan ulang **`Claude Code Usage: Get AI Usage Advice`**', costComposition: 'Komposisi Biaya', date: 'Tanggal', @@ -3565,7 +3599,7 @@ const SETTINGS_I18N: Partial = new Map(SETTINGS.map((d) => [d.key, d])) /** A snapshot of one setting for the webview panel: definition + current value. */ export interface SettingView extends SettingDef { value: boolean | number | string; + /** Secret values never cross into the webview; it receives only this bit. */ + configured?: boolean; } /** - * Read/write layer over the two stores. Core settings live in VS Code config; - * the rest live in globalState. Both are addressed by the same dotted key. + * Read/write layer over VS Code configuration, globalState, and SecretStorage. + * All entries are addressed by the same dotted catalog key. */ export class SettingsStore { + private readonly secretValues = new Map(); + constructor(private context: vscode.ExtensionContext) {} + private secretKey(key: string): string { + return SECRET_PREFIX + key; + } + + /** + * Load secrets before the extension starts. Legacy plaintext values migrate + * only after SecretStorage succeeds, then are removed from globalState and + * the old global configuration entry. This method is idempotent. + */ + async initializeSecrets(): Promise { + const secrets = this.context.secrets; + if (!secrets) { + throw new SettingsSecretMigrationError('secret-storage-unavailable'); + } + try { + const rootConfiguration = this.cfg(); + for (const def of SETTINGS) { + if (def.storage !== 'secret') { + continue; + } + const storageKey = this.secretKey(def.key); + let value = await secrets.get(storageKey); + const legacyStateKey = STATE_PREFIX + def.key; + const legacyState = this.context.globalState.get(legacyStateKey); + const inspected = rootConfiguration.inspect(def.key); + const folderScopes = (vscode.workspace.workspaceFolders ?? []).map((folder) => { + const configuration = vscode.workspace.getConfiguration('claudeCodeUsage', folder.uri); + return { + configuration, + value: configuration.inspect(def.key)?.workspaceFolderValue, + }; + }); + const legacyScopes = [ + legacyState, + inspected?.globalValue, + inspected?.workspaceValue, + ...folderScopes.map((scope) => scope.value), + ]; + const strings = legacyScopes + .filter((candidate): candidate is string => typeof candidate === 'string') + .map((candidate) => candidate.trim()) + .filter((candidate) => candidate !== ''); + + if (!value) { + const machineCandidates = [legacyState, inspected?.globalValue] + .filter((candidate): candidate is string => typeof candidate === 'string') + .map((candidate) => candidate.trim()) + .filter((candidate) => candidate !== ''); + const distinctMachineCandidates = [...new Set(machineCandidates)]; + if (distinctMachineCandidates.length > 1) { + throw new SettingsSecretMigrationError('legacy-secret-conflict'); + } + const machineCandidate = distinctMachineCandidates[0]; + const workspaceCandidates = [ + inspected?.workspaceValue, + ...folderScopes.map((scope) => scope.value), + ].filter((candidate): candidate is string => typeof candidate === 'string'); + + if (!machineCandidate && workspaceCandidates.some((candidate) => candidate.trim() !== '')) { + // A global SecretStorage slot cannot safely preserve a workspace- + // specific key. Leave the plaintext untouched and ask the user to + // migrate it explicitly instead of selecting one folder silently. + throw new SettingsSecretMigrationError('workspace-secret-requires-manual-migration'); + } + if ( + machineCandidate && + workspaceCandidates.some((candidate) => candidate.trim() !== machineCandidate) + ) { + throw new SettingsSecretMigrationError('legacy-secret-conflict'); + } + if (machineCandidate) { + // A second Extension Host may have completed the same machine-wide + // migration after our first read. Re-read before writing; all safe + // automatic candidates are already required to be identical. + const concurrentValue = await secrets.get(storageKey); + if (concurrentValue) { + value = concurrentValue; + } else { + await secrets.store(storageKey, machineCandidate); + value = machineCandidate; + } + } + } + + // Delete legacy entries only when a secure value is present, or when + // every explicit legacy entry is empty. This prevents a partial + // migration from destroying the only recoverable credential. + if (value || strings.length === 0) { + if (legacyState !== undefined) { + await this.context.globalState.update(legacyStateKey, undefined); + } + for (const scope of folderScopes) { + if (scope.value !== undefined) { + await scope.configuration.update( + def.key, + undefined, + vscode.ConfigurationTarget.WorkspaceFolder, + ); + } + } + if (inspected?.workspaceValue !== undefined) { + await rootConfiguration.update(def.key, undefined, vscode.ConfigurationTarget.Workspace); + } + if (inspected?.globalValue !== undefined) { + await rootConfiguration.update(def.key, undefined, vscode.ConfigurationTarget.Global); + } + } + if (value) { + this.secretValues.set(def.key, value); + } + } + } catch (error) { + if (error instanceof SettingsSecretMigrationError) { + throw error; + } + throw new SettingsSecretMigrationError('secret-storage-failed'); + } + } + private cfg(): vscode.WorkspaceConfiguration { return vscode.workspace.getConfiguration('claudeCodeUsage'); } @@ -674,6 +814,9 @@ export class SettingsStore { if (def.storage === 'config') { return this.cfg().get(def.key, def.default as unknown as T); } + if (def.storage === 'secret') { + return (this.secretValues.get(def.key) ?? def.default) as unknown as T; + } return this.context.globalState.get(STATE_PREFIX + def.key, def.default as unknown as T); } @@ -686,6 +829,15 @@ export class SettingsStore { const coerced = this.coerce(def, value); if (def.storage === 'config') { await this.cfg().update(def.key, coerced, vscode.ConfigurationTarget.Global); + } else if (def.storage === 'secret') { + const secret = String(coerced).trim(); + if (secret === '') { + await this.context.secrets.delete(this.secretKey(def.key)); + this.secretValues.delete(def.key); + } else { + await this.context.secrets.store(this.secretKey(def.key), secret); + this.secretValues.set(def.key, secret); + } } else { await this.context.globalState.update(STATE_PREFIX + def.key, coerced); } @@ -699,6 +851,9 @@ export class SettingsStore { } if (def.storage === 'config') { await this.cfg().update(def.key, undefined, vscode.ConfigurationTarget.Global); + } else if (def.storage === 'secret') { + await this.context.secrets.delete(this.secretKey(def.key)); + this.secretValues.delete(def.key); } else { await this.context.globalState.update(STATE_PREFIX + def.key, undefined); } @@ -731,7 +886,13 @@ export class SettingsStore { /** Catalog + current values, for rendering the dashboard settings panel. */ snapshot(): SettingView[] { - return SETTINGS.map((def) => ({ ...def, value: this.get(def.key) })); + return SETTINGS.map((def) => def.storage === 'secret' + ? { + ...def, + value: '', + configured: this.secretValues.has(def.key), + } + : { ...def, value: this.get(def.key) }); } /** diff --git a/src/test/quotaI18n.test.ts b/src/test/quotaI18n.test.ts index 01d74828..61fb56ea 100644 --- a/src/test/quotaI18n.test.ts +++ b/src/test/quotaI18n.test.ts @@ -126,3 +126,30 @@ test('every settings-catalog key has a label in every non-English locale', () => } assert.deepEqual(missing, [], `untranslated setting labels: ${missing.join(', ')}`); }); + +test('BYOK privacy copy points every locale to SecretStorage, not plaintext settings', () => { + for (const lang of LOCALES) { + withLanguage(lang, () => { + const text = lang === 'en' + ? (() => { + const src = fs.readFileSync(path.join(__dirname, '..', '..', 'src', 'settings.ts'), 'utf8'); + const match = src.match(/key: 'advice\.apiKey',[\s\S]*?help: '([^']+)'/); + return { help: match?.[1] ?? '' }; + })() + : I18n.settingText('advice.apiKey'); + assert.match(text.help ?? '', /SecretStorage/, `${lang}: BYOK help must name SecretStorage`); + assert.doesNotMatch( + I18n.t.popup.adviceDemoNotice, + /claudeCodeUsage\.advice\.apiKey/, + `${lang}: demo must not direct users to the retired plaintext setting`, + ); + assert.ok(I18n.t.popup.secretMigrationFailed.length > 20, `${lang}: needs migration failure copy`); + assert.ok(I18n.t.popup.secretMigrationWorkspace.length > 20, `${lang}: needs workspace migration copy`); + assert.doesNotMatch( + I18n.t.popup.secretMigrationFailed + I18n.t.popup.secretMigrationWorkspace, + /\$\{|apiKey\}|\.message/, + `${lang}: failure copy must be fixed and never interpolate a provider error`, + ); + }); + } +}); diff --git a/src/test/settingsPrivacy.test.ts b/src/test/settingsPrivacy.test.ts new file mode 100644 index 00000000..58a2e754 --- /dev/null +++ b/src/test/settingsPrivacy.test.ts @@ -0,0 +1,303 @@ +import { test } from 'node:test'; +import * as assert from 'node:assert/strict'; +import * as fs from 'node:fs'; +import * as path from 'node:path'; + +type SettingsModule = typeof import('../settings'); + +interface FakeConfiguration { + values: { + global?: unknown; + workspace?: unknown; + folder?: unknown; + }; + updates: Array<{ key: string; value: unknown; target: number }>; + get(key: string, fallback: T): T; + inspect(key: string): { + globalValue?: T; + workspaceValue?: T; + workspaceFolderValue?: T; + }; + update(key: string, value: unknown, target: number): Promise; +} + +function fakeConfiguration(initial: FakeConfiguration['values'] = {}): FakeConfiguration { + const config: FakeConfiguration = { + values: { ...initial }, + updates: [], + get(_key: string, fallback: T): T { + return (config.values.folder ?? config.values.workspace ?? config.values.global ?? fallback) as T; + }, + inspect(_key: string) { + return { + globalValue: config.values.global as T | undefined, + workspaceValue: config.values.workspace as T | undefined, + workspaceFolderValue: config.values.folder as T | undefined, + }; + }, + async update(key: string, value: unknown, target: number): Promise { + config.updates.push({ key, value, target }); + const slot = target === 3 ? 'folder' : target === 2 ? 'workspace' : 'global'; + config.values[slot] = value; + }, + }; + return config; +} + +let activeConfiguration = fakeConfiguration(); +let activeWorkspaceFolders: Array<{ uri: string }> = []; +let activeFolderConfigurations = new Map(); + +function loadSettingsModule(): SettingsModule { + const moduleLoader = require('node:module') as { + _load: (request: string, parent: unknown, isMain: boolean) => unknown; + }; + const originalLoad = moduleLoader._load; + moduleLoader._load = function (request, parent, isMain): unknown { + if (request === 'vscode') { + return { + ConfigurationTarget: { Global: 1, Workspace: 2, WorkspaceFolder: 3 }, + workspace: { + get workspaceFolders() { return activeWorkspaceFolders; }, + getConfiguration: (_section: string, resource?: string) => + resource ? activeFolderConfigurations.get(resource) ?? activeConfiguration : activeConfiguration, + }, + }; + } + return Reflect.apply(originalLoad, this, [request, parent, isMain]); + }; + try { + return require('../settings') as SettingsModule; + } finally { + moduleLoader._load = originalLoad; + } +} + +const { SettingsStore } = loadSettingsModule(); + +function fakeContext(options: { + state?: Map; + secrets?: Map; + failSecretStore?: boolean; +} = {}): any { + const state = options.state ?? new Map(); + const secrets = options.secrets ?? new Map(); + const secretStores: Array<{ key: string; value: string }> = []; + const secretDeletes: string[] = []; + return { + globalState: { + get: (key: string, fallback?: T): T | undefined => + (state.has(key) ? state.get(key) : fallback) as T | undefined, + update: async (key: string, value: unknown): Promise => { + if (value === undefined) state.delete(key); + else state.set(key, value); + }, + }, + secrets: { + get: async (key: string): Promise => secrets.get(key), + store: async (key: string, value: string): Promise => { + if (options.failSecretStore) throw new Error('synthetic secret-store failure'); + secretStores.push({ key, value }); + secrets.set(key, value); + }, + delete: async (key: string): Promise => { + secretDeletes.push(key); + secrets.delete(key); + }, + }, + _state: state, + _secrets: secrets, + _secretStores: secretStores, + _secretDeletes: secretDeletes, + }; +} + +test('legacy plaintext BYOK migrates to SecretStorage and never enters a settings snapshot', async () => { + const canary = 'sk-v2.3.1-privacy-canary'; + activeConfiguration = fakeConfiguration({ global: canary, workspace: canary }); + const folderConfiguration = fakeConfiguration({ folder: canary }); + activeWorkspaceFolders = [{ uri: 'folder-a' }]; + activeFolderConfigurations = new Map([['folder-a', folderConfiguration]]); + const context = fakeContext({ + state: new Map([['ccu.setting.advice.apiKey', canary]]), + }); + const store = new SettingsStore(context); + + await store.initializeSecrets(); + + assert.equal(context._secrets.get('claudeCodeUsage.secret.advice.apiKey'), canary); + assert.equal(context._state.has('ccu.setting.advice.apiKey'), false); + assert.deepEqual(activeConfiguration.values, { + global: undefined, + workspace: undefined, + }); + assert.equal(folderConfiguration.values.folder, undefined); + assert.equal(store.get('advice.apiKey'), canary, 'the extension host can still use the key'); + const apiKeyView = store.snapshot().find((entry) => entry.key === 'advice.apiKey'); + assert.equal(apiKeyView?.configured, true); + assert.equal(apiKeyView?.value, ''); + assert.doesNotMatch(JSON.stringify(store.snapshot()), /v2\.3\.1-privacy-canary/); +}); + +test('conflicting machine and workspace secrets fail closed without deleting plaintext', async () => { + activeConfiguration = fakeConfiguration({ + global: 'sk-global-canary', + workspace: 'sk-workspace-canary', + }); + activeWorkspaceFolders = []; + activeFolderConfigurations = new Map(); + const context = fakeContext(); + const store = new SettingsStore(context); + + await assert.rejects( + () => store.initializeSecrets(), + /settings-secret-migration:legacy-secret-conflict/, + ); + + assert.equal(context._secrets.size, 0); + assert.equal(activeConfiguration.values.global, 'sk-global-canary'); + assert.equal(activeConfiguration.values.workspace, 'sk-workspace-canary'); + assert.equal(activeConfiguration.updates.length, 0); +}); + +test('workspace-only secrets require explicit migration and every open folder is inspected', async () => { + activeConfiguration = fakeConfiguration(); + const first = fakeConfiguration({ folder: 'sk-folder-one' }); + const second = fakeConfiguration({ folder: 'sk-folder-two' }); + activeWorkspaceFolders = [{ uri: 'folder-a' }, { uri: 'folder-b' }]; + activeFolderConfigurations = new Map([ + ['folder-a', first], + ['folder-b', second], + ]); + const context = fakeContext(); + const store = new SettingsStore(context); + + await assert.rejects( + () => store.initializeSecrets(), + /settings-secret-migration:workspace-secret-requires-manual-migration/, + ); + assert.equal(first.values.folder, 'sk-folder-one'); + assert.equal(second.values.folder, 'sk-folder-two'); + assert.equal(context._secrets.size, 0); +}); + +test('failed SecretStorage migration leaves legacy plaintext in place for recovery', async () => { + const canary = 'sk-recovery-canary'; + activeConfiguration = fakeConfiguration({ global: canary }); + activeWorkspaceFolders = []; + activeFolderConfigurations = new Map(); + const context = fakeContext({ + state: new Map([['ccu.setting.advice.apiKey', canary]]), + failSecretStore: true, + }); + const store = new SettingsStore(context); + + await assert.rejects( + () => store.initializeSecrets(), + (error: unknown) => { + assert.match(String(error), /settings-secret-migration:secret-storage-failed/); + assert.doesNotMatch(String(error), /recovery-canary|synthetic secret-store failure/); + return true; + }, + ); + assert.equal(context._state.get('ccu.setting.advice.apiKey'), canary); + assert.equal(activeConfiguration.values.global, canary); +}); + +test('BYOK set and reset use only SecretStorage', async () => { + activeConfiguration = fakeConfiguration(); + activeWorkspaceFolders = []; + activeFolderConfigurations = new Map(); + const context = fakeContext(); + const store = new SettingsStore(context); + + await store.set('advice.apiKey', ' sk-runtime-canary '); + assert.equal(store.get('advice.apiKey'), 'sk-runtime-canary'); + assert.equal(context._secrets.get('claudeCodeUsage.secret.advice.apiKey'), 'sk-runtime-canary'); + assert.equal(context._state.size, 0); + assert.equal(activeConfiguration.updates.length, 0); + + await store.reset('advice.apiKey'); + assert.equal(store.get('advice.apiKey'), ''); + assert.equal(context._secrets.size, 0); +}); + +test('repeat activation is idempotent and secret metadata is independent of key length', async () => { + activeConfiguration = fakeConfiguration(); + activeWorkspaceFolders = []; + activeFolderConfigurations = new Map(); + const context = fakeContext({ + secrets: new Map([['claudeCodeUsage.secret.advice.apiKey', 'sk-short']]), + }); + const store = new SettingsStore(context); + + await store.initializeSecrets(); + const shortView = store.snapshot().find((entry) => entry.key === 'advice.apiKey'); + await store.initializeSecrets(); + await store.set('advice.apiKey', 'sk-a-much-longer-secret-value'); + const longView = store.snapshot().find((entry) => entry.key === 'advice.apiKey'); + + assert.deepEqual(shortView, longView); + assert.deepEqual(shortView && { value: shortView.value, configured: shortView.configured }, { + value: '', + configured: true, + }); + assert.equal(context._secretStores.length, 1, 'only the explicit user edit writes SecretStorage'); + assert.equal(activeConfiguration.updates.length, 0); +}); + +test('the extension manifest no longer contributes a plaintext API-key setting', () => { + const manifest = JSON.parse( + fs.readFileSync(path.join(__dirname, '..', '..', 'package.json'), 'utf8'), + ) as any; + assert.equal( + Object.prototype.hasOwnProperty.call( + manifest.contributes.configuration.properties, + 'claudeCodeUsage.advice.apiKey', + ), + false, + ); +}); + +test('activation reports a fixed localized migration failure without echoing provider errors', () => { + const source = fs.readFileSync(path.join(__dirname, '..', '..', 'src', 'extension.ts'), 'utf8'); + assert.match(source, /showErrorMessage\([\s\S]*secretMigrationWorkspace[\s\S]*secretMigrationFailed/); + assert.doesNotMatch(source, /secretMigrationFailed[^;]*\.message/); +}); + +test('the tracked bilingual data contract covers quota, retention, clearing, and remote boundaries', () => { + const root = path.join(__dirname, '..', '..'); + const documents = [ + 'docs/superpowers/specs/2026-09-02-v2.3.1-local-data-contract.md', + 'docs/superpowers/specs/2026-09-02-v2.3.1-local-data-contract.zh-CN.md', + ].map((relative) => fs.readFileSync(path.join(root, relative), 'utf8')); + + for (const document of documents) { + for (const required of [ + 'schemaVersion', + 'provider', + 'accountFingerprint', + 'observedAt', + 'periodType', + 'usedFraction', + 'remainingFraction', + 'resetAt', + 'source', + 'windowId', + 'confidence', + 'SecretStorage', + '180', + '512', + 'ccu.heatmapRepo', + 'ccu.heatmapPath', + 'Codex processed', + ]) { + assert.match(document, new RegExp(required.replace('.', '\\.'))); + } + assert.match(document, /OAuth token/i); + assert.match(document, /cookie/i); + assert.match(document, /API key/i); + assert.match(document, /clear|清除/i); + assert.match(document, /remote|network|远程|网络/i); + } +}); diff --git a/src/webview.ts b/src/webview.ts index 8763de4c..6318f953 100644 --- a/src/webview.ts +++ b/src/webview.ts @@ -1942,9 +1942,9 @@ export class UsageWebviewProvider { /** * The ⚙ Settings tab: every setting, grouped, editable in place. Core - * settings (language / dataDirectory / advice.apiKey) still write to VS Code - * config; the rest write to the dashboard-managed store. Setting labels/help - * are English (technical); group headers + chrome are localised. + * settings (language / dataDirectory) still write to VS Code config; secrets + * go to SecretStorage and the rest use the dashboard-managed store. Setting + * labels/help are English (technical); group headers + chrome are localised. */ private renderSettingsPanel(provider: SettingProvider): string { const t = I18n.t.popup; @@ -2053,6 +2053,9 @@ export class UsageWebviewProvider { '" value="' + esc(String(it.value)) + '"' + + (it.secret + ? ' autocomplete="new-password" placeholder="' + (it.configured ? '••••••••' : '') + '"' + : '') + onCh('string') + '>'; } From c4b5d613520d34d11c46c4c5273ac4ad223b5f28 Mon Sep 17 00:00:00 2001 From: Carl <74130028+Carl723000@users.noreply.github.com> Date: Wed, 2 Sep 2026 00:31:41 +0800 Subject: [PATCH 21/72] docs: record G1 checkpoint and G2 audit Co-authored-by: OpenAI Codex <215057067+openai-codex[bot]@users.noreply.github.com> --- .../2026-09-01-v2.3.1-goal-ledger.en.md | 17 +++++++++++++++-- .../2026-09-01-v2.3.1-goal-ledger.zh-CN.md | 17 +++++++++++++++-- 2 files changed, 30 insertions(+), 4 deletions(-) diff --git a/docs/releases/2026-09-01-v2.3.1-goal-ledger.en.md b/docs/releases/2026-09-01-v2.3.1-goal-ledger.en.md index ae5fe7de..2fa2e100 100644 --- a/docs/releases/2026-09-01-v2.3.1-goal-ledger.en.md +++ b/docs/releases/2026-09-01-v2.3.1-goal-ledger.en.md @@ -15,8 +15,8 @@ This tracked ledger records G0–G6 stage status, reproducible acceptance eviden | Stage | Status | Local commit | Verification | Claude review | Residual risk | |---|---|---|---|---|---| | G0 baseline and fixtures | Complete | `f0af506` | Node 795 passed / 1 skipped; UI 90 passed; targeted 29 passed | N/A | No blocking risk | -| G1 data contract | Complete | This checkpoint | Focused 17 passed; Node 806 passed / 1 skipped; compile passed | Conditional GO, then revision GO | Blocked legacy workspace plaintext remains until explicit recovery; host SecretStorage backing varies; unified quota store is a G3 deliverable | -| G2 statistics/order/effort | Not started | — | — | N/A | — | +| G1 data contract | Complete | `86d95df` | Focused 17 passed; Node 806 passed / 1 skipped; compile passed | Conditional GO, then revision GO | Blocked legacy workspace plaintext remains until explicit recovery; host SecretStorage backing varies; unified quota store is a G3 deliverable | +| G2 statistics/order/effort | In progress | — | Entry audit complete; red fixtures pending | N/A | Six aggregation/dedup/effort hypotheses require fixture confirmation | | G3 quota observations/estimates | Not started | — | — | Pending | — | | G4A visual/interaction/accessibility | Not started | — | — | N/A | — | | G4B combined heatmap/share | Not started | — | — | Pending | — | @@ -77,6 +77,19 @@ This tracked ledger records G0–G6 stage status, reproducible acceptance eviden - [x] Fail closed and preserve recoverability for ambiguous or failed legacy migration. - [x] Pass the G1 test gate and Claude revision review. +## G2 entry audit + +The following implementation risks are test hypotheses, not accepted behavior. Each must first be reproduced by a minimal fixture before its implementation changes: + +1. Weekly pricing coverage appears to use the whole window rather than the observation-time prefix. +2. Exact-content deduplication does not include day/period distribution and can collapse records with equal totals but different dates. +3. A `codex-spark` limit may be classified account-wide when its scope is model-specific. +4. A delayed post-reset current observation may mask a valid cached pre-reset observation. +5. Cumulative-delta repair may permit `reasoning > output` or `cached input > input`. +6. Residual usage in a partial effort map may disappear instead of being attributed to `unknown`. + +Additional guards cover the complete 30-day definition in the configured timezone, cross-year chronological month order, alias/invalid reasoning-effort variants, and conservation across daily, monthly, model, effort, and total views. + ## Update rule Each stage must finish its automated verification before this ledger records its commit, evidence, Claude decision, and risk. The G6 acceptance matrix must be derived from these records; screenshots or copy cannot substitute for a failed gate. diff --git a/docs/releases/2026-09-01-v2.3.1-goal-ledger.zh-CN.md b/docs/releases/2026-09-01-v2.3.1-goal-ledger.zh-CN.md index 38773d90..f862dbc7 100644 --- a/docs/releases/2026-09-01-v2.3.1-goal-ledger.zh-CN.md +++ b/docs/releases/2026-09-01-v2.3.1-goal-ledger.zh-CN.md @@ -15,8 +15,8 @@ | 阶段 | 状态 | 本地 commit | 验证摘要 | Claude 审阅 | 剩余风险 | |---|---|---|---|---|---| | G0 基线与夹具 | 已完成 | `f0af506` | Node 795 passed / 1 skipped;UI 90 passed;定向 29 passed | 不适用 | 无阻塞风险 | -| G1 数据契约 | 已完成 | 本 checkpoint | 定向 17 passed;Node 806 passed / 1 skipped;compile passed | 首轮有条件 GO,修订复核 GO | 受阻 workspace 明文需用户显式恢复;SecretStorage 底层保护依赖宿主;统一额度存储属于 G3 | -| G2 统计/排序/effort | 未开始 | — | — | 不适用 | — | +| G1 数据契约 | 已完成 | `86d95df` | 定向 17 passed;Node 806 passed / 1 skipped;compile passed | 首轮有条件 GO,修订复核 GO | 受阻 workspace 明文需用户显式恢复;SecretStorage 底层保护依赖宿主;统一额度存储属于 G3 | +| G2 统计/排序/effort | 进行中 | — | 入口审计完成;待建立 red fixture | 不适用 | 六项聚合/去重/effort 假设需用 fixture 确认 | | G3 额度观测与估算 | 未开始 | — | — | 待审阅 | — | | G4A 视觉/交互/可访问性 | 未开始 | — | — | 不适用 | — | | G4B 综合热力图与分享 | 未开始 | — | — | 待审阅 | — | @@ -77,6 +77,19 @@ - [x] 对歧义或失败的旧数据迁移 fail closed 并保留可恢复性。 - [x] 通过 G1 测试门和 Claude 修订复核。 +## G2 入口审计 + +以下是必须先用最小 fixture 复现、再修改实现的风险假设,不代表已接受的现有行为: + +1. 每周定价覆盖率疑似使用完整窗口,而不是截至观测时刻的前缀。 +2. 精确内容去重签名未纳入日/周期分布,可能把总量相同但日期不同的记录折叠掉。 +3. `codex-spark` 限额可能被误判成账号级,而实际作用域是特定模型。 +4. 重置后延迟到达的当前观测可能掩盖有效的重置前缓存观测。 +5. 累计 delta 修复可能产生 `reasoning > output` 或 `cached input > input`。 +6. 部分 effort map 的剩余用量可能消失,而不是归入 `unknown`。 + +额外 guard 覆盖配置时区下完整的 30 天定义、跨年月份正序、reasoning-effort 别名/非法变体,以及日、月、模型、effort 和总计之间的守恒关系。 + ## 后续更新规则 每阶段必须先完成对应自动化验证,再更新本表中的 commit、测试证据、Claude 结论和风险。G6 的最终验收矩阵应由本台账中的证据生成;任何未通过项不得用截图或文案替代。 From 49f2041661e3700aae6f40605e462fecce712919 Mon Sep 17 00:00:00 2001 From: Carl <74130028+Carl723000@users.noreply.github.com> Date: Wed, 2 Sep 2026 00:50:01 +0800 Subject: [PATCH 22/72] fix: reconcile Codex usage and effort statistics Co-authored-by: OpenAI Codex <215057067+openai-codex[bot]@users.noreply.github.com> --- .../2026-09-01-v2.3.1-goal-ledger.en.md | 33 ++++- .../2026-09-01-v2.3.1-goal-ledger.zh-CN.md | 33 ++++- src/providers/codex/codexDedup.ts | 23 +++- src/providers/codex/codexParser.ts | 130 ++++++++++++++++-- src/providers/codex/codexProvider.ts | 6 +- src/providers/codex/codexQuotaHistory.ts | 8 +- src/providers/codex/codexUsage.ts | 48 ++++++- src/test/codexDedup.test.ts | 33 +++++ src/test/codexParser.test.ts | 80 +++++++++++ src/test/codexProvider.test.ts | 55 ++++++++ src/test/codexQuotaHistory.test.ts | 11 ++ src/test/codexUsage.test.ts | 87 ++++++++++++ src/test/weeklyValue.test.ts | 26 ++++ src/weeklyValue.ts | 9 +- 14 files changed, 552 insertions(+), 30 deletions(-) diff --git a/docs/releases/2026-09-01-v2.3.1-goal-ledger.en.md b/docs/releases/2026-09-01-v2.3.1-goal-ledger.en.md index 2fa2e100..e481ff54 100644 --- a/docs/releases/2026-09-01-v2.3.1-goal-ledger.en.md +++ b/docs/releases/2026-09-01-v2.3.1-goal-ledger.en.md @@ -16,7 +16,7 @@ This tracked ledger records G0–G6 stage status, reproducible acceptance eviden |---|---|---|---|---|---| | G0 baseline and fixtures | Complete | `f0af506` | Node 795 passed / 1 skipped; UI 90 passed; targeted 29 passed | N/A | No blocking risk | | G1 data contract | Complete | `86d95df` | Focused 17 passed; Node 806 passed / 1 skipped; compile passed | Conditional GO, then revision GO | Blocked legacy workspace plaintext remains until explicit recovery; host SecretStorage backing varies; unified quota store is a G3 deliverable | -| G2 statistics/order/effort | In progress | — | Entry audit complete; red fixtures pending | N/A | Six aggregation/dedup/effort hypotheses require fixture confirmation | +| G2 statistics/order/effort | Complete | This checkpoint | Node 815 passed / 1 skipped; UI 90 passed; all focused regressions passed | N/A | Unified observation/account attribution remains a G3 concern | | G3 quota observations/estimates | Not started | — | — | Pending | — | | G4A visual/interaction/accessibility | Not started | — | — | N/A | — | | G4B combined heatmap/share | Not started | — | — | Pending | — | @@ -90,6 +90,37 @@ The following implementation risks are test hypotheses, not accepted behavior. E Additional guards cover the complete 30-day definition in the configured timezone, cross-year chronological month order, alias/invalid reasoning-effort variants, and conservation across daily, monthly, model, effort, and total views. +## G2 correctness evidence + +### Root-cause fixes + +- Exact active/archive deduplication now includes the authoritative all-time `byDay` distribution, so equal totals on different dates cannot silently collapse. Rebuilding `period`/`today` projections intentionally remain outside identity, preserving refresh idempotence while their independent readability checks run. +- Weekly allowance inference gates on priced-token coverage up to the observation timestamp, not later rows in the same week. A later priced row can no longer make a weak prefix look sufficiently priced. +- An explicit model-specific Codex `limitId` takes precedence over a generic `limitName`, preventing Spark and similar allowances from entering the account-wide series. +- A delayed poll written at or after reset no longer shadows a useful cached pre-reset observation. +- Cumulative token deltas clamp cached input to input and reasoning to output, mark the repaired event partial, and continue to count processed tokens exactly once. +- Partial model/effort maps send positive residual components to `unknown`; zero-value buckets are removed from aggregate and thread projections. + +### Actual-schema effort audit + +- A local field-path-only audit emitted counts, never filenames, values, timestamps, or log text. It observed 51,912 `turn_context.payload.effort` fields, 33,403 `thread_settings_applied` variants, and 33 explicit completed-item variants during the bounded scan. +- Every string encountered by the allowlisted paths normalized to `none`, `minimal`, `low`, `medium`, `high`, `xhigh`, `max`, or `ultra`; one explicit null remained unattributed. The parser now accepts these explicit current/legacy paths and never infers effort from model names. + +### Verification + +- Focused red/green fixtures cover authoritative day distribution, sidecar migration stability, observation-prefix pricing coverage, explicit limit-ID precedence, delayed rollover polls, cumulative component repair, real effort schema variants, partial/zero unknown buckets, 30-day conservation, and cross-year month order. +- Final full Node suite: 816 tests, 815 passed, 0 failed, 1 filesystem-capability skip. +- Final Playwright suite: 90/90 passed, including accessibility, keyboard, light/dark, 360 px, chronology, and Compare regressions. + +### G2 exit gate + +- [x] “Last 30 days” remains today plus the previous 29 configured-zone civil dates and reconciles with its daily/model/effort rows. +- [x] `today ≤ last 30 days ≤ all time`; daily/monthly totals reconcile with all time. +- [x] Exact duplicates are stable across refresh while differing day distributions are retained as ambiguous. +- [x] Monthly chart and table data are chronological, including a year boundary. +- [x] Explicit reasoning-effort variants normalize; missing/invalid evidence becomes `unknown`; zero unknown is hidden. +- [x] Full Node and browser gates pass. + ## Update rule Each stage must finish its automated verification before this ledger records its commit, evidence, Claude decision, and risk. The G6 acceptance matrix must be derived from these records; screenshots or copy cannot substitute for a failed gate. diff --git a/docs/releases/2026-09-01-v2.3.1-goal-ledger.zh-CN.md b/docs/releases/2026-09-01-v2.3.1-goal-ledger.zh-CN.md index f862dbc7..f27e25f7 100644 --- a/docs/releases/2026-09-01-v2.3.1-goal-ledger.zh-CN.md +++ b/docs/releases/2026-09-01-v2.3.1-goal-ledger.zh-CN.md @@ -16,7 +16,7 @@ |---|---|---|---|---|---| | G0 基线与夹具 | 已完成 | `f0af506` | Node 795 passed / 1 skipped;UI 90 passed;定向 29 passed | 不适用 | 无阻塞风险 | | G1 数据契约 | 已完成 | `86d95df` | 定向 17 passed;Node 806 passed / 1 skipped;compile passed | 首轮有条件 GO,修订复核 GO | 受阻 workspace 明文需用户显式恢复;SecretStorage 底层保护依赖宿主;统一额度存储属于 G3 | -| G2 统计/排序/effort | 进行中 | — | 入口审计完成;待建立 red fixture | 不适用 | 六项聚合/去重/effort 假设需用 fixture 确认 | +| G2 统计/排序/effort | 已完成 | 本 checkpoint | Node 815 passed / 1 skipped;UI 90 passed;全部定向回归通过 | 不适用 | 统一观测与账号归属仍由 G3 处理 | | G3 额度观测与估算 | 未开始 | — | — | 待审阅 | — | | G4A 视觉/交互/可访问性 | 未开始 | — | — | 不适用 | — | | G4B 综合热力图与分享 | 未开始 | — | — | 待审阅 | — | @@ -90,6 +90,37 @@ 额外 guard 覆盖配置时区下完整的 30 天定义、跨年月份正序、reasoning-effort 别名/非法变体,以及日、月、模型、effort 和总计之间的守恒关系。 +## G2 正确性证据 + +### 根因修复 + +- active/archive 精确去重签名现包含权威 all-time `byDay` 分布,因此“总量相同、日期不同”不会被静默折叠。可重建的 `period`/`today` 投影不进入身份签名,避免 sidecar 迁移时破坏刷新幂等;它们继续由独立可读性检查保护。 +- 周额度推断改用截至观测时刻的已定价 Token 覆盖率,而不是同周后续记录;后到的已定价行不能再把证据不足的前缀变成可推断样本。 +- 显式模型级 Codex `limitId` 优先于通用 `limitName`,防止 Spark 等独立额度混入账号级序列。 +- reset 时刻或之后才写入的延迟 poll 不再遮蔽有效的重置前缓存观测。 +- 累计 Token delta 强制 cached input ≤ input、reasoning ≤ output;修复事件标为 partial,并保持 processed 只计一次。 +- 部分 model/effort map 的正残差进入 `unknown`;aggregate 与线程投影均删除零值 bucket。 + +### 真实 schema 的 effort 审计 + +- 本地只做字段路径计数,不输出文件名、字段值、时间或日志正文。有限扫描中观察到 51,912 个 `turn_context.payload.effort`、33,403 个 `thread_settings_applied` 变体和 33 个显式 completed-item 变体。 +- 白名单路径中的字符串都可规范为 `none`、`minimal`、`low`、`medium`、`high`、`xhigh`、`max` 或 `ultra`;另有 1 个显式 null 保持未归属。解析器现支持这些显式新旧路径,绝不按模型名称推断 effort。 + +### 验证 + +- 定向 red/green fixture 覆盖权威日期分布、sidecar 迁移稳定性、观测前缀定价覆盖、显式 limit ID 优先、延迟 rollover poll、累计分量修复、真实 effort schema、部分/零值 unknown、30 天守恒和跨年月份正序。 +- 最终完整 Node:816 项,815 passed,0 failed,1 项因文件系统能力 skipped。 +- 最终 Playwright:90/90 passed,覆盖可访问性、键盘、深浅主题、360 px、时间正序和 Compare 回归。 + +### G2 通过条件 + +- [x] “最近 30 天”保持为配置时区下今天及此前 29 个自然日,并与日/model/effort 行守恒。 +- [x] `今日 ≤ 最近 30 天 ≤ 全部时间`;日/月总计与 all-time 核对一致。 +- [x] 精确副本在刷新间稳定;日期分布不同的记录保留为 ambiguous。 +- [x] 月度图表与表格数据按时间正序,包含跨年边界。 +- [x] 显式 reasoning-effort 新旧变体可规范化;缺失/非法证据进入 `unknown`;零 unknown 隐藏。 +- [x] 完整 Node 与浏览器门均通过。 + ## 后续更新规则 每阶段必须先完成对应自动化验证,再更新本表中的 commit、测试证据、Claude 结论和风险。G6 的最终验收矩阵应由本台账中的证据生成;任何未通过项不得用截图或文案替代。 diff --git a/src/providers/codex/codexDedup.ts b/src/providers/codex/codexDedup.ts index 228ecec4..0fa60b0b 100644 --- a/src/providers/codex/codexDedup.ts +++ b/src/providers/codex/codexDedup.ts @@ -37,11 +37,28 @@ function bucketSignature( .map(([key, tokens]) => [key, tokenSignature(tokens)]); } +function structuralSignature(structural: { + patchCalls: number; + toolCalls: number; + postPatchToolCalls: number; + compactCount: number; + taskCompleteCount: number; +}): StableNumber[] { + return [ + stableNumber(structural.patchCalls), + stableNumber(structural.toolCalls), + stableNumber(structural.postPatchToolCalls), + stableNumber(structural.compactCount), + stableNumber(structural.taskCompleteCount), + ]; +} + function contributionSignature(contribution: CodexFileContribution): string { const { aggregate } = contribution; const { session, structural } = aggregate; return JSON.stringify([ tokenSignature(aggregate.total), + bucketSignature(aggregate.byDay), bucketSignature(aggregate.byModel), bucketSignature(aggregate.byEffort), session.parentSessionKey ?? null, @@ -49,11 +66,7 @@ function contributionSignature(contribution: CodexFileContribution): string { session.role, stableNumber(session.startedAt), stableNumber(session.endedAt), - stableNumber(structural.patchCalls), - stableNumber(structural.toolCalls), - stableNumber(structural.postPatchToolCalls), - stableNumber(structural.compactCount), - stableNumber(structural.taskCompleteCount), + structuralSignature(structural), ]); } diff --git a/src/providers/codex/codexParser.ts b/src/providers/codex/codexParser.ts index e67adb98..9f96f070 100644 --- a/src/providers/codex/codexParser.ts +++ b/src/providers/codex/codexParser.ts @@ -161,16 +161,43 @@ function componentDelta( current: CodexRawTokenCounts, previous: CodexRawTokenCounts, ): ProviderTokenCounts { + const inputTotal = Math.max(0, current.inputTokens - previous.inputTokens); + const outputTotal = Math.max(0, current.outputTokens - previous.outputTokens); return { - inputTotal: current.inputTokens - previous.inputTokens, - cachedInput: current.cachedInputTokens - previous.cachedInputTokens, - outputTotal: current.outputTokens - previous.outputTokens, - reasoningOutput: - current.reasoningOutputTokens - previous.reasoningOutputTokens, - sourceTotal: current.totalTokens - previous.totalTokens, + inputTotal, + cachedInput: Math.min( + inputTotal, + Math.max(0, current.cachedInputTokens - previous.cachedInputTokens), + ), + outputTotal, + reasoningOutput: Math.min( + outputTotal, + Math.max( + 0, + current.reasoningOutputTokens - previous.reasoningOutputTokens, + ), + ), + sourceTotal: inputTotal + outputTotal, }; } +function componentDeltaNeedsClamp( + current: CodexRawTokenCounts, + previous: CodexRawTokenCounts, +): boolean { + const inputTotal = Math.max(0, current.inputTokens - previous.inputTokens); + const cachedInput = Math.max( + 0, + current.cachedInputTokens - previous.cachedInputTokens, + ); + const outputTotal = Math.max(0, current.outputTokens - previous.outputTokens); + const reasoningOutput = Math.max( + 0, + current.reasoningOutputTokens - previous.reasoningOutputTokens, + ); + return cachedInput > inputTotal || reasoningOutput > outputTotal; +} + function containedHighWater( current: CodexRawTokenCounts, previous: CodexRawTokenCounts, @@ -315,9 +342,8 @@ function parseTurnContext( const model = Object.prototype.hasOwnProperty.call(entry.payload, 'model') ? sanitizeCodexMetadataLabel(entry.payload.model) : state.model; - const effort = Object.prototype.hasOwnProperty.call(entry.payload, 'effort') - ? sanitizeCodexMetadataLabel(entry.payload.effort) - : state.effort; + const effortEvidence = explicitReasoningEffort(entry.payload); + const effort = effortEvidence.present ? effortEvidence.value : state.effort; return { state: { ...state, @@ -328,6 +354,68 @@ function parseTurnContext( }; } +const CODEX_REASONING_EFFORTS = new Set([ + 'none', + 'minimal', + 'low', + 'medium', + 'high', + 'xhigh', + 'max', + 'ultra', +]); + +function normalizedReasoningEffort(value: unknown): string | undefined { + const label = sanitizeCodexMetadataLabel(value)?.toLowerCase(); + return label && CODEX_REASONING_EFFORTS.has(label) ? label : undefined; +} + +function explicitReasoningEffort(payload: JsonObject): { + present: boolean; + value?: string; +} { + const raw: unknown[] = []; + const collect = (object: JsonObject | undefined, key: string): void => { + if (object && Object.prototype.hasOwnProperty.call(object, key)) { + raw.push(object[key]); + } + }; + collect(payload, 'effort'); + const collaborationMode = isObject(payload.collaboration_mode) + ? payload.collaboration_mode + : undefined; + const collaborationSettings = collaborationMode && + isObject(collaborationMode.settings) + ? collaborationMode.settings + : undefined; + collect(collaborationSettings, 'reasoning_effort'); + const threadSettings = isObject(payload.thread_settings) + ? payload.thread_settings + : undefined; + collect(threadSettings, 'reasoning_effort'); + const threadCollaboration = threadSettings && + isObject(threadSettings.collaboration_mode) + ? threadSettings.collaboration_mode + : undefined; + const threadCollaborationSettings = threadCollaboration && + isObject(threadCollaboration.settings) + ? threadCollaboration.settings + : undefined; + collect(threadCollaborationSettings, 'reasoning_effort'); + const item = isObject(payload.item) ? payload.item : undefined; + collect(item, 'reasoning_effort'); + if (raw.length === 0) { + return { present: false }; + } + const normalized = raw.map(normalizedReasoningEffort); + const values = new Set( + normalized.filter((value): value is string => value !== undefined), + ); + return normalized.some((value) => value === undefined) || values.size !== 1 + ? { present: true } + : { present: true, value: [...values][0] }; +} + function nestedObject( object: JsonObject, ...keys: string[] @@ -524,8 +612,20 @@ function parseTokenCount( : last ? exactTokenCounts(last) : componentDelta(nextHighWater!, previous); + const componentClamped = Boolean( + !duplicate && !last && nextHighWater && + componentDeltaNeedsClamp(nextHighWater, previous), + ); + const flaggedState = componentClamped + ? withFlag( + regressed ? withFlag(state, 'counter-regression') : state, + 'component-delta-clamped', + ) + : regressed + ? withFlag(state, 'counter-regression') + : state; let nextState: CodexParserState = { - ...(regressed ? withFlag(state, 'counter-regression') : state), + ...flaggedState, ...(nextHighWater ? { highWater: nextHighWater } : {}), previousSnapshotSignature: signature, }; @@ -582,14 +682,18 @@ export function parseCodexLine( if (!isObject(entry.payload)) { return { state: withFlag(state, 'invalid-event-payload'), events: [] }; } + const effortEvidence = explicitReasoningEffort(entry.payload); + const eventState = effortEvidence.present + ? { ...state, effort: effortEvidence.value } + : state; if (stringField(entry.payload, 'type') === 'token_count') { - return parseTokenCount(entry, entry.payload, state, pseudonymize); + return parseTokenCount(entry, entry.payload, eventState, pseudonymize); } if (stringField(entry.payload, 'type') === 'task_started') { - return { state, events: [], lineageBoundary: true }; + return { state: eventState, events: [], lineageBoundary: true }; } return { - state, + state: eventState, events: [], structural: structuralFromEventMessage(entry, entry.payload), }; diff --git a/src/providers/codex/codexProvider.ts b/src/providers/codex/codexProvider.ts index af4899d9..32dcbb90 100644 --- a/src/providers/codex/codexProvider.ts +++ b/src/providers/codex/codexProvider.ts @@ -205,7 +205,11 @@ function codexWeeklyValueInputs( const cachedObservations = weeklyQuotaObservationsFromCodexHistory( cachedHistory, ).filter((cached) => !currentObservations.some((current) => - Math.abs(current.resetAt - cached.resetAt) <= CODEX_QUOTA_RESET_CLUSTER_MS, + Math.abs(current.resetAt - cached.resetAt) <= CODEX_QUOTA_RESET_CLUSTER_MS && + // A delayed line written at/after reset is ineligible for the closed + // window. It cannot shadow a useful pre-reset observation retained by the + // durable history. + current.observedAt < current.resetAt, )); const observations = [...currentObservations, ...cachedObservations]; diff --git a/src/providers/codex/codexQuotaHistory.ts b/src/providers/codex/codexQuotaHistory.ts index 08ba50d6..7568c7da 100644 --- a/src/providers/codex/codexQuotaHistory.ts +++ b/src/providers/codex/codexQuotaHistory.ts @@ -43,8 +43,12 @@ function isAccountWideCodexLimit(snapshot: ProviderLimitSnapshot): boolean { const id = (snapshot.limitId ?? '').trim().toLowerCase(); const name = (snapshot.limitName ?? '').trim().toLowerCase(); // Spark has its own rolling allowance and must never be folded into the - // account-wide Codex estimate. - return id === 'codex' || name === 'codex' || (!id && !name); + // account-wide Codex estimate. An explicit ID is authoritative: a generic + // display name must not broaden a model-specific allowance. + if (id) { + return id === 'codex'; + } + return name === 'codex' || !name; } function normalizeObservation(value: unknown): CodexQuotaObservation | undefined { diff --git a/src/providers/codex/codexUsage.ts b/src/providers/codex/codexUsage.ts index 08954ad3..a569a534 100644 --- a/src/providers/codex/codexUsage.ts +++ b/src/providers/codex/codexUsage.ts @@ -293,6 +293,37 @@ function addBuckets( addTokens(current, tokens); target.set(key, current); } + const attributed = zeroTokens(); + for (const [, tokens] of entries) { + addTokens(attributed, tokens); + } + const inputTotal = Math.max(0, fallback.inputTotal - attributed.inputTotal); + const outputTotal = Math.max(0, fallback.outputTotal - attributed.outputTotal); + const residual: ProviderTokenCounts = { + inputTotal, + cachedInput: Math.min( + inputTotal, + Math.max(0, (fallback.cachedInput ?? 0) - (attributed.cachedInput ?? 0)), + ), + outputTotal, + reasoningOutput: Math.min( + outputTotal, + Math.max( + 0, + (fallback.reasoningOutput ?? 0) - (attributed.reasoningOutput ?? 0), + ), + ), + }; + if ( + residual.inputTotal > 0 || + (residual.cachedInput ?? 0) > 0 || + residual.outputTotal > 0 || + (residual.reasoningOutput ?? 0) > 0 + ) { + const unknown = target.get('unknown') ?? zeroTokens(); + addTokens(unknown, residual); + target.set('unknown', unknown); + } } function bucketRows( @@ -300,6 +331,14 @@ function bucketRows( ): Array<{ key: string; totals: CodexMetricTotals }> { return [...buckets.entries()] .map(([key, tokens]) => ({ key, totals: metrics(tokens) })) + .filter(({ totals }) => + totals.processed > 0 || + totals.fresh > 0 || + totals.input > 0 || + totals.cachedInput > 0 || + totals.output > 0 || + totals.reasoning > 0, + ) .sort( (left, right) => right.totals.processed - left.totals.processed || @@ -515,10 +554,15 @@ function observedAt(file: CodexFileAggregate): number { function sortedBucketKeys( buckets: Record, ): string[] { - const rows = Object.entries(buckets); - if (rows.length === 0) { + const entries = Object.entries(buckets); + if (entries.length === 0) { return ['unknown']; } + const rows = entries.filter(([, tokens]) => + processedTokens(tokens) > 0 || + (tokens.cachedInput ?? 0) > 0 || + (tokens.reasoningOutput ?? 0) > 0, + ); return rows .sort( ([leftKey, left], [rightKey, right]) => diff --git a/src/test/codexDedup.test.ts b/src/test/codexDedup.test.ts index 90100e3e..6444d531 100644 --- a/src/test/codexDedup.test.ts +++ b/src/test/codexDedup.test.ts @@ -102,6 +102,39 @@ test('a token-total mismatch retains both copies as one ambiguous group', () => assert.equal(ambiguous.exactDuplicateFileKeys.size, 0); }); +test('equal totals with different authoritative day distributions are not exact duplicates', () => { + const active = contribution('active-key', 'sessions'); + const archive = contribution('archive-key', 'archive'); + active.aggregate.byDay = { + '2026-08-30': { ...active.aggregate.total }, + }; + archive.aggregate.byDay = { + '2026-08-31': { ...archive.aggregate.total }, + }; + + const result = classify(active, archive); + + assert.equal(result.ambiguousSessionGroups, 1); + assert.deepEqual([...result.canonicalFileKeys], ['active-key', 'archive-key']); + assert.equal(result.exactDuplicateFileKeys.size, 0); +}); + +test('a one-sided rebuilding period projection does not destabilize exact dedup', () => { + const active = contribution('active-key', 'sessions'); + const archive = contribution('archive-key', 'archive'); + active.aggregate.period = { + timeZone: 'Asia/Shanghai', + indexedThrough: 50, + days: {}, + }; + + const result = classify(active, archive); + + assert.deepEqual([...result.canonicalFileKeys], ['active-key']); + assert.deepEqual([...result.exactDuplicateFileKeys], ['archive-key']); + assert.equal(result.ambiguousSessionGroups, 0); +}); + test('a missing optional token field does not equal an explicit zero', () => { const active = contribution('active-key', 'sessions'); const archive = contribution('archive-key', 'archive'); diff --git a/src/test/codexParser.test.ts b/src/test/codexParser.test.ts index 4ae6d9bb..df7dde35 100644 --- a/src/test/codexParser.test.ts +++ b/src/test/codexParser.test.ts @@ -685,6 +685,86 @@ test('counter regression keeps exact last usage and marks it partial', () => { assert.equal(result.events[0].confidence, 'partial'); }); +test('cumulative deltas preserve cached-input and reasoning subset invariants', () => { + let state = createCodexParserState('root-file'); + state = parseCodexLine( + tokenSnapshot({ + second: 1, + total: { input: 100, cached: 0, output: 100, reasoning: 0 }, + }), + state, + ).state; + const result = parseCodexLine( + tokenSnapshot({ + second: 2, + total: { input: 101, cached: 100, output: 101, reasoning: 100 }, + }), + state, + ); + + assert.equal(result.events.length, 1); + assert.deepEqual(result.events[0].tokens, { + inputTotal: 1, + cachedInput: 1, + outputTotal: 1, + reasoningOutput: 1, + sourceTotal: 2, + }); + assert.ok(result.events[0].qualityFlags.includes('component-delta-clamped')); + assert.equal(result.events[0].confidence, 'partial'); +}); + +test('actual reasoning-effort schema variants normalize without model inference', () => { + let state = createCodexParserState('root-file'); + state = parseCodexLine( + JSON.stringify({ + timestamp: '2026-07-20T00:00:00.000Z', + type: 'turn_context', + payload: { + model: 'gpt-5.6-sol', + collaboration_mode: { settings: { reasoning_effort: 'ULTRA' } }, + }, + }), + state, + ).state; + assert.equal(state.effort, 'ultra'); + + state = parseCodexLine( + JSON.stringify({ + timestamp: '2026-07-20T00:00:01.000Z', + type: 'event_msg', + payload: { + type: 'thread_settings_applied', + thread_settings: { + reasoning_effort: 'low', + collaboration_mode: { settings: { reasoning_effort: 'low' } }, + }, + }, + }), + state, + ).state; + assert.equal(state.effort, 'low'); + + state = parseCodexLine( + JSON.stringify({ + timestamp: '2026-07-20T00:00:02.000Z', + type: 'event_msg', + payload: { + type: 'thread_settings_applied', + thread_settings: { reasoning_effort: 'turbo' }, + }, + }), + state, + ).state; + assert.equal(state.effort, undefined); + + const usage = parseCodexLine( + tokenLine({ inputTotal: 10, inputLast: 10 }), + state, + ); + assert.equal(usage.events[0].effort, undefined); +}); + test('session metadata is pseudonymized and auto-review stays distinct', () => { const pseudonyms: Record = { 'raw-session': 'session:001', diff --git a/src/test/codexProvider.test.ts b/src/test/codexProvider.test.ts index e22e5d86..10469ef6 100644 --- a/src/test/codexProvider.test.ts +++ b/src/test/codexProvider.test.ts @@ -520,6 +520,61 @@ test('provider snapshots promote exact period slices for scoped consumers', asyn } }); +test('a delayed post-reset current poll does not hide cached pre-reset evidence', async () => { + const root = await mkdtemp(path.join(os.tmpdir(), 'ccu-codex-provider-quota-rollover-')); + try { + await mkdir(path.join(root, 'sessions'), { recursive: true }); + const index = duplicateIndex(false); + const file = index.files['active-key']; + const resetAt = Date.parse('2026-08-31T03:00:00.000Z'); + file.limits = { + codex: { + provider: 'codex', + limitId: 'codex', + observedAt: resetAt + 30 * 60 * 1000, + source: 'local-log', + confidence: 'last-observed', + windows: [{ + label: 'secondary', + usedPercent: 1, + windowMinutes: 7 * 24 * 60, + resetsAt: resetAt, + }], + }, + }; + index.quotaHistory = [{ + provider: 'codex', + seriesKey: 'codex', + observedAt: resetAt - 30 * 60 * 1000, + resetAt, + usedPercent: 96, + }]; + const provider = new CodexProvider( + { + enabled: true, + codexHome: root, + indexPath: 'index', + salt: 'salt', + timeZone: 'UTC', + }, + () => new FakeClient([{ ...workerResult(index), failedFiles: 0 }]), + ); + + const refreshed = await provider.refresh(); + const observations = refreshed.snapshot.weeklyValueInputs?.observations ?? []; + + assert.deepEqual( + observations.map((item) => [item.observedAt, item.usedPercent]), + [ + [resetAt + 30 * 60 * 1000, 1], + [resetAt - 30 * 60 * 1000, 96], + ], + ); + } finally { + await rm(root, { recursive: true, force: true }); + } +}); + test('an invalid period sidecar cannot create a false weekly priced estimate', async () => { const root = await mkdtemp(path.join(os.tmpdir(), 'ccu-codex-provider-period-guard-')); try { diff --git a/src/test/codexQuotaHistory.test.ts b/src/test/codexQuotaHistory.test.ts index ca03218e..a8290f31 100644 --- a/src/test/codexQuotaHistory.test.ts +++ b/src/test/codexQuotaHistory.test.ts @@ -15,12 +15,14 @@ function limit( usedPercent: number, options: { limitId?: string; + limitName?: string; windowMinutes?: number; } = {}, ): ProviderLimitSnapshot { return { provider: 'codex', ...(options.limitId ? { limitId: options.limitId } : {}), + ...(options.limitName ? { limitName: options.limitName } : {}), observedAt, source: 'local-log', confidence: 'last-observed', @@ -59,6 +61,15 @@ test('only account-wide weekly Codex windows become neutral reset evidence', () ), [], ); + assert.deepEqual( + codexQuotaObservationsFromLimit( + limit(observedAt, resetAt, 42, { + limitId: 'codex-spark', + limitName: 'Codex', + }), + ), + [], + ); }); test('irregular reset boundaries survive polling compaction', () => { diff --git a/src/test/codexUsage.test.ts b/src/test/codexUsage.test.ts index f2076b64..224c929b 100644 --- a/src/test/codexUsage.test.ts +++ b/src/test/codexUsage.test.ts @@ -841,18 +841,59 @@ test('v2.3.1 acceptance fixture keeps non-negative ranges monotonic and reconcil (sum, row) => sum + row.total.processed, 0, ); + const last30DailyProcessed = view.last30DaysDaily.reduce( + (sum, row) => sum + row.total.processed, + 0, + ); + const last30ModelProcessed = view.last30Days.models.reduce( + (sum, row) => sum + row.totals.processed, + 0, + ); + const last30EffortProcessed = view.last30Days.efforts.reduce( + (sum, row) => sum + row.totals.processed, + 0, + ); assert.equal(view.today.total.processed >= 0, true); assert.equal(view.today.total.processed <= view.last30Days.total.processed, true); assert.equal(view.last30Days.total.processed <= view.allTime.total.processed, true); assert.equal(monthProcessed, view.allTime.total.processed); assert.equal(allDailyProcessed, view.allTime.total.processed); + assert.equal(last30DailyProcessed, view.last30Days.total.processed); + assert.equal(last30ModelProcessed, view.last30Days.total.processed); + assert.equal(last30EffortProcessed, view.last30Days.total.processed); assert.deepEqual( view.monthly.map((row) => row.period), [...view.monthly.map((row) => row.period)].sort(), ); }); +test('monthly rows remain chronological across a year boundary', () => { + const snapshot = snapshotFixture(); + snapshot.files = snapshot.files.slice(0, 2); + const days = ['2026-01-01', '2025-12-31']; + snapshot.files.forEach((file, index) => { + const day = days[index]; + file.byDay = { [day]: { ...file.total } }; + file.period = { + ...file.period!, + days: { + [day]: { + ...Object.values(file.period!.days)[0], + total: { ...file.total }, + }, + }, + }; + }); + + const view = buildCodexUsageView(snapshot, NOW); + + assert.deepEqual(view.monthly.map((row) => row.period), [ + '2025-12', + '2026-01', + ]); +}); + test('behavior exposes patch and tool call proxies without file or command claims', () => { const snapshot = snapshotFixture(); snapshot.files[0].structural = { @@ -986,6 +1027,52 @@ test('missing model and effort values are grouped as unknown', () => { assert.equal(view.last7Days.efforts.find((row) => row.key === 'unknown')?.totals.processed, 600); }); +test('partial effort buckets attribute every residual component to unknown', () => { + const snapshot = snapshotFixture(); + const file = snapshot.files[0]; + const partial = { + inputTotal: 100, + cachedInput: 60, + outputTotal: 20, + reasoningOutput: 10, + }; + file.byEffort = { high: partial }; + const periodDay = file.period?.days['2026-07-20']; + assert.ok(periodDay); + periodDay.byEffort = { high: partial }; + + const view = buildCodexUsageView(snapshot, NOW); + const unknown = view.last7Days.efforts.find((row) => row.key === 'unknown'); + const effortProcessed = view.last7Days.efforts.reduce( + (sum, row) => sum + row.totals.processed, + 0, + ); + + assert.equal(unknown?.totals.processed, 480); + assert.equal(unknown?.totals.cachedInput, 340); + assert.equal(unknown?.totals.reasoning, 50); + assert.equal(effortProcessed, view.last7Days.total.processed); +}); + +test('zero-value unknown effort buckets stay out of every rendered scope', () => { + const snapshot = snapshotFixture(); + for (const file of snapshot.files) { + file.byEffort.unknown = { inputTotal: 0, outputTotal: 0 }; + for (const day of Object.values(file.period?.days ?? {})) { + day.byEffort.unknown = { inputTotal: 0, outputTotal: 0 }; + } + } + + const view = buildCodexUsageView(snapshot, NOW); + + assert.equal(view.allTime.efforts.some((row) => row.key === 'unknown'), false); + assert.equal(view.last30Days.efforts.some((row) => row.key === 'unknown'), false); + assert.equal( + view.recentThreads.some((row) => row.efforts.includes('unknown')), + false, + ); +}); + test('incomplete session timestamps never invent a multi-year duration', () => { const snapshot = snapshotFixture(); snapshot.files[0].session.startedAt = undefined; diff --git a/src/test/weeklyValue.test.ts b/src/test/weeklyValue.test.ts index 6af3132e..a1eaf992 100644 --- a/src/test/weeklyValue.test.ts +++ b/src/test/weeklyValue.test.ts @@ -109,6 +109,32 @@ test('tiny utilization and poor price coverage do not manufacture a full allowan assert.equal(partialPricing[0].fullEquivalentUsd, null); }); +test('pricing coverage after the observation cannot qualify an underpriced prefix', () => { + const points = buildWeeklyValueTimeline('codex', { + observations: [observation({ usedPercent: 50 })], + usage: [ + { + timestamp: RESET - 2 * HOUR, + equivalentUsd: 1, + pricedTokens: 1, + totalTokens: 1_000, + }, + { + timestamp: RESET - 30 * 60 * 1000, + equivalentUsd: 99, + pricedTokens: 9_000, + totalTokens: 9_000, + }, + ], + }, { now: RESET + HOUR }); + + assert.ok(points[0].pricingCoverage > 0.8); + assert.equal(points[0].utilizationPercent, 50); + assert.equal(points[0].fullEquivalentUsd, null); + assert.equal(points[0].unusedEquivalentUsd, null); + assert.equal(points[0].confidence, 'usage-only'); +}); + test('source keys keep overlapping Codex account windows from sharing usage', () => { const points = buildWeeklyValueTrend({ observations: [ diff --git a/src/weeklyValue.ts b/src/weeklyValue.ts index 2c8cf913..b75ca499 100644 --- a/src/weeklyValue.ts +++ b/src/weeklyValue.ts @@ -737,9 +737,8 @@ export function buildWeeklyValueTimeline( ) ); const observed = totals(seriesRows.filter((row) => row.timestamp <= latest.observedAt)); - const seriesTotal = totals(seriesRows); - const seriesPricingCoverage = seriesTotal.totalTokens > 0 - ? Math.min(1, seriesTotal.pricedTokens / seriesTotal.totalTokens) + const observedPricingCoverage = observed.totalTokens > 0 + ? Math.min(1, observed.pricedTokens / observed.totalTokens) : 0; const observationGapMs = Math.max( 0, @@ -760,7 +759,7 @@ export function buildWeeklyValueTimeline( point.usageAvailable !== false && latest.usedPercent >= MIN_EXTRAPOLATION_PERCENT && observed.equivalentUsd > 0 && - seriesPricingCoverage >= MIN_PRICED_SHARE + observedPricingCoverage >= MIN_PRICED_SHARE ) { fullEquivalentUsd = observed.equivalentUsd / (latest.usedPercent / 100); if (!Number.isFinite(fullEquivalentUsd) || fullEquivalentUsd <= 0) { @@ -785,7 +784,7 @@ export function buildWeeklyValueTimeline( ? 'usage-only' : observationOverrun || approximateInference ? 'low' - : confidenceFor(point.current, observationGapMs, point.pricingCoverage), + : confidenceFor(point.current, observationGapMs, observedPricingCoverage), basis: 'quota-observation', }; }).slice(0, limit); From 39074315b0a02fb333dd5e2aed42a99e8e28733e Mon Sep 17 00:00:00 2001 From: Carl <74130028+Carl723000@users.noreply.github.com> Date: Wed, 2 Sep 2026 00:50:33 +0800 Subject: [PATCH 23/72] docs: close G2 checkpoint Co-authored-by: OpenAI Codex <215057067+openai-codex[bot]@users.noreply.github.com> --- docs/releases/2026-09-01-v2.3.1-goal-ledger.en.md | 4 ++-- docs/releases/2026-09-01-v2.3.1-goal-ledger.zh-CN.md | 4 ++-- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/docs/releases/2026-09-01-v2.3.1-goal-ledger.en.md b/docs/releases/2026-09-01-v2.3.1-goal-ledger.en.md index e481ff54..7ed0ff7a 100644 --- a/docs/releases/2026-09-01-v2.3.1-goal-ledger.en.md +++ b/docs/releases/2026-09-01-v2.3.1-goal-ledger.en.md @@ -16,8 +16,8 @@ This tracked ledger records G0–G6 stage status, reproducible acceptance eviden |---|---|---|---|---|---| | G0 baseline and fixtures | Complete | `f0af506` | Node 795 passed / 1 skipped; UI 90 passed; targeted 29 passed | N/A | No blocking risk | | G1 data contract | Complete | `86d95df` | Focused 17 passed; Node 806 passed / 1 skipped; compile passed | Conditional GO, then revision GO | Blocked legacy workspace plaintext remains until explicit recovery; host SecretStorage backing varies; unified quota store is a G3 deliverable | -| G2 statistics/order/effort | Complete | This checkpoint | Node 815 passed / 1 skipped; UI 90 passed; all focused regressions passed | N/A | Unified observation/account attribution remains a G3 concern | -| G3 quota observations/estimates | Not started | — | — | Pending | — | +| G2 statistics/order/effort | Complete | `49f2041` | Node 815 passed / 1 skipped; UI 90 passed; all focused regressions passed | N/A | Unified observation/account attribution remains a G3 concern | +| G3 quota observations/estimates | In progress | — | Architecture and fixture audit underway | Pending | — | | G4A visual/interaction/accessibility | Not started | — | — | N/A | — | | G4B combined heatmap/share | Not started | — | — | Pending | — | | G5 README/screenshots | Not started | — | — | Pending | — | diff --git a/docs/releases/2026-09-01-v2.3.1-goal-ledger.zh-CN.md b/docs/releases/2026-09-01-v2.3.1-goal-ledger.zh-CN.md index f27e25f7..6d81e9d0 100644 --- a/docs/releases/2026-09-01-v2.3.1-goal-ledger.zh-CN.md +++ b/docs/releases/2026-09-01-v2.3.1-goal-ledger.zh-CN.md @@ -16,8 +16,8 @@ |---|---|---|---|---|---| | G0 基线与夹具 | 已完成 | `f0af506` | Node 795 passed / 1 skipped;UI 90 passed;定向 29 passed | 不适用 | 无阻塞风险 | | G1 数据契约 | 已完成 | `86d95df` | 定向 17 passed;Node 806 passed / 1 skipped;compile passed | 首轮有条件 GO,修订复核 GO | 受阻 workspace 明文需用户显式恢复;SecretStorage 底层保护依赖宿主;统一额度存储属于 G3 | -| G2 统计/排序/effort | 已完成 | 本 checkpoint | Node 815 passed / 1 skipped;UI 90 passed;全部定向回归通过 | 不适用 | 统一观测与账号归属仍由 G3 处理 | -| G3 额度观测与估算 | 未开始 | — | — | 待审阅 | — | +| G2 统计/排序/effort | 已完成 | `49f2041` | Node 815 passed / 1 skipped;UI 90 passed;全部定向回归通过 | 不适用 | 统一观测与账号归属仍由 G3 处理 | +| G3 额度观测与估算 | 进行中 | — | 正在审计架构与验收夹具 | 待审阅 | — | | G4A 视觉/交互/可访问性 | 未开始 | — | — | 不适用 | — | | G4B 综合热力图与分享 | 未开始 | — | — | 待审阅 | — | | G5 README/截图 | 未开始 | — | — | 待审阅 | — | From a04ab04a5447cc3b338612628a2c2922a665a836 Mon Sep 17 00:00:00 2001 From: Carl <74130028+Carl723000@users.noreply.github.com> Date: Wed, 2 Sep 2026 01:41:34 +0800 Subject: [PATCH 24/72] feat: persist and estimate quota windows Co-authored-by: OpenAI Codex <215057067+openai-codex[bot]@users.noreply.github.com> --- .../2026-09-02-v2.3.1-local-data-contract.md | 8 +- ...-09-02-v2.3.1-local-data-contract.zh-CN.md | 8 +- src/claudeApiClient.ts | 28 +- src/extension.ts | 386 +++++-- src/i18n.ts | 16 +- src/quotaObservationStore.ts | 1006 +++++++++++++++++ src/test/claudeApiClient.test.ts | 43 +- src/test/extensionWindowActivity.test.ts | 227 +++- src/test/quotaObservationStore.test.ts | 368 ++++++ src/test/weeklyValue.test.ts | 137 ++- src/weeklyValue.ts | 220 +++- tests/ui/codex-visual.spec.mjs | 5 +- 12 files changed, 2310 insertions(+), 142 deletions(-) create mode 100644 src/quotaObservationStore.ts create mode 100644 src/test/quotaObservationStore.test.ts diff --git a/docs/superpowers/specs/2026-09-02-v2.3.1-local-data-contract.md b/docs/superpowers/specs/2026-09-02-v2.3.1-local-data-contract.md index 6386c8c5..f530829f 100644 --- a/docs/superpowers/specs/2026-09-02-v2.3.1-local-data-contract.md +++ b/docs/superpowers/specs/2026-09-02-v2.3.1-local-data-contract.md @@ -61,6 +61,7 @@ interface QuotaObservationV2 { resetAt: number | null; // Unix milliseconds from structured evidence when available source: 'claude-official-api' | 'codex-local-structured-event'; windowId: string; // HMAC(provider, fingerprint, period, reset identity/epoch) + providerWindowFingerprint: string | null; // HMAC of a known provider window ID; never the raw ID confidence: 'high' | 'medium' | 'low'; captureReason: 'refresh' | 'reset-at-change' | 'window-id-change' | 'usage-drop' | 'profile-change' | 'migration'; flags: Array<'approximate-boundary' | 'low-log-coverage' | 'low-price-coverage' | 'account-ambiguous' | 'clock-anomaly'>; @@ -73,6 +74,7 @@ interface QuotaObservationV2 { - A filesystem path alone identifies a local profile, not a verified account. Such observations use `profile-continuity` and cannot be merged after a detected profile/login change. - If Codex exposes no safe stable identity signal, generate a random unattributed epoch. Never label a Codex home hash as an account. Window changes, `resetAt` changes, implausible percentage rollback, or explicit profile changes close the epoch. - Only identical fingerprints and compatible attribution may be robustly aggregated. `unattributed` epochs never merge with one another, even when reset times coincide. +- Persistent account, provider-window, and window HMACs never enter Webview state or exports. Host projections replace them with invocation-local aliases before rendering. ### 3.2 Validation, deduplication, and atomicity @@ -85,7 +87,7 @@ interface QuotaObservationV2 { ### 3.3 Reset events and derived estimates -A reset/window boundary is detected from, in priority order: a stable provider window identifier change, `resetAt` identity change, or a significant used-fraction rollback that exceeds the configured noise tolerance. Multiple boundaries on the same day are retained as separate events. Event timestamps split usage when available; day-only aggregates are explicitly approximate. +A reset/window boundary is detected from, in priority order: a stable provider window identifier change, `resetAt` identity change, or a significant used-fraction rollback that exceeds the configured noise tolerance. Multiple boundaries on the same day are retained as separate events. Reset events are deterministic runtime projections of boundary observations, not a second persisted fact table. Event timestamps split usage when available; day-only aggregates are explicitly approximate. For each valid observation in one stable fingerprint/window: @@ -93,7 +95,7 @@ For each valid observation in one stable fingerprint/window: candidateFullEquivalent = pricedUsedEquivalentThroughObservedAt / usedFraction ``` -Candidates require `0 < usedFraction <= 1`. They are robustly aggregated with weights for price coverage, log/index coverage, time-boundary quality, account attribution, and observation recency. The displayed total is never below final confirmed used equivalent. Unused value is `max(full - confirmedUsed, 0)` and is shown only when account continuity and window boundaries are adequate. Multi-account or unattributed evidence may show used equivalent and an explanation, but no account-level unused claim. +Candidates require `0 < usedFraction <= 1`. They are robustly aggregated with weights for price coverage, log/index coverage, time-boundary quality, account attribution, and observation recency. The displayed total is never below final confirmed used equivalent. One coherent unattributed Codex window may therefore show a low-confidence total estimate. Unused value is `max(full - confirmedUsed, 0)` and is shown only when account continuity and window boundaries are adequate. Overlapping unattributed epochs, conflicting account evidence, or ambiguous multi-account usage falls back to used equivalent only; no account split or account-level unused claim is invented. Full/unused estimates are not persisted facts. They are recomputed when pricing, coverage, logs, or observation evidence changes. @@ -135,7 +137,7 @@ Uninstall behavior is disclosed rather than assumed: VS Code may retain extensio ## 7. Migration and release gates -- Migration order: load and validate current schema; migrate legacy quota/config values into a new temporary document; validate invariants; atomically install it; only then delete legacy plaintext/duplicate state. +- Migration order: load and validate current schema; migrate legacy quota/config values into a new temporary document; validate invariants; atomically install it; only then delete legacy plaintext/duplicate state. The P1 migration-complete marker is written only after P1 was read successfully and every resulting P2 observation was durably appended; a read/write failure remains retryable on the next activation. - Secret migration writes SecretStorage first, then removes all explicit Global, current Workspace, every open Workspace Folder, and old globalState value. Every closed workspace is inspected when it is next opened. A workspace-only or conflicting key is never silently promoted into the one global secret slot: activation stops with fixed, localized recovery instructions and leaves plaintext recoverable. Until the user removes or reconciles that blocked legacy workspace value, it remains ordinary configuration and may remain eligible for Settings Sync. Repeat/concurrent machine-wide migration is idempotent because only identical global candidates are automatic. Other failures likewise preserve the legacy value and stop activation rather than pretending migration succeeded. - [VS Code's extension storage documentation](https://code.visualstudio.com/api/extension-capabilities/common-capabilities#data-storage) states that SecretStorage is encrypted and is not synchronized across machines. The extension relies on the active VS Code-compatible host's SecretStorage implementation, whose concrete backing protection can vary by host. Ordinary VS Code configuration can still participate in Settings Sync according to the user's VS Code settings. - Unknown reasoning effort is retained only when the field is missing or unrecognized. Non-zero unknown is visible and explained; zero unknown is omitted. Effort components reconcile with provider totals. diff --git a/docs/superpowers/specs/2026-09-02-v2.3.1-local-data-contract.zh-CN.md b/docs/superpowers/specs/2026-09-02-v2.3.1-local-data-contract.zh-CN.md index 7577e80d..42a67fe0 100644 --- a/docs/superpowers/specs/2026-09-02-v2.3.1-local-data-contract.zh-CN.md +++ b/docs/superpowers/specs/2026-09-02-v2.3.1-local-data-contract.zh-CN.md @@ -61,6 +61,7 @@ interface QuotaObservationV2 { resetAt: number | null; source: 'claude-official-api' | 'codex-local-structured-event'; windowId: string; // HMAC(provider, fingerprint, period, reset identity/epoch) + providerWindowFingerprint: string | null; // 已知 provider window ID 的 HMAC,绝不保存原值 confidence: 'high' | 'medium' | 'low'; captureReason: 'refresh' | 'reset-at-change' | 'window-id-change' | 'usage-drop' | 'profile-change' | 'migration'; flags: Array<'approximate-boundary' | 'low-log-coverage' | 'low-price-coverage' | 'account-ambiguous' | 'clock-anomaly'>; @@ -73,6 +74,7 @@ interface QuotaObservationV2 { - 文件路径只能说明本机 profile,不能证明真实账号。此类观测标为 `profile-continuity`,检测到登录/profile 改变后不得继续合并。 - Codex 若没有安全稳定的账号信号,就生成随机 `unattributed` epoch,绝不能把 Codex home hash 冒充账号。窗口变化、`resetAt` 变化、明显百分比回落或显式 profile 变化都会关闭 epoch。 - 只有 fingerprint 相同且 attribution 兼容的观测才能稳健聚合。不同 `unattributed` epoch 即使 reset 时间相同也绝不合并。 +- 持久化的账号、provider window 与 window HMAC 不得进入 Webview 状态或导出;宿主在渲染前必须替换为仅对本次投影有效的临时别名。 ### 3.2 校验、去重和原子写入 @@ -85,7 +87,7 @@ interface QuotaObservationV2 { ### 3.3 重置事件与派生估算 -重置/窗口边界按以下优先级判断:稳定 window ID 改变、`resetAt` identity 改变、超过噪声阈值的 used fraction 回落。同一天多次重置必须分别保留。有事件时间戳时按事件切分;只有日汇总时明确标记近似。 +重置/窗口边界按以下优先级判断:稳定 window ID 改变、`resetAt` identity 改变、超过噪声阈值的 used fraction 回落。同一天多次重置必须分别保留。重置事件由边界观测在运行期确定性派生,不另建第二张持久事实表。有事件时间戳时按事件切分;只有日汇总时明确标记近似。 同一稳定 fingerprint/window 内,每个有效观测产生候选: @@ -93,7 +95,7 @@ interface QuotaObservationV2 { candidateFullEquivalent = pricedUsedEquivalentThroughObservedAt / usedFraction ``` -只接受 `0 < usedFraction <= 1`。多个候选按定价覆盖、日志/索引覆盖、边界质量、账号归属和时效做稳健聚合。展示的总额度不得低于最终确认的已用等价值;未用值为 `max(full - confirmedUsed, 0)`,仅在账号连续性与窗口边界足够可靠时显示。多账号或未归属证据可以显示已用值及原因,但不得声称账号级未用额度。 +只接受 `0 < usedFraction <= 1`。多个候选按定价覆盖、日志/索引覆盖、边界质量、账号归属和时效做稳健聚合。展示的总额度不得低于最终确认的已用等价值。一个内部一致的 Codex 未归属窗口可以显示低可信度总额度估算;未用值为 `max(full - confirmedUsed, 0)`,仅在账号连续性与窗口边界足够可靠时显示。未归属 epoch 重叠、账号证据冲突或多账号用量含混时,只显示已用等价值,不虚构账号拆分或账号级未用额度。 总额/未用估算不作为永久事实落盘;单价、覆盖、日志或观测变化后重新计算。 @@ -135,7 +137,7 @@ G6 前,设置/Data 页面和命令面板应提供: ## 7. 迁移与发版门槛 -- 迁移顺序:加载并校验当前 schema;将旧额度/config 写入新临时文档;验证不变量;原子安装;最后删除旧明文或重复状态。 +- 迁移顺序:加载并校验当前 schema;将旧额度/config 写入新临时文档;验证不变量;原子安装;最后删除旧明文或重复状态。只有成功读取 P1 且由此得到的 P2 观测均已持久追加后,才写入 P1 迁移完成标记;读取或写入失败时保留为下次激活可重试状态。 - Secret 迁移先写 SecretStorage,再清除 Global、当前 Workspace、所有已打开 Workspace Folder 和旧 globalState 中的明文值;关闭的 workspace 会在下次打开时检查。workspace-only 或相互冲突的 key 不会静默提升为一个全局 secret:插件用固定、已本地化的恢复提示停止激活,并保留可恢复明文。在用户删除或协调该受阻的旧 workspace 值之前,它仍属于普通 configuration,因而仍可能参与 Settings Sync。机器级自动迁移只接受相同候选,因此重复/并发激活保持幂等。其他失败同样保留旧值并停止,不得假装迁移成功。 - [VS Code 扩展存储官方说明](https://code.visualstudio.com/api/extension-capabilities/common-capabilities#data-storage)指出 SecretStorage 会加密且不跨设备同步;插件依赖当前 VS Code 兼容宿主的 SecretStorage 实现,而其具体底层保护方式可能因宿主而异。普通 VS Code configuration 仍可能按用户的 Settings Sync 设置同步。 - `unknown` effort 只在字段缺失或无法识别时保留;非零时展示并解释,零时隐藏;各 effort 分量必须与 provider 总计守恒。 diff --git a/src/claudeApiClient.ts b/src/claudeApiClient.ts index 081e1b5a..dd4c4bb5 100644 --- a/src/claudeApiClient.ts +++ b/src/claudeApiClient.ts @@ -1,4 +1,5 @@ import { execFileSync } from 'child_process'; +import { createHmac } from 'node:crypto'; import * as fs from 'fs'; import * as os from 'os'; import * as path from 'path'; @@ -13,6 +14,15 @@ export interface ResolvedClaudeProfile { allowKeychainFallback: boolean; } +export function claudeQuotaIdentitySignal( + machineSalt: string, + refreshToken: string, +): string { + return createHmac('sha256', machineSalt) + .update(`claude-quota-account-v1|${refreshToken}`) + .digest('hex'); +} + function isDirectory(candidate: string): boolean { try { return fs.existsSync(candidate) && fs.statSync(candidate).isDirectory(); @@ -85,10 +95,12 @@ export class ClaudeApiClient { // Once curl has succeeded after fetch failed, remember so we don't keep // paying the cost of a doomed fetch attempt on every refresh. private preferCurl: boolean = false; + private lastQuotaIdentitySignal: string | null = null; constructor( out: vscode.OutputChannel | null = null, dataDirectory?: string | null, + private readonly quotaIdentitySalt?: string, ) { const profile = resolveClaudeProfile(dataDirectory); this.credentialsPath = profile.credentialsPath; @@ -134,6 +146,13 @@ export class ClaudeApiClient { return this.credentialsPath; } + /** Machine-local one-way continuity signal for the exact credentials used + * by the last successful quota request. No token or account identifier is + * returned to the extension host. */ + getLastQuotaIdentitySignal(): string | null { + return this.lastQuotaIdentitySignal; + } + private loadCredentialsFromKeychain(): ClaudeCredentials | null { if (process.platform !== 'darwin') { return null; @@ -325,7 +344,7 @@ export class ClaudeApiClient { } try { - const credentials = await this.getValidCredentials(signal); + let credentials = await this.getValidCredentials(signal); if (!credentials) { return null; } @@ -351,6 +370,7 @@ export class ClaudeApiClient { this.log('401: forcing token refresh and retrying once'); try { const refreshed = await this.refreshAccessToken(credentials, signal); + credentials = refreshed; response = await this.callUsageApi( refreshed.claudeAiOauth.accessToken, signal, @@ -366,6 +386,12 @@ export class ClaudeApiClient { return null; } const data = JSON.parse(response.body) as ClaudeApiUsageResponse; + this.lastQuotaIdentitySignal = this.quotaIdentitySalt + ? claudeQuotaIdentitySignal( + this.quotaIdentitySalt, + credentials.claudeAiOauth.refreshToken, + ) + : null; this.log(`usage: ok — 5h=${data.five_hour?.utilization ?? 'n/a'}%, wk=${data.seven_day?.utilization ?? 'n/a'}%`); return data; } catch (e) { diff --git a/src/extension.ts b/src/extension.ts index 1697abc7..ddb9ac28 100644 --- a/src/extension.ts +++ b/src/extension.ts @@ -27,10 +27,20 @@ import { ClaudeApiUsageResponse, ContentAnalysis, ExtensionConfig } from './type import { SettingsSecretMigrationError, SettingsStore } from './settings'; import { normalizeQuotaWindows } from './quotaWindows'; import { - appendWeeklyQuotaObservations, - claudeWeeklyQuotaObservations, WeeklyQuotaObservation, } from './weeklyValue'; +import { + claudeQuotaCapturesFromUsage, + codexQuotaCapturesFromWeeklyObservations, + createEmptyQuotaObservationStore, + fingerprintForStableIdentity, + mergeQuotaCaptures, + QUOTA_OBSERVATION_FILE, + QuotaCapture, + QuotaObservationRepository, + QuotaObservationStoreV2, + quotaStoreWeeklyObservations, +} from './quotaObservationStore'; import { diffUsageManifests, scanUsageManifest, @@ -58,7 +68,11 @@ import { CodexProvider, CodexProviderSnapshot, } from './providers/codex/codexProvider'; -import { CodexIndexProgress } from './providers/codex/codexIndex'; +import { + CodexIndexProgress, + loadCodexIndex, +} from './providers/codex/codexIndex'; +import { weeklyQuotaObservationsFromCodexHistory } from './providers/codex/codexQuotaHistory'; import { resolveCodexHome } from './providers/codex/codexManifest'; import { buildCodexUsageView, CodexUsageView } from './providers/codex/codexUsage'; import { @@ -115,6 +129,171 @@ interface ActiveNetworkOperation { readonly settled: Promise; } +interface QuotaObservationRuntime { + repository: QuotaObservationRepository; + store: QuotaObservationStoreV2; + salt: string; +} + +const QUOTA_FINGERPRINT_SALT_KEY = 'ccu.quota.fingerprintSalt.v1'; +const QUOTA_P5_MIGRATION_KEY = 'ccu.quota.migratedCodexIndex.v2'; +const LEGACY_QUOTA_PREFIX = 'ccu.usageLimits.'; +const LEGACY_WEEKLY_PREFIX = 'ccu.weeklyQuotaHistory.v1.'; + +function legacyProfileSuffix(credentialsPath: string): string { + return createHash('sha256').update(credentialsPath).digest('hex').slice(0, 16); +} + +function objectRecord(value: unknown): Record | null { + return typeof value === 'object' && value !== null && !Array.isArray(value) + ? value as Record + : null; +} + +function legacyWeeklyCaptures( + value: unknown, + identitySignal: string, +): QuotaCapture[] { + if (!Array.isArray(value)) return []; + return value.flatMap((entry) => { + const item = objectRecord(entry); + if ( + !item || + item.provider !== 'claude' || + typeof item.observedAt !== 'number' || + !Number.isFinite(item.observedAt) || + typeof item.resetAt !== 'number' || + !Number.isFinite(item.resetAt) || + typeof item.usedPercent !== 'number' || + !Number.isFinite(item.usedPercent) + ) return []; + return [{ + provider: 'claude' as const, + stableIdentitySignal: identitySignal, + accountAttribution: 'profile-continuity' as const, + observedAt: item.observedAt, + periodType: 'seven-day' as const, + usedFraction: Math.max(0, Math.min(1, item.usedPercent / 100)), + resetAt: item.resetAt, + source: 'claude-official-api' as const, + confidence: 'low' as const, + captureReason: 'migration' as const, + flags: ['account-ambiguous' as const], + }]; + }); +} + +/** Load and transactionally migrate quota facts before any provider refresh + * starts. Legacy values are removed only after the schema-2 file is durable. */ +export async function initializeQuotaObservationRuntime( + context: vscode.ExtensionContext, + settings: SettingsStore, +): Promise { + let salt = context.globalState.get(QUOTA_FINGERPRINT_SALT_KEY); + if (!salt) { + salt = randomBytes(32).toString('hex'); + await context.globalState.update(QUOTA_FINGERPRINT_SALT_KEY, salt); + } + const repository = new QuotaObservationRepository( + path.join(context.globalStorageUri.fsPath, QUOTA_OBSERVATION_FILE), + salt, + ); + let store = await repository.load(); + const migrationCaptures: QuotaCapture[] = []; + const removableLegacyKeys: string[] = []; + let codexLegacyMigrationReady = context.globalState.get( + QUOTA_P5_MIGRATION_KEY, + ) === true; + const profileClient = new ClaudeApiClient( + null, + settings.get('dataDirectory'), + salt, + ); + const currentCredentialsPath = profileClient.getCredentialsPath(); + const currentSuffix = legacyProfileSuffix(currentCredentialsPath); + const keys = typeof context.globalState.keys === 'function' + ? context.globalState.keys() + : []; + for (const key of keys) { + if (key.startsWith(LEGACY_QUOTA_PREFIX)) { + const suffix = key.slice(LEGACY_QUOTA_PREFIX.length); + const saved = objectRecord(context.globalState.get(key)); + const data = saved ? saved.data as ClaudeApiUsageResponse | undefined : undefined; + const ts = saved?.ts; + if (data && typeof ts === 'number' && Number.isFinite(ts) && ts > 0) { + const identitySignal = suffix === currentSuffix + ? `profile-path-v1|${currentCredentialsPath}` + : `legacy-profile-v1|${suffix}`; + const additions = claudeQuotaCapturesFromUsage( + data, + identitySignal, + 'profile-continuity', + ts, + 'migration', + ); + if (additions.length > 0) { + migrationCaptures.push(...additions.map((item) => ({ + ...item, + confidence: 'low' as const, + flags: [...(item.flags ?? []), 'account-ambiguous' as const], + }))); + removableLegacyKeys.push(key); + } + } + continue; + } + if (key.startsWith(LEGACY_WEEKLY_PREFIX)) { + const suffix = key.slice(LEGACY_WEEKLY_PREFIX.length); + const identitySignal = suffix === currentSuffix + ? `profile-path-v1|${currentCredentialsPath}` + : `legacy-profile-v1|${suffix}`; + const additions = legacyWeeklyCaptures( + context.globalState.get(key), + identitySignal, + ); + if (additions.length > 0) { + migrationCaptures.push(...additions); + removableLegacyKeys.push(key); + } + } + } + + if (!codexLegacyMigrationReady) { + try { + const index = await loadCodexIndex( + path.join(context.globalStorageUri.fsPath, 'codex-index-v1.json'), + resolveTimeZone(settings.get('timezone')), + ); + const legacyHistory = [ + ...(index.quotaHistory ?? []), + ...Object.values(index.files).flatMap((file) => file.quotaHistory ?? []), + ]; + migrationCaptures.push(...codexQuotaCapturesFromWeeklyObservations( + weeklyQuotaObservationsFromCodexHistory(legacyHistory), + 'migration', + )); + codexLegacyMigrationReady = true; + // Set only after the P2 append below succeeds. + } catch { + // A corrupt/absent P1 cannot block activation or destroy its only copy. + } + } + + if (migrationCaptures.length > 0) { + store = await repository.append(migrationCaptures); + } + for (const key of removableLegacyKeys) { + await context.globalState.update(key, undefined); + } + if ( + codexLegacyMigrationReady && + !context.globalState.get(QUOTA_P5_MIGRATION_KEY) + ) { + await context.globalState.update(QUOTA_P5_MIGRATION_KEY, true); + } + return { repository, store, salt }; +} + // Full-version entries only: an installed patch must never inherit stale notes // from an older major/minor release. const WHATS_NEW: ReleaseAnnouncementCatalog = { @@ -195,6 +374,10 @@ export class ClaudeCodeUsageExtension { private quotaColdRetryTimerLease: ResourceLease | undefined; private claudeProfileGeneration: number = 0; private claudeWeeklyQuotaHistory: WeeklyQuotaObservation[] = []; + private readonly quotaObservationRepository: QuotaObservationRepository; + private quotaObservationStore: QuotaObservationStoreV2; + private readonly quotaFingerprintSalt: string; + private activeClaudeQuotaFingerprint: string | undefined; private codexProvider: CodexProvider; private readonly codexSalt: string; private codexView: CodexUsageView | null = null; @@ -232,6 +415,7 @@ export class ClaudeCodeUsageExtension { constructor( private context: vscode.ExtensionContext, settings?: SettingsStore, + quotaRuntime?: QuotaObservationRuntime, ) { console.log('Claude Code Usage Extension: Constructor called'); this.outputChannel = vscode.window.createOutputChannel('Claude Code Usage'); @@ -263,9 +447,33 @@ export class ClaudeCodeUsageExtension { void this.saveCodexBackgroundState().catch(() => undefined); } this.webviewProvider = new UsageWebviewProvider(context); + const existingQuotaSalt = context.globalState.get( + 'ccu.quota.fingerprintSalt.v1', + ); + this.quotaFingerprintSalt = quotaRuntime?.salt ?? + existingQuotaSalt ?? randomBytes(32).toString('hex'); + if (!existingQuotaSalt && !quotaRuntime) { + void context.globalState.update( + 'ccu.quota.fingerprintSalt.v1', + this.quotaFingerprintSalt, + ); + } + this.quotaObservationRepository = quotaRuntime?.repository ?? + new QuotaObservationRepository( + path.join(context.globalStorageUri.fsPath, QUOTA_OBSERVATION_FILE), + this.quotaFingerprintSalt, + ); + this.quotaObservationStore = quotaRuntime?.store ?? + createEmptyQuotaObservationStore(); this.apiClient = new ClaudeApiClient( this.outputChannel, this.settings.get('dataDirectory'), + this.quotaFingerprintSalt, + ); + this.activeClaudeQuotaFingerprint = fingerprintForStableIdentity( + this.quotaFingerprintSalt, + 'claude', + this.claudeProfileContinuitySignal(), ); const existingCodexSalt = context.globalState.get('ccu.codex.machineSalt'); this.codexSalt = existingCodexSalt ?? randomBytes(32).toString('hex'); @@ -356,7 +564,7 @@ export class ClaudeCodeUsageExtension { this.setupCommands(); this.loadConfiguration(); - this.loadPersistedQuota(); + this.refreshQuotaObservationViews(); if (this.windowActivity.focused) { this.startAutoRefresh(); const startupGeneration = this.configurationGeneration; @@ -1729,9 +1937,36 @@ export class ClaudeCodeUsageExtension { private applyCodexSnapshot(snapshot: CodexProviderSnapshot): void { this.codexView = buildCodexUsageView(snapshot); + const capturedObservations = snapshot.weeklyValueInputs?.observations ?? []; + const capturedQuotaFacts = codexQuotaCapturesFromWeeklyObservations( + capturedObservations, + ); + const previewStore = capturedQuotaFacts.length > 0 + ? mergeQuotaCaptures(this.quotaObservationStore, capturedQuotaFacts, { + salt: this.quotaFingerprintSalt, + now: capturedQuotaFacts.reduce( + (latest, item) => Math.max(latest, item.observedAt), + Date.now(), + ), + }) + : this.quotaObservationStore; + const storedCodexObservations = quotaStoreWeeklyObservations( + previewStore, + 'codex', + ); + if (storedCodexObservations.length > 0 && this.codexView.weeklyValueInputs) { + this.codexView.weeklyValueInputs = { + ...this.codexView.weeklyValueInputs, + observations: storedCodexObservations, + }; + } this.codexInsights = buildScopedCodexInsights(this.codexView); this.codexAvailable = true; this.codexHasData = snapshot.coverage.totalFiles > 0; + void this.recordCodexQuotaObservations(capturedObservations).catch(() => { + // P1 remains readable when P2 is temporarily unavailable. The next + // refresh retries the exact idempotent observation set. + }); } /** Adopt the first atomic checkpoint during a brand-new cold index. */ @@ -1901,7 +2136,11 @@ export class ClaudeCodeUsageExtension { * A profile switch invalidates every in-memory quota/backoff value because it * belongs to a different account. */ private selectClaudeProfile(dataDirectory?: string | null): void { - const nextClient = new ClaudeApiClient(this.outputChannel, dataDirectory); + const nextClient = new ClaudeApiClient( + this.outputChannel, + dataDirectory, + this.quotaFingerprintSalt, + ); if (nextClient.getCredentialsPath() === this.apiClient.getCredentialsPath()) { return; } @@ -1917,8 +2156,13 @@ export class ClaudeCodeUsageExtension { this.statusBar.updateQuota(null); this.webviewProvider.updateQuota(null); this.claudeWeeklyQuotaHistory = []; + this.activeClaudeQuotaFingerprint = fingerprintForStableIdentity( + this.quotaFingerprintSalt, + 'claude', + this.claudeProfileContinuitySignal(), + ); this.webviewProvider.updateWeeklyQuotaHistory([]); - this.loadPersistedQuota(); + this.refreshQuotaObservationViews(); } /** @@ -2214,6 +2458,10 @@ export class ClaudeCodeUsageExtension { this.cache.usageLimitsLastUpdate = new Date(0); this.cache.usageLimitsFailStreak = 0; this.cache.usageLimitsBackoffUntil = new Date(0); + this.claudeProfileGeneration += 1; + this.activeClaudeQuotaFingerprint = undefined; + this.claudeWeeklyQuotaHistory = []; + this.webviewProvider.updateWeeklyQuotaHistory([]); void this.refreshData(false, 'credentials'); } @@ -2410,81 +2658,73 @@ export class ClaudeCodeUsageExtension { tick(); } - /** Fetch real usage limits via OAuth, cached for 2 minutes. */ - // Persist the last-known quota across reloads/restarts. - private static readonly QUOTA_STATE_KEY = 'ccu.usageLimits'; - private static readonly WEEKLY_QUOTA_STATE_KEY = 'ccu.weeklyQuotaHistory.v1'; - - private quotaProfileHash(): string { - return createHash('sha256') - .update(this.apiClient.getCredentialsPath()) - .digest('hex') - .slice(0, 16); - } - - private quotaStateKey(): string { - return `${ClaudeCodeUsageExtension.QUOTA_STATE_KEY}.${this.quotaProfileHash()}`; + /** Fetch real usage limits via OAuth, cached in memory for at most two + * minutes. Durable history contains only schema-2 normalized observations; + * the raw OAuth response is never retained by the extension. */ + private claudeProfileContinuitySignal(): string { + return `profile-path-v1|${this.apiClient.getCredentialsPath()}`; } - private weeklyQuotaStateKey(): string { - return `${ClaudeCodeUsageExtension.WEEKLY_QUOTA_STATE_KEY}.${this.quotaProfileHash()}`; - } - - private loadPersistedQuota(): void { - if (!this.getConfiguration().usageLimitTracking) { - return; - } - try { - const storedHistory = this.context.globalState.get( - this.weeklyQuotaStateKey(), - ) ?? []; - const saved = this.context.globalState.get<{ data: ClaudeApiUsageResponse; ts: number }>( - this.quotaStateKey() - ); - const savedObservation = saved?.data - ? claudeWeeklyQuotaObservations( - saved.data, - this.quotaProfileHash(), - saved.ts || 0, - ) - : []; - this.claudeWeeklyQuotaHistory = appendWeeklyQuotaObservations( - [], - [...storedHistory, ...savedObservation], - ); - this.webviewProvider.updateWeeklyQuotaHistory(this.claudeWeeklyQuotaHistory); - if (saved && saved.data) { - this.cache.usageLimits = saved.data; - this.cache.usageLimitsLastUpdate = new Date(saved.ts || 0); - this.statusBar.updateQuota(saved.data); - this.webviewProvider.updateQuota(saved.data); - } - } catch { - /* ignore corrupt persisted state */ + private refreshQuotaObservationViews(): void { + this.claudeWeeklyQuotaHistory = this.activeClaudeQuotaFingerprint + ? quotaStoreWeeklyObservations( + this.quotaObservationStore, + 'claude', + this.activeClaudeQuotaFingerprint, + ) + : []; + this.webviewProvider.updateWeeklyQuotaHistory(this.claudeWeeklyQuotaHistory); + if (this.codexView?.weeklyValueInputs) { + this.codexView.weeklyValueInputs = { + ...this.codexView.weeklyValueInputs, + observations: quotaStoreWeeklyObservations( + this.quotaObservationStore, + 'codex', + ), + }; } } - private recordClaudeWeeklyQuota( + private async recordClaudeQuotaObservation( usage: ClaudeApiUsageResponse, observedAt: number, - ): void { - const additions = claudeWeeklyQuotaObservations( + ): Promise { + const verifiedSignal = this.apiClient.getLastQuotaIdentitySignal(); + const identitySignal = verifiedSignal ?? this.claudeProfileContinuitySignal(); + const accountAttribution = verifiedSignal + ? 'verified-local-signal' as const + : 'profile-continuity' as const; + const additions = claudeQuotaCapturesFromUsage( usage, - this.quotaProfileHash(), + identitySignal, + accountAttribution, observedAt, ); + if (additions.length === 0) return; + this.quotaObservationStore = await this.quotaObservationRepository.append( + additions, + ); + this.activeClaudeQuotaFingerprint = fingerprintForStableIdentity( + this.quotaFingerprintSalt, + 'claude', + identitySignal, + ); + this.refreshQuotaObservationViews(); + } + + private async recordCodexQuotaObservations( + observations: readonly WeeklyQuotaObservation[], + ): Promise { + const additions = codexQuotaCapturesFromWeeklyObservations(observations); if (additions.length === 0) { + this.refreshQuotaObservationViews(); return; } - this.claudeWeeklyQuotaHistory = appendWeeklyQuotaObservations( - this.claudeWeeklyQuotaHistory, + this.quotaObservationStore = await this.quotaObservationRepository.append( additions, ); - this.webviewProvider.updateWeeklyQuotaHistory(this.claudeWeeklyQuotaHistory); - void this.context.globalState.update( - this.weeklyQuotaStateKey(), - this.claudeWeeklyQuotaHistory, - ); + this.refreshQuotaObservationViews(); + if (!this.disposed) this.syncProviderUi(); } private async maybeFetchUsageLimits(config: ExtensionConfig): Promise { @@ -2572,12 +2812,7 @@ export class ClaudeCodeUsageExtension { // expired-window bypass so a just-rolled window can't trigger an immediate // refetch. this.cache.usageLimitsBackoffUntil = new Date(Date.now() + 30000); - // Write through to disk so the next startup/reload has it instantly. - void this.context.globalState.update( - this.quotaStateKey(), - { data: fetched, ts: observedAt } - ); - this.recordClaudeWeeklyQuota(fetched, observedAt); + await this.recordClaudeQuotaObservation(fetched, observedAt); return fetched; } // Failed (usually a 429 or invalid/expired credentials). Exponentially back @@ -2946,7 +3181,12 @@ export async function activate(context: vscode.ExtensionContext): Promise ); throw error; } - const extension = new ClaudeCodeUsageExtension(context, settings); + const quotaRuntime = await initializeQuotaObservationRuntime(context, settings); + const extension = new ClaudeCodeUsageExtension( + context, + settings, + quotaRuntime, + ); activeExtension = extension; context.subscriptions.push({ dispose: () => { void extension.dispose(); } diff --git a/src/i18n.ts b/src/i18n.ts index fba8a26b..5f91fc1f 100644 --- a/src/i18n.ts +++ b/src/i18n.ts @@ -1049,7 +1049,7 @@ const WEEKLY_VALUE_COPY: Record = { noData: 'No locally recorded weekly usage is available yet.', historyFromLogs: 'Historical used equivalents come directly from local token logs. Full and unused estimates appear only for windows with a real quota-utilization observation.', calendarFallback: 'No historical weekly reset was observed; usage-only history is grouped into Monday-to-Monday UTC calendar weeks.', - multiAccount: 'Codex usage-only history combines all sign-ins in this Codex home. When usage cannot be reliably attributed to a single quota observation, only the used equivalent is shown; no account split or allowance estimate is invented.', + multiAccount: 'Codex local history may span multiple sign-ins. One coherent unattributed quota window can show a low-confidence total estimate, but not unused value. If epochs overlap or attribution conflicts, only used equivalent is shown; no account split is invented.', boundaryApproximation: 'Codex usage is aggregated by day. If an official reset falls within a recorded day, the affected period may show a low-confidence full-allowance approximation; current unused value remains withheld.', boundaryApproximate: 'Boundary approx.', indexedSubtotal: 'Indexing is incomplete; weekly values are conservative subtotals.', @@ -1064,7 +1064,7 @@ const WEEKLY_VALUE_COPY: Record = { noData: '尚无可用于按周计算的本地用量记录。', historyFromLogs: '历史“已用等价值”直接由本地 Token 日志计算;只有某个窗口存在真实额度用量观测时,才显示总额度和未用额度估算。', calendarFallback: '未观测到可用于对齐历史的每周重置时间;仅已用历史按 UTC 周一至周一的自然周分组。', - multiAccount: 'Codex 的仅已用历史会合并此 Codex home 中的全部登录。用量无法可靠归属到单一额度观测时,只显示已用等价值;不会虚构账号拆分或额度估算。', + multiAccount: 'Codex 本地历史可能跨多个登录。单个内部一致的未归属额度窗口可显示低可信度总额估算,但不显示未用值;epoch 重叠或归属冲突时只显示已用等价值,不虚构账号拆分。', boundaryApproximation: 'Codex 用量按日汇总。如果官方重置发生在某个已记录日期内,受影响周期仍可显示低可信度的总额度近似值;当前周期仍不显示未用估算。', boundaryApproximate: '边界近似', indexedSubtotal: '索引尚未完成;每周价值目前是保守小计。', @@ -1079,7 +1079,7 @@ const WEEKLY_VALUE_COPY: Record = { noData: '尚無可用於每週計算的本機用量記錄。', historyFromLogs: '歷史「已用等價值」直接由本機 Token 日誌計算;只有視窗存在真實額度用量觀測時,才顯示總額度與未用額度估算。', calendarFallback: '未觀測到可用於對齊歷史的每週重設時間;僅已用歷史依 UTC 週一至週一的自然週分組。', - multiAccount: 'Codex 的僅已用歷史會合併此 Codex home 中的所有登入。用量無法可靠歸屬到單一額度觀測時,只顯示已用等價值;不會虛構帳號拆分或額度估算。', + multiAccount: 'Codex 本機歷史可能跨多個登入。單一且內部一致的未歸屬額度視窗可顯示低可信度總額估算,但不顯示未用值;epoch 重疊或歸屬衝突時只顯示已用等價值,不虛構帳號拆分。', boundaryApproximation: 'Codex 用量按日彙總。如果官方重設發生在某個已記錄日期內,受影響週期仍可顯示低可信度的總額度近似值;目前週期仍不顯示未用估算。', boundaryApproximate: '邊界近似', indexedSubtotal: '索引尚未完成;每週價值目前是保守小計。', @@ -1094,7 +1094,7 @@ const WEEKLY_VALUE_COPY: Record = { noData: '週単位で計算できるローカル使用記録がまだありません。', historyFromLogs: '過去の使用済み等価値はローカルの Token ログから直接計算します。総上限と未使用分は、実際の上限利用率が観測された枠だけで推定します。', calendarFallback: '履歴を揃える週間リセットが観測されていないため、使用分のみの履歴は UTC の月曜から月曜の暦週で集計します。', - multiAccount: 'Codex の使用分のみの履歴は、この Codex home の全ログインを合算します。使用量を単一の上限観測に確実に帰属できない場合は使用済み等価値だけを表示し、アカウント分割や上限推定を作りません。', + multiAccount: 'Codex のローカル履歴には複数のログインが含まれる場合があります。一貫した単一の未帰属クォータ枠では低信頼度の総量推定を表示できますが、未使用量は表示しません。epoch の重複や帰属競合がある場合は使用済み等価値のみを表示し、アカウント分割を推測しません。', boundaryApproximation: 'Codex の使用量は日単位で集計されます。公式リセットが記録日の途中にある場合、影響する期間には低信頼度の総上限近似を表示できます。現在の未使用分は表示しません。', boundaryApproximate: '境界近似', indexedSubtotal: '索引作成中のため、週間価値は保守的な小計です。', @@ -1109,7 +1109,7 @@ const WEEKLY_VALUE_COPY: Record = { noData: '주간 계산에 사용할 로컬 사용 기록이 아직 없습니다.', historyFromLogs: '과거 사용 등가치는 로컬 Token 로그에서 직접 계산합니다. 실제 한도 사용률 관측이 있는 창에서만 총한도와 미사용분을 추정합니다.', calendarFallback: '과거를 정렬할 주간 재설정이 관측되지 않아 사용분 전용 기록은 UTC 월요일부터 월요일까지의 달력 주로 묶습니다.', - multiAccount: 'Codex 사용분 전용 기록은 이 Codex home의 모든 로그인을 합산합니다. 사용량을 단일 할당량 관측에 안정적으로 귀속할 수 없으면 사용 등가치만 표시하며 계정 분리나 한도 추정을 만들어 내지 않습니다.', + multiAccount: 'Codex 로컬 기록에는 여러 로그인이 포함될 수 있습니다. 일관된 단일 미귀속 할당량 창은 신뢰도 낮은 총량 추정을 표시할 수 있지만 미사용 값은 표시하지 않습니다. epoch가 겹치거나 귀속이 충돌하면 사용 등가치만 표시하며 계정 분리를 추정하지 않습니다.', boundaryApproximation: 'Codex 사용량은 일별로 집계됩니다. 공식 재설정이 기록된 하루 중간에 발생하면 영향을 받는 기간에 낮은 신뢰도의 총한도 근사를 표시할 수 있으며, 현재 미사용 값은 표시하지 않습니다.', boundaryApproximate: '경계 근사', indexedSubtotal: '인덱싱이 끝나지 않아 주간 가치는 보수적인 소계입니다.', @@ -1124,7 +1124,7 @@ const WEEKLY_VALUE_COPY: Record = { noData: 'Ainda não há uso local registrado para o cálculo semanal.', historyFromLogs: 'Os equivalentes usados no histórico vêm diretamente dos logs locais de Token. O limite total e o não usado só são estimados quando há uma observação real da utilização da cota.', calendarFallback: 'Nenhuma redefinição semanal histórica foi observada; o histórico somente de uso é agrupado em semanas UTC de segunda a segunda.', - multiAccount: 'O histórico somente de uso do Codex combina todos os logins deste Codex home. Quando o uso não pode ser atribuído com segurança a uma única observação de cota, só o equivalente usado é mostrado; nenhuma divisão por conta ou estimativa de limite é inventada.', + multiAccount: 'O histórico local do Codex pode abranger vários logins. Uma única janela de cota não atribuída e coerente pode mostrar uma estimativa total de baixa confiança, mas não o valor não usado. Se épocas se sobrepõem ou a atribuição entra em conflito, só o equivalente usado é mostrado; nenhuma divisão de conta é inventada.', boundaryApproximation: 'O uso do Codex é agregado por dia. Se uma redefinição oficial ocorrer dentro de um dia registrado, o período afetado pode mostrar uma aproximação de baixa confiança do limite total; o não usado atual continua oculto.', boundaryApproximate: 'Fronteira aprox.', indexedSubtotal: 'A indexação está incompleta; os valores semanais são subtotais conservadores.', @@ -1139,7 +1139,7 @@ const WEEKLY_VALUE_COPY: Record = { noData: 'Noch keine lokal erfasste Nutzung für die Wochenberechnung verfügbar.', historyFromLogs: 'Historische genutzte Gegenwerte stammen direkt aus lokalen Token-Logs. Gesamtlimit und ungenutzter Anteil werden nur bei real beobachteter Quotenauslastung geschätzt.', calendarFallback: 'Kein historischer Wochen-Reset wurde beobachtet; reine Nutzungsverläufe werden in UTC-Kalenderwochen von Montag bis Montag gruppiert.', - multiAccount: 'Der reine Codex-Nutzungsverlauf kombiniert alle Anmeldungen in diesem Codex home. Lässt sich die Nutzung nicht zuverlässig einer einzelnen Kontingentbeobachtung zuordnen, wird nur der genutzte Gegenwert angezeigt; weder Kontotrennung noch Limitschätzung werden erfunden.', + multiAccount: 'Der lokale Codex-Verlauf kann mehrere Anmeldungen umfassen. Ein einzelnes, stimmiges, nicht zugeordnetes Kontingentfenster kann eine Gesamtschätzung mit niedriger Zuverlässigkeit zeigen, jedoch keinen ungenutzten Wert. Bei überlappenden Epochen oder Zuordnungskonflikten erscheint nur der genutzte Gegenwert; eine Kontotrennung wird nicht erfunden.', boundaryApproximation: 'Die Codex-Nutzung wird tageweise aggregiert. Fällt ein offizieller Reset in einen erfassten Tag, kann der betroffene Zeitraum eine Schätzung des Gesamtlimits mit niedriger Sicherheit zeigen; der aktuelle ungenutzte Wert bleibt verborgen.', boundaryApproximate: 'Grenznäherung', indexedSubtotal: 'Die Indizierung ist unvollständig; Wochenwerte sind konservative Zwischensummen.', @@ -1154,7 +1154,7 @@ const WEEKLY_VALUE_COPY: Record = { noData: 'Belum ada penggunaan lokal yang tercatat untuk perhitungan mingguan.', historyFromLogs: 'Ekuivalen terpakai historis dihitung langsung dari log Token lokal. Total batas dan sisa hanya diestimasi jika ada pengamatan nyata atas persentase kuota.', calendarFallback: 'Tidak ada reset mingguan historis yang teramati; riwayat khusus pemakaian dikelompokkan dalam minggu UTC Senin-ke-Senin.', - multiAccount: 'Riwayat khusus pemakaian Codex menggabungkan semua login di Codex home ini. Jika penggunaan tidak dapat dikaitkan secara andal ke satu pengamatan kuota, hanya ekuivalen terpakai yang ditampilkan; pemisahan akun atau perkiraan batas tidak direka.', + multiAccount: 'Riwayat lokal Codex dapat mencakup beberapa login. Satu jendela kuota tanpa atribusi yang konsisten dapat menampilkan perkiraan total berkeyakinan rendah, tetapi bukan nilai yang belum dipakai. Jika epoch tumpang tindih atau atribusi bertentangan, hanya ekuivalen terpakai yang ditampilkan; pemisahan akun tidak direka.', boundaryApproximation: 'Penggunaan Codex diagregasi per hari. Jika reset resmi terjadi di tengah hari yang tercatat, periode terdampak dapat menampilkan perkiraan batas total dengan keyakinan rendah; sisa saat ini tetap disembunyikan.', boundaryApproximate: 'Perkiraan batas', indexedSubtotal: 'Pengindeksan belum selesai; nilai mingguan masih berupa subtotal konservatif.', diff --git a/src/quotaObservationStore.ts b/src/quotaObservationStore.ts new file mode 100644 index 00000000..b261e416 --- /dev/null +++ b/src/quotaObservationStore.ts @@ -0,0 +1,1006 @@ +import { createHmac, randomBytes } from 'node:crypto'; +import { + mkdir, + open, + readFile, + rename, + unlink, +} from 'node:fs/promises'; +import * as path from 'node:path'; + +import type { UsageProvider } from './providers/providerTypes'; +import { acquireCodexIndexLease } from './providers/codex/codexIndexLease'; +import type { ClaudeApiUsageResponse } from './types'; +import { normalizeQuotaWindows } from './quotaWindows'; +import type { WeeklyQuotaObservation } from './weeklyValue'; + +export const QUOTA_OBSERVATION_SCHEMA_VERSION = 2 as const; +export const QUOTA_OBSERVATION_FILE = 'quota-observations-v2.json'; +export const QUOTA_OBSERVATION_RETENTION_DAYS = 180; +export const QUOTA_OBSERVATION_LIMIT_PER_SERIES = 512; + +const DAY_MS = 24 * 60 * 60 * 1000; +const FUTURE_SKEW_MS = DAY_MS; +const RESET_CLUSTER_MS = 5 * 60 * 1000; +const USAGE_DROP_THRESHOLD = 0.15; + +export type QuotaPeriodType = + | 'five-hour' + | 'seven-day' + | 'provider-scoped-seven-day'; +export type QuotaAccountAttribution = + | 'verified-local-signal' + | 'profile-continuity' + | 'unattributed'; +export type QuotaObservationSource = + | 'claude-official-api' + | 'codex-local-structured-event'; +export type QuotaEvidenceConfidence = 'high' | 'medium' | 'low'; +export type QuotaCaptureReason = + | 'refresh' + | 'reset-at-change' + | 'window-id-change' + | 'usage-drop' + | 'profile-change' + | 'migration'; +export type QuotaObservationFlag = + | 'approximate-boundary' + | 'low-log-coverage' + | 'low-price-coverage' + | 'account-ambiguous' + | 'clock-anomaly'; +export type QuotaResetEvidence = + | 'reset-at-change' + | 'window-id-change' + | 'usage-drop'; + +/** Runtime-only input. Raw identity/window signals are HMACed and discarded. */ +export interface QuotaCapture { + provider: UsageProvider; + stableIdentitySignal?: string; + /** Caller-owned random epoch nonce for evidence without a safe account + * identity. It is HMACed immediately and never persisted verbatim. */ + unattributedEpochSignal?: string; + accountAttribution: QuotaAccountAttribution; + observedAt: number; + periodType: QuotaPeriodType; + usedFraction: number; + resetAt: number | null; + source: QuotaObservationSource; + /** A provider-supplied window identity only when its semantics are known. + * It is HMACed before persistence. */ + sourceWindowId?: string; + confidence: QuotaEvidenceConfidence; + captureReason: QuotaCaptureReason; + flags?: QuotaObservationFlag[]; +} + +export interface QuotaObservationV2 { + schemaVersion: 2; + provider: UsageProvider; + accountFingerprint: string; + accountAttribution: QuotaAccountAttribution; + observedAt: number; + periodType: QuotaPeriodType; + usedFraction: number; + remainingFraction: number; + resetAt: number | null; + source: QuotaObservationSource; + windowId: string; + providerWindowFingerprint: string | null; + confidence: QuotaEvidenceConfidence; + captureReason: QuotaCaptureReason; + flags: QuotaObservationFlag[]; +} + +export interface QuotaResetEventV2 { + schemaVersion: 2; + provider: UsageProvider; + periodType: QuotaPeriodType; + detectedAt: number; + boundaryAt: number; + previousAccountFingerprint: string; + nextAccountFingerprint: string; + previousWindowId: string; + nextWindowId: string; + evidence: QuotaResetEvidence; + confidence: QuotaEvidenceConfidence; + flags: QuotaObservationFlag[]; +} + +export interface QuotaObservationStoreV2 { + schemaVersion: 2; + fingerprintAlgorithm: 'hmac-sha256-v1'; + observations: QuotaObservationV2[]; +} + +export interface MergeQuotaCaptureOptions { + salt: string; + now?: number; + retentionDays?: number; + limitPerSeries?: number; + usageDropThreshold?: number; +} + +export interface QuotaObservationLoadResult { + store: QuotaObservationStoreV2; + disposition: 'missing' | 'valid' | 'quarantined'; + quarantinePath?: string; +} + +const PERIOD_TYPES = new Set([ + 'five-hour', + 'seven-day', + 'provider-scoped-seven-day', +]); +const ATTRIBUTIONS = new Set([ + 'verified-local-signal', + 'profile-continuity', + 'unattributed', +]); +const SOURCES = new Set([ + 'claude-official-api', + 'codex-local-structured-event', +]); +const CONFIDENCES = new Set(['high', 'medium', 'low']); +const CAPTURE_REASONS = new Set([ + 'refresh', + 'reset-at-change', + 'window-id-change', + 'usage-drop', + 'profile-change', + 'migration', +]); +const FLAGS = new Set([ + 'approximate-boundary', + 'low-log-coverage', + 'low-price-coverage', + 'account-ambiguous', + 'clock-anomaly', +]); +const RESET_EVIDENCE = new Set([ + 'reset-at-change', + 'window-id-change', + 'usage-drop', +]); + +function hmac(salt: string, value: string, prefix: string): string { + return `${prefix}_${createHmac('sha256', salt).update(value).digest('hex').slice(0, 32)}`; +} + +export function fingerprintForStableIdentity( + salt: string, + provider: UsageProvider, + stableIdentitySignal: string, +): string { + return hmac( + salt, + `account|v1|${provider}|${stableIdentitySignal}`, + 'acct', + ); +} + +function providerWindowFingerprint( + salt: string, + capture: QuotaCapture, +): string | null { + const raw = capture.sourceWindowId?.trim(); + return raw + ? hmac(salt, `provider-window|${capture.provider}|${capture.periodType}|${raw}`, 'pwin') + : null; +} + +function observationIdentity(item: QuotaObservationV2): string { + return [ + item.provider, + item.accountFingerprint, + item.periodType, + item.windowId, + item.observedAt, + item.usedFraction, + item.resetAt ?? '', + item.source, + ].join('|'); +} + +function orderedUniqueFlags(flags: readonly QuotaObservationFlag[] = []): QuotaObservationFlag[] { + return [...new Set(flags.filter((flag) => FLAGS.has(flag)))].sort(); +} + +export function createEmptyQuotaObservationStore(): QuotaObservationStoreV2 { + return { + schemaVersion: QUOTA_OBSERVATION_SCHEMA_VERSION, + fingerprintAlgorithm: 'hmac-sha256-v1', + observations: [], + }; +} + +function validCapture(capture: QuotaCapture, now: number): boolean { + return ( + (capture.provider === 'claude' || capture.provider === 'codex') && + ATTRIBUTIONS.has(capture.accountAttribution) && + PERIOD_TYPES.has(capture.periodType) && + SOURCES.has(capture.source) && + CONFIDENCES.has(capture.confidence) && + CAPTURE_REASONS.has(capture.captureReason) && + Number.isFinite(capture.observedAt) && + capture.observedAt > 0 && + capture.observedAt <= now + FUTURE_SKEW_MS && + Number.isFinite(capture.usedFraction) && + capture.usedFraction >= 0 && + capture.usedFraction <= 1 && + ( + capture.resetAt === null || + (Number.isFinite(capture.resetAt) && capture.resetAt > 0) + ) && + ( + capture.accountAttribution === 'unattributed' + ? typeof capture.unattributedEpochSignal === 'string' && + capture.unattributedEpochSignal.length > 0 + : + typeof capture.stableIdentitySignal === 'string' && + capture.stableIdentitySignal.length > 0 + ) + ); +} + +function latestBefore( + observations: readonly QuotaObservationV2[], + capture: QuotaCapture, + stableFingerprint: string, +): QuotaObservationV2 | undefined { + const eligible = observations + .filter((item) => + item.provider === capture.provider && + item.periodType === capture.periodType && + item.observedAt <= capture.observedAt && + item.accountFingerprint === stableFingerprint, + ) + .sort((left, right) => + left.observedAt - right.observedAt || + left.windowId.localeCompare(right.windowId), + ); + return eligible[eligible.length - 1]; +} + +function latestProviderPeriodBefore( + observations: readonly QuotaObservationV2[], + capture: QuotaCapture, +): QuotaObservationV2 | undefined { + const eligible = observations + .filter((item) => + item.provider === capture.provider && + item.periodType === capture.periodType && + item.observedAt <= capture.observedAt, + ) + .sort((left, right) => + left.observedAt - right.observedAt || + left.windowId.localeCompare(right.windowId), + ); + return eligible[eligible.length - 1]; +} + +function boundaryEvidence( + previous: QuotaObservationV2 | undefined, + capture: QuotaCapture, + currentProviderWindowFingerprint: string | null, + usageDropThreshold: number, +): QuotaResetEvidence | null { + if (!previous) return null; + if ( + previous.providerWindowFingerprint !== null && + currentProviderWindowFingerprint !== null && + previous.providerWindowFingerprint !== currentProviderWindowFingerprint + ) { + return 'window-id-change'; + } + if ( + previous.resetAt !== null && + capture.resetAt !== null && + Math.abs(previous.resetAt - capture.resetAt) > RESET_CLUSTER_MS + ) { + return 'reset-at-change'; + } + if ( + previous.usedFraction - capture.usedFraction >= usageDropThreshold + ) { + return 'usage-drop'; + } + return null; +} + +function confidenceRank(value: QuotaEvidenceConfidence): number { + return value === 'high' ? 3 : value === 'medium' ? 2 : 1; +} + +function lowerConfidence( + left: QuotaEvidenceConfidence, + right: QuotaEvidenceConfidence, +): QuotaEvidenceConfidence { + return confidenceRank(left) <= confidenceRank(right) ? left : right; +} + +function normalizedObservation( + capture: QuotaCapture, + salt: string, + previous: QuotaObservationV2 | undefined, + boundary: QuotaResetEvidence | null, + currentProviderWindowFingerprint: string | null, +): QuotaObservationV2 { + const stableFingerprint = capture.accountAttribution === 'unattributed' + ? hmac( + salt, + `unattributed-account-epoch|${capture.provider}|${capture.unattributedEpochSignal}`, + 'anon', + ) + : fingerprintForStableIdentity( + salt, + capture.provider, + capture.stableIdentitySignal as string, + ); + const keepPreviousWindow = previous !== undefined && boundary === null; + const hasStructuredWindowIdentity = + currentProviderWindowFingerprint !== null || capture.resetAt !== null; + const epochSeed = boundary === 'usage-drop' || !hasStructuredWindowIdentity + ? [ + capture.provider, + capture.periodType, + currentProviderWindowFingerprint ?? '', + capture.resetAt ?? '', + capture.observedAt, + boundary ?? 'initial', + ].join('|') + : [ + capture.provider, + capture.periodType, + currentProviderWindowFingerprint ?? '', + capture.resetAt ?? '', + ].join('|'); + const accountFingerprint = stableFingerprint; + const windowId = keepPreviousWindow + ? previous.windowId + : hmac( + salt, + `window|${accountFingerprint}|${epochSeed}`, + 'win', + ); + return { + schemaVersion: QUOTA_OBSERVATION_SCHEMA_VERSION, + provider: capture.provider, + accountFingerprint, + accountAttribution: capture.accountAttribution, + observedAt: capture.observedAt, + periodType: capture.periodType, + usedFraction: capture.usedFraction, + remainingFraction: Math.max(0, Math.min(1, 1 - capture.usedFraction)), + resetAt: capture.resetAt, + source: capture.source, + windowId, + providerWindowFingerprint: currentProviderWindowFingerprint, + confidence: capture.confidence, + captureReason: boundary ?? capture.captureReason, + flags: orderedUniqueFlags(capture.flags), + }; +} + +function compactBySeries( + observations: QuotaObservationV2[], + limit: number, +): QuotaObservationV2[] { + const groups = new Map(); + for (const item of observations) { + const key = [item.provider, item.accountFingerprint, item.periodType].join('|'); + const group = groups.get(key) ?? []; + group.push(item); + groups.set(key, group); + } + const retained: QuotaObservationV2[] = []; + for (const group of groups.values()) { + const ordered = group.sort((left, right) => + left.observedAt - right.observedAt || + left.usedFraction - right.usedFraction, + ); + if (ordered.length <= limit) { + retained.push(...ordered); + continue; + } + const mandatory = new Map(); + const windows = new Map(); + for (const item of ordered) { + const window = windows.get(item.windowId) ?? []; + window.push(item); + windows.set(item.windowId, window); + if (item.captureReason !== 'refresh' || item.flags.length > 0) { + mandatory.set(observationIdentity(item), item); + } + } + for (const window of windows.values()) { + mandatory.set(observationIdentity(window[0]), window[0]); + mandatory.set(observationIdentity(window[window.length - 1]), window[window.length - 1]); + } + const selected = [...mandatory.values()] + .sort((left, right) => right.observedAt - left.observedAt) + .slice(0, limit); + const selectedIds = new Set(selected.map(observationIdentity)); + for (const item of [...ordered].reverse()) { + if (selected.length >= limit) break; + const identity = observationIdentity(item); + if (!selectedIds.has(identity)) { + selected.push(item); + selectedIds.add(identity); + } + } + retained.push(...selected); + } + return retained.sort((left, right) => + left.observedAt - right.observedAt || + left.provider.localeCompare(right.provider) || + left.periodType.localeCompare(right.periodType) || + left.windowId.localeCompare(right.windowId), + ); +} + +function persistedBoundaryEvidence( + previous: QuotaObservationV2, + current: QuotaObservationV2, + usageDropThreshold: number, +): QuotaResetEvidence | null { + if (previous.windowId === current.windowId) return null; + if ( + previous.providerWindowFingerprint !== null && + current.providerWindowFingerprint !== null && + previous.providerWindowFingerprint !== current.providerWindowFingerprint + ) return 'window-id-change'; + if ( + previous.resetAt !== null && + current.resetAt !== null && + Math.abs(previous.resetAt - current.resetAt) > RESET_CLUSTER_MS + ) return 'reset-at-change'; + if (previous.usedFraction - current.usedFraction >= usageDropThreshold) { + return 'usage-drop'; + } + return null; +} + +function annotateBoundaries( + observations: readonly QuotaObservationV2[], + usageDropThreshold: number, +): QuotaObservationV2[] { + const ordered = [...observations].sort((left, right) => + left.observedAt - right.observedAt || + left.provider.localeCompare(right.provider) || + left.periodType.localeCompare(right.periodType) || + left.windowId.localeCompare(right.windowId), + ); + const previousBySeries = new Map(); + return ordered.map((item) => { + const key = `${item.provider}|${item.periodType}`; + const previous = previousBySeries.get(key); + const inferred = previous + ? persistedBoundaryEvidence(previous, item, usageDropThreshold) + : null; + const normalized = { + ...item, + captureReason: inferred ?? item.captureReason, + flags: orderedUniqueFlags([ + ...item.flags, + ...(previous && previous.accountFingerprint !== item.accountFingerprint + ? ['account-ambiguous' as const] + : []), + ]), + }; + previousBySeries.set(key, normalized); + return normalized; + }); +} + +export function mergeQuotaCaptures( + store: QuotaObservationStoreV2, + captures: readonly QuotaCapture[], + options: MergeQuotaCaptureOptions, +): QuotaObservationStoreV2 { + const now = options.now ?? Date.now(); + const retentionDays = Math.max( + 1, + Math.floor(options.retentionDays ?? QUOTA_OBSERVATION_RETENTION_DAYS), + ); + const limitPerSeries = Math.max( + 1, + Math.floor(options.limitPerSeries ?? QUOTA_OBSERVATION_LIMIT_PER_SERIES), + ); + const usageDropThreshold = Math.max( + 0, + Math.min(1, options.usageDropThreshold ?? USAGE_DROP_THRESHOLD), + ); + const cutoff = now - retentionDays * DAY_MS; + const observations = store.observations + .filter((item) => item.observedAt >= cutoff && item.observedAt <= now + FUTURE_SKEW_MS) + .map((item) => ({ ...item, flags: [...item.flags] })); + const observationIds = new Set(observations.map(observationIdentity)); + const orderedCaptures = [...captures] + .filter((item) => validCapture(item, now)) + .sort((left, right) => + left.observedAt - right.observedAt || + left.provider.localeCompare(right.provider) || + left.periodType.localeCompare(right.periodType), + ); + for (const capture of orderedCaptures) { + const stableFingerprint = capture.accountAttribution === 'unattributed' + ? hmac( + options.salt, + `unattributed-account-epoch|${capture.provider}|${capture.unattributedEpochSignal}`, + 'anon', + ) + : fingerprintForStableIdentity( + options.salt, + capture.provider, + capture.stableIdentitySignal as string, + ); + const previous = latestBefore(observations, capture, stableFingerprint); + // Codex does not expose a safe stable account identity. Its reset windows + // intentionally use separate unattributed epochs, but a resetAt/window + // transition still remains useful low-confidence reset evidence. Consult + // the preceding provider-period observation only for boundary detection; + // never reuse its fingerprint or combine account-level estimates. + const boundaryPrevious = previous ?? ( + capture.accountAttribution === 'unattributed' + ? latestProviderPeriodBefore(observations, capture) + : undefined + ); + const currentProviderWindowFingerprint = providerWindowFingerprint( + options.salt, + capture, + ); + const boundary = boundaryEvidence( + boundaryPrevious, + capture, + currentProviderWindowFingerprint, + usageDropThreshold, + ); + const captureWithBoundaryFlags = boundaryPrevious && + boundaryPrevious.accountFingerprint !== stableFingerprint + ? { + ...capture, + flags: orderedUniqueFlags([ + ...(capture.flags ?? []), + 'account-ambiguous', + ]), + } + : capture; + const observation = normalizedObservation( + captureWithBoundaryFlags, + options.salt, + previous, + boundary, + currentProviderWindowFingerprint, + ); + const identity = observationIdentity(observation); + if (!observationIds.has(identity)) { + observations.push(observation); + observationIds.add(identity); + } + } + return { + schemaVersion: QUOTA_OBSERVATION_SCHEMA_VERSION, + fingerprintAlgorithm: 'hmac-sha256-v1', + observations: compactBySeries( + annotateBoundaries(observations, usageDropThreshold), + limitPerSeries, + ), + }; +} + +/** Reset events are a deterministic runtime projection, not a second persisted + * fact table. Boundary observations carry the evidence needed to reconstruct + * consecutive and same-day resets without duplicating sensitive identifiers. */ +export function deriveQuotaResetEvents( + store: QuotaObservationStoreV2, +): QuotaResetEventV2[] { + const events: QuotaResetEventV2[] = []; + const ordered = [...store.observations].sort((left, right) => + left.observedAt - right.observedAt || left.windowId.localeCompare(right.windowId), + ); + const previousBySeries = new Map(); + for (let index = 0; index < ordered.length; index += 1) { + const current = ordered[index]; + const seriesKey = `${current.provider}|${current.periodType}`; + const previous = previousBySeries.get(seriesKey); + previousBySeries.set(seriesKey, current); + if (!previous || previous.windowId === current.windowId) continue; + const inferred = persistedBoundaryEvidence( + previous, + current, + USAGE_DROP_THRESHOLD, + ); + const evidence = inferred ?? ( + RESET_EVIDENCE.has(current.captureReason as QuotaResetEvidence) + ? current.captureReason as QuotaResetEvidence + : null + ); + if (!evidence) continue; + const crossesAccountEpoch = + previous.accountFingerprint !== current.accountFingerprint; + events.push({ + schemaVersion: 2, + provider: current.provider, + periodType: current.periodType, + detectedAt: current.observedAt, + boundaryAt: evidence === 'usage-drop' + ? current.observedAt + : previous.resetAt ?? current.observedAt, + previousAccountFingerprint: previous.accountFingerprint, + nextAccountFingerprint: current.accountFingerprint, + previousWindowId: previous.windowId, + nextWindowId: current.windowId, + evidence, + confidence: crossesAccountEpoch + ? 'low' + : evidence === 'usage-drop' + ? lowerConfidence(current.confidence, 'medium') + : current.confidence, + flags: orderedUniqueFlags([ + ...current.flags, + ...(crossesAccountEpoch ? ['account-ambiguous' as const] : []), + ]), + }); + } + return events; +} + +function exactKeys(value: Record, allowed: readonly string[]): boolean { + const allowedSet = new Set(allowed); + return Object.keys(value).every((key) => allowedSet.has(key)); +} + +function record(value: unknown): Record | null { + return typeof value === 'object' && value !== null && !Array.isArray(value) + ? value as Record + : null; +} + +function finiteFraction(value: unknown): value is number { + return typeof value === 'number' && Number.isFinite(value) && value >= 0 && value <= 1; +} + +function finitePositive(value: unknown): value is number { + return typeof value === 'number' && Number.isFinite(value) && value > 0; +} + +function validFingerprint(value: unknown): value is string { + return typeof value === 'string' && /^(?:acct|anon)_[a-f0-9]{32}$/.test(value); +} + +function validWindowId(value: unknown): value is string { + return typeof value === 'string' && /^win_[a-f0-9]{32}$/.test(value); +} + +function parseFlags(value: unknown): QuotaObservationFlag[] | null { + if (!Array.isArray(value) || !value.every((item) => typeof item === 'string' && FLAGS.has(item as QuotaObservationFlag))) { + return null; + } + return orderedUniqueFlags(value as QuotaObservationFlag[]); +} + +function parseObservation(value: unknown, now: number): QuotaObservationV2 | null { + const item = record(value); + if (!item || !exactKeys(item, [ + 'schemaVersion', 'provider', 'accountFingerprint', 'accountAttribution', + 'observedAt', 'periodType', 'usedFraction', 'remainingFraction', 'resetAt', + 'source', 'windowId', 'providerWindowFingerprint', 'confidence', + 'captureReason', 'flags', + ])) return null; + const flags = parseFlags(item.flags); + if ( + item.schemaVersion !== 2 || + (item.provider !== 'claude' && item.provider !== 'codex') || + !validFingerprint(item.accountFingerprint) || + typeof item.accountAttribution !== 'string' || + !ATTRIBUTIONS.has(item.accountAttribution as QuotaAccountAttribution) || + !finitePositive(item.observedAt) || + item.observedAt > now + FUTURE_SKEW_MS || + typeof item.periodType !== 'string' || + !PERIOD_TYPES.has(item.periodType as QuotaPeriodType) || + !finiteFraction(item.usedFraction) || + !finiteFraction(item.remainingFraction) || + Math.abs(item.usedFraction + item.remainingFraction - 1) > 1e-9 || + !(item.resetAt === null || finitePositive(item.resetAt)) || + typeof item.source !== 'string' || + !SOURCES.has(item.source as QuotaObservationSource) || + !validWindowId(item.windowId) || + !(item.providerWindowFingerprint === null || + typeof item.providerWindowFingerprint === 'string' && + /^pwin_[a-f0-9]{32}$/.test(item.providerWindowFingerprint)) || + typeof item.confidence !== 'string' || + !CONFIDENCES.has(item.confidence as QuotaEvidenceConfidence) || + typeof item.captureReason !== 'string' || + !CAPTURE_REASONS.has(item.captureReason as QuotaCaptureReason) || + flags === null + ) return null; + return { + schemaVersion: 2, + provider: item.provider, + accountFingerprint: item.accountFingerprint, + accountAttribution: item.accountAttribution as QuotaAccountAttribution, + observedAt: item.observedAt, + periodType: item.periodType as QuotaPeriodType, + usedFraction: item.usedFraction, + remainingFraction: item.remainingFraction, + resetAt: item.resetAt as number | null, + source: item.source as QuotaObservationSource, + windowId: item.windowId, + providerWindowFingerprint: item.providerWindowFingerprint as string | null, + confidence: item.confidence as QuotaEvidenceConfidence, + captureReason: item.captureReason as QuotaCaptureReason, + flags, + }; +} + +function parseStore(value: unknown, now: number): QuotaObservationStoreV2 | null { + const item = record(value); + if (!item || !exactKeys(item, [ + 'schemaVersion', 'fingerprintAlgorithm', 'observations', + ])) return null; + if ( + item.schemaVersion !== 2 || + item.fingerprintAlgorithm !== 'hmac-sha256-v1' || + !Array.isArray(item.observations) + ) return null; + const observations = item.observations.map((entry) => parseObservation(entry, now)); + if (observations.some((entry) => entry === null)) { + return null; + } + const observationMap = new Map( + (observations as QuotaObservationV2[]).map((entry) => [observationIdentity(entry), entry]), + ); + return { + schemaVersion: 2, + fingerprintAlgorithm: 'hmac-sha256-v1', + observations: [...observationMap.values()].sort((left, right) => + left.observedAt - right.observedAt || left.windowId.localeCompare(right.windowId), + ), + }; +} + +async function quarantine(filePath: string, now: number): Promise { + const quarantinePath = `${filePath}.quarantine-${Math.floor(now)}-${randomBytes(4).toString('hex')}`; + try { + await rename(filePath, quarantinePath); + return quarantinePath; + } catch { + return undefined; + } +} + +export async function loadQuotaObservationStore( + filePath: string, + options: { now?: number } = {}, +): Promise { + const now = options.now ?? Date.now(); + let body: string; + try { + body = await readFile(filePath, 'utf8'); + } catch (error) { + if ((error as NodeJS.ErrnoException).code === 'ENOENT') { + return { store: createEmptyQuotaObservationStore(), disposition: 'missing' }; + } + return { store: createEmptyQuotaObservationStore(), disposition: 'quarantined' }; + } + let parsed: unknown; + try { + parsed = JSON.parse(body) as unknown; + } catch { + const quarantinePath = await quarantine(filePath, now); + return { + store: createEmptyQuotaObservationStore(), + disposition: 'quarantined', + ...(quarantinePath ? { quarantinePath } : {}), + }; + } + const store = parseStore(parsed, now); + if (!store) { + const quarantinePath = await quarantine(filePath, now); + return { + store: createEmptyQuotaObservationStore(), + disposition: 'quarantined', + ...(quarantinePath ? { quarantinePath } : {}), + }; + } + return { store, disposition: 'valid' }; +} + +export async function saveQuotaObservationStoreAtomic( + filePath: string, + store: QuotaObservationStoreV2, +): Promise { + const sanitized = parseStore(store, Date.now()); + if (!sanitized) { + throw new Error('invalid quota observation store'); + } + const directory = path.dirname(filePath); + await mkdir(directory, { recursive: true }); + const temporaryPath = path.join( + directory, + `.${path.basename(filePath)}.${process.pid}.${randomBytes(6).toString('hex')}.tmp`, + ); + let handle: Awaited> | undefined; + try { + handle = await open(temporaryPath, 'wx', 0o600); + await handle.writeFile(JSON.stringify(sanitized), 'utf8'); + await handle.sync(); + await handle.close(); + handle = undefined; + await rename(temporaryPath, filePath); + try { + const directoryHandle = await open(directory, 'r'); + await directoryHandle.sync(); + await directoryHandle.close(); + } catch { + // Some VS Code hosts/filesystems do not permit directory fsync. The file + // itself was still flushed and installed by same-directory rename. + } + } finally { + await handle?.close().catch(() => undefined); + await unlink(temporaryPath).catch((error) => { + if ((error as NodeJS.ErrnoException).code !== 'ENOENT') throw error; + }); + } +} + +function cloneStore(store: QuotaObservationStoreV2): QuotaObservationStoreV2 { + return { + schemaVersion: 2, + fingerprintAlgorithm: 'hmac-sha256-v1', + observations: store.observations.map((item) => ({ + ...item, + flags: [...item.flags], + })), + }; +} + +export class QuotaObservationRepository { + private state: QuotaObservationStoreV2 | undefined; + private pending: Promise = Promise.resolve(); + + constructor( + private readonly filePath: string, + private readonly salt: string, + private readonly now: () => number = Date.now, + ) {} + + private serialized(operation: () => Promise): Promise { + const result = this.pending.then(operation, operation); + this.pending = result.then(() => undefined, () => undefined); + return result; + } + + load(): Promise { + return this.serialized(async () => { + this.state = (await loadQuotaObservationStore(this.filePath, { + now: this.now(), + })).store; + return cloneStore(this.state); + }); + } + + append(captures: readonly QuotaCapture[]): Promise { + return this.serialized(async () => { + const lease = await acquireCodexIndexLease(this.filePath); + try { + // Always re-read under the process-shared lease. A second VS Code + // Extension Host may have committed observations since our last call. + const current = (await loadQuotaObservationStore(this.filePath, { + now: this.now(), + })).store; + const next = mergeQuotaCaptures(current, captures, { + salt: this.salt, + now: this.now(), + }); + if (JSON.stringify(next) !== JSON.stringify(current)) { + await saveQuotaObservationStoreAtomic(this.filePath, next); + } + this.state = next; + return cloneStore(next); + } finally { + await lease.release(); + } + }); + } +} + +export function quotaStoreWeeklyObservations( + store: QuotaObservationStoreV2, + provider: UsageProvider, + accountFingerprint?: string, +): WeeklyQuotaObservation[] { + const selected = store.observations + .filter((item) => + item.provider === provider && + item.periodType === 'seven-day' && + item.resetAt !== null && + (accountFingerprint === undefined || item.accountFingerprint === accountFingerprint), + ); + const fingerprints = [...new Set(selected.map((item) => item.accountFingerprint))] + .sort(); + const aliases = new Map(fingerprints.map((fingerprint, index) => [ + fingerprint, + accountFingerprint === undefined + ? `${provider}-epoch-${index + 1}` + : `active-${provider}`, + ])); + const windows = [...new Set(selected.map((item) => item.windowId))].sort(); + const windowAliases = new Map(windows.map((windowId, index) => [ + windowId, + `${provider}-window-${index + 1}`, + ])); + return selected.map((item) => ({ + provider: item.provider, + seriesKey: aliases.get(item.accountFingerprint) as string, + observedAt: item.observedAt, + resetAt: item.resetAt as number, + usedPercent: item.usedFraction * 100, + accountAttribution: item.accountAttribution, + windowId: windowAliases.get(item.windowId) as string, + observationConfidence: item.confidence, + flags: [...item.flags], + })); +} + +export function claudeQuotaCapturesFromUsage( + usage: ClaudeApiUsageResponse | null, + identitySignal: string, + accountAttribution: Extract< + QuotaAccountAttribution, + 'verified-local-signal' | 'profile-continuity' + >, + observedAt: number, + captureReason: QuotaCaptureReason = 'refresh', +): QuotaCapture[] { + return normalizeQuotaWindows(usage).map((window): QuotaCapture => { + const resetAt = Date.parse(window.resetsAt); + return { + provider: 'claude', + stableIdentitySignal: identitySignal, + accountAttribution, + observedAt, + periodType: window.kind === 'session' + ? 'five-hour' + : window.kind === 'weekly_all' + ? 'seven-day' + : 'provider-scoped-seven-day', + usedFraction: Math.max(0, Math.min(1, window.utilization / 100)), + resetAt: Number.isFinite(resetAt) && resetAt > 0 ? resetAt : null, + source: 'claude-official-api', + // Scope labels never persist: this runtime string is HMACed into the + // provider-window fingerprint by mergeQuotaCaptures. + sourceWindowId: `${window.kind}|${window.scopeLabel ?? 'all'}`, + confidence: Number.isFinite(resetAt) && resetAt > 0 ? 'high' : 'medium', + captureReason, + flags: Number.isFinite(resetAt) && resetAt > 0 ? [] : ['approximate-boundary'], + }; + }); +} + +export function codexQuotaCapturesFromWeeklyObservations( + observations: readonly WeeklyQuotaObservation[], + captureReason: QuotaCaptureReason = 'refresh', +): QuotaCapture[] { + return observations.map((item): QuotaCapture => { + // A Codex file key identifies one local log, not an account. The reset + // window is the narrowest safe epoch we can infer without inventing an + // account split; attribution therefore remains explicitly unattributed. + const sourceEpoch = `codex-window|${item.windowId ?? item.resetAt}`; + return { + provider: 'codex', + unattributedEpochSignal: sourceEpoch, + accountAttribution: 'unattributed', + observedAt: item.observedAt, + periodType: 'seven-day', + usedFraction: Math.max(0, Math.min(1, item.usedPercent / 100)), + resetAt: item.resetAt, + source: 'codex-local-structured-event', + confidence: item.observationConfidence ?? 'medium', + captureReason, + flags: orderedUniqueFlags(item.flags), + }; + }); +} diff --git a/src/test/claudeApiClient.test.ts b/src/test/claudeApiClient.test.ts index 464a2f8d..de226e2d 100644 --- a/src/test/claudeApiClient.test.ts +++ b/src/test/claudeApiClient.test.ts @@ -27,7 +27,11 @@ function loadApiModule(): ApiModule { } } -const { ClaudeApiClient, resolveClaudeProfile } = loadApiModule(); +const { + ClaudeApiClient, + claudeQuotaIdentitySignal, + resolveClaudeProfile, +} = loadApiModule(); const tempRoots: string[] = []; after(() => { @@ -68,6 +72,43 @@ test('the default profile retains the macOS Keychain fallback', () => { assert.equal(resolved.allowKeychainFallback, true); }); +test('quota identity follows refresh-token continuity without exposing the token', () => { + const first = claudeQuotaIdentitySignal('machine-salt', 'refresh-token-a'); + const repeated = claudeQuotaIdentitySignal('machine-salt', 'refresh-token-a'); + const rotated = claudeQuotaIdentitySignal('machine-salt', 'refresh-token-b'); + + assert.equal(first, repeated); + assert.notEqual(first, rotated); + assert.equal(first.includes('refresh-token-a'), false); +}); + +test('successful quota fetch records only the safe account continuity signal', async () => { + const client = new ClaudeApiClient(null, undefined, 'machine-salt') as any; + client.getValidCredentials = async () => ({ + claudeAiOauth: { + accessToken: 'access-token', + refreshToken: 'refresh-token', + expiresAt: Date.now() + 60_000, + }, + }); + client.callUsageApi = async () => ({ + status: 200, + body: JSON.stringify({ + seven_day: { + utilization: 25, + resets_at: '2026-09-07T03:24:00.000Z', + }, + }), + }); + + assert.ok(await client.fetchUsageLimits()); + assert.equal( + client.getLastQuotaIdentitySignal(), + claudeQuotaIdentitySignal('machine-salt', 'refresh-token'), + ); + assert.equal(client.getLastQuotaIdentitySignal().includes('refresh-token'), false); +}); + test('a custom profile without a credentials file never falls back to Keychain', async () => { const profile = fs.mkdtempSync(path.join(os.tmpdir(), 'ccu-claude-profile-')); tempRoots.push(profile); diff --git a/src/test/extensionWindowActivity.test.ts b/src/test/extensionWindowActivity.test.ts index 0771d966..d1eaf450 100644 --- a/src/test/extensionWindowActivity.test.ts +++ b/src/test/extensionWindowActivity.test.ts @@ -15,6 +15,10 @@ import { } from '../backgroundWorkState'; import { ResourceOwnershipRegistry } from '../resourceOwnership'; import { snapshotFixture } from './codexFixtures'; +import { + createEmptyQuotaObservationStore, + mergeQuotaCaptures, +} from '../quotaObservationStore'; type ExtensionModule = typeof import('../extension'); @@ -41,10 +45,14 @@ function loadExtensionModule(): ExtensionModule { } } -const { ClaudeCodeUsageExtension } = loadExtensionModule(); +const { + ClaudeCodeUsageExtension, + initializeQuotaObservationRuntime, +} = loadExtensionModule(); function bareExtension(): any { const extension = Object.create(ClaudeCodeUsageExtension.prototype) as any; + const quotaSalt = 'extension-window-activity-test-salt'; extension.resourceOwnership = new ResourceOwnershipRegistry(); extension.codexWatcherLeases = new Map(); extension.debounceTimerLeases = new Map(); @@ -74,6 +82,22 @@ function bareExtension(): any { }, }; extension.outputChannel = { appendLine: () => undefined }; + extension.quotaFingerprintSalt = quotaSalt; + extension.quotaObservationStore = createEmptyQuotaObservationStore(); + extension.quotaObservationRepository = { + append: async (captures: any[]) => { + extension.quotaObservationStore = mergeQuotaCaptures( + extension.quotaObservationStore, + captures, + { salt: quotaSalt, now: Date.now() }, + ); + return extension.quotaObservationStore; + }, + }; + extension.webviewProvider = { + updateQuota: () => undefined, + updateWeeklyQuotaHistory: () => undefined, + }; return extension; } @@ -1214,6 +1238,9 @@ test('credentials change clears quota failure backoff before refreshing', async calls.push(`refresh:${trigger}`); return Promise.resolve(); }; + extension.webviewProvider = { + updateWeeklyQuotaHistory: () => undefined, + }; extension.handleCredentialsChange(); await Promise.resolve(); @@ -1297,6 +1324,63 @@ test('a quota response from the previous profile is discarded after a switch', a assert.equal(extension.cache.usageLimitsLastUpdate.getTime(), 0); }); +test('a same-path credential rotation cannot persist the stale in-flight quota response', async () => { + const extension = bareExtension(); + let releaseRequest: ((value: unknown) => void) | undefined; + const client = { + getCredentialsPath: () => '/private/profile/.credentials.json', + getLastQuotaIdentitySignal: () => 'stale-continuity-signal', + fetchUsageLimits: () => new Promise((resolve) => { + releaseRequest = resolve; + }), + }; + extension.apiClient = client; + extension.claudeProfileGeneration = 0; + extension.cache = { + usageLimits: null, + usageLimitsLastUpdate: new Date(0), + usageLimitsBackoffUntil: new Date(0), + usageLimitsFailStreak: 0, + }; + extension.isActive = () => false; + extension.refreshData = async () => undefined; + + const pending = extension.maybeFetchUsageLimits({ usageLimitTracking: true }); + extension.handleCredentialsChange(); + releaseRequest?.({ + limits: [{ + kind: 'weekly_all', + group: 'weekly', + percent: 75, + resets_at: '2026-09-07T03:24:00.000Z', + scope: null, + is_active: true, + }], + }); + + assert.equal(await pending, null); + assert.equal(extension.quotaObservationStore.observations.length, 0); + assert.equal(extension.cache.usageLimits, null); +}); + +test('disabled quota tracking neither calls the provider nor records an observation', async () => { + const extension = bareExtension(); + let fetches = 0; + extension.apiClient = { + fetchUsageLimits: async () => { + fetches += 1; + return null; + }, + }; + + assert.equal( + await extension.maybeFetchUsageLimits({ usageLimitTracking: false }), + null, + ); + assert.equal(fetches, 0); + assert.equal(extension.quotaObservationStore.observations.length, 0); +}); + test('repeated quota failures use the one-hour backoff cap', async () => { const extension = bareExtension(); const originalNow = Date.now; @@ -1333,6 +1417,7 @@ test('a successful Claude quota fetch persists sanitized weekly observations per const historyUpdates: unknown[] = []; extension.apiClient = { getCredentialsPath: () => '/private/profile-a/.credentials.json', + getLastQuotaIdentitySignal: () => 'safe-local-continuity-signal', fetchUsageLimits: async () => ({ limits: [{ kind: 'weekly_all', @@ -1368,16 +1453,140 @@ test('a successful Claude quota fetch persists sanitized weekly observations per try { const result = await extension.maybeFetchUsageLimits({ usageLimitTracking: true }); assert.ok(result); - const historyWrite = writes.find((write) => write.key.startsWith('ccu.weeklyQuotaHistory.v1.')); - assert.ok(historyWrite); - const history = historyWrite.value as Array>; - assert.equal(history.length, 1); - assert.deepEqual(Object.keys(history[0]).sort(), [ - 'observedAt', 'provider', 'resetAt', 'seriesKey', 'usedPercent', - ]); - assert.equal(history[0].usedPercent, 40); + assert.equal( + writes.some((write) => write.key.startsWith('ccu.weeklyQuotaHistory.v1.')), + false, + ); + const observations = extension.quotaObservationStore.observations; + assert.equal(observations.length, 1); + assert.equal(observations[0].provider, 'claude'); + assert.equal(observations[0].usedFraction, 0.4); + assert.equal(observations[0].accountAttribution, 'verified-local-signal'); + assert.match(observations[0].accountFingerprint, /^acct_[a-f0-9]{32}$/); + assert.equal( + JSON.stringify(extension.quotaObservationStore).includes('/private/profile-a'), + false, + ); + assert.equal( + JSON.stringify(extension.quotaObservationStore).includes('safe-local-continuity-signal'), + false, + ); assert.equal(historyUpdates.length, 1); } finally { Date.now = originalNow; } }); + +test('legacy quota migration is atomic, private, idempotent, and independent of tracking state', async () => { + const root = await fs.promises.mkdtemp(path.join(os.tmpdir(), 'ccu-quota-migration-')); + const observedAt = Date.now() - 60_000; + const resetAt = Date.now() + 7 * 24 * 60 * 60 * 1000; + const state = new Map([ + ['ccu.usageLimits.legacy-profile', { + ts: observedAt, + data: { + limits: [{ + kind: 'weekly_all', + group: 'weekly', + percent: 40, + resets_at: new Date(resetAt).toISOString(), + scope: null, + is_active: true, + }], + unsafeCredentialCanary: 'oauth-token-canary', + }, + }], + ['ccu.weeklyQuotaHistory.v1.legacy-profile', [{ + provider: 'claude', + seriesKey: '/private/legacy-profile', + observedAt, + resetAt, + usedPercent: 40, + }]], + ]); + const globalState = { + keys: () => [...state.keys()], + get: (key: string): T | undefined => state.get(key) as T | undefined, + update: async (key: string, value: unknown) => { + if (value === undefined) state.delete(key); + else state.set(key, value); + }, + }; + const context = { + globalState, + globalStorageUri: { fsPath: root }, + } as any; + const settings = { + get: (key: string) => key === 'dataDirectory' + ? path.join(root, 'profile-without-credentials') + : key === 'timezone' + ? 'UTC' + : key === 'usageLimitTracking' + ? false + : undefined, + } as any; + + const first = await initializeQuotaObservationRuntime(context, settings); + assert.equal(first.store.observations.length, 1); + assert.equal(first.store.observations[0].usedFraction, 0.4); + assert.equal(first.store.observations[0].captureReason, 'migration'); + assert.ok(first.store.observations[0].flags.includes('account-ambiguous')); + assert.equal(state.has('ccu.usageLimits.legacy-profile'), false); + assert.equal(state.has('ccu.weeklyQuotaHistory.v1.legacy-profile'), false); + assert.equal(state.get('ccu.quota.migratedCodexIndex.v2'), true); + + const file = path.join(root, 'quota-observations-v2.json'); + const persisted = await fs.promises.readFile(file, 'utf8'); + assert.doesNotMatch(persisted, /oauth-token-canary|private\/legacy-profile/); + const second = await initializeQuotaObservationRuntime(context, settings); + assert.deepEqual(second.store, first.store); +}); + +test('a failed P2 migration write preserves every legacy quota source and retry marker', async () => { + const root = await fs.promises.mkdtemp(path.join(os.tmpdir(), 'ccu-quota-migration-failure-')); + const blockedStorage = path.join(root, 'not-a-directory'); + await fs.promises.writeFile(blockedStorage, 'fixture', 'utf8'); + const observedAt = Date.now() - 60_000; + const state = new Map([[ + 'ccu.usageLimits.retryable-profile', + { + ts: observedAt, + data: { + limits: [{ + kind: 'weekly_all', + group: 'weekly', + percent: 25, + resets_at: new Date(Date.now() + 7 * 24 * 60 * 60 * 1000).toISOString(), + scope: null, + is_active: true, + }], + }, + }, + ]]); + const context = { + globalState: { + keys: () => [...state.keys()], + get: (key: string): T | undefined => state.get(key) as T | undefined, + update: async (key: string, value: unknown) => { + if (value === undefined) state.delete(key); + else state.set(key, value); + }, + }, + globalStorageUri: { fsPath: blockedStorage }, + } as any; + const settings = { + get: (key: string) => key === 'timezone' ? 'UTC' : '', + } as any; + + await assert.rejects( + initializeQuotaObservationRuntime(context, settings), + (error: unknown) => + typeof error === 'object' && + error !== null && + 'code' in error && + (error as NodeJS.ErrnoException).code === 'EEXIST' && + String((error as NodeJS.ErrnoException).path).endsWith('not-a-directory'), + ); + assert.equal(state.has('ccu.usageLimits.retryable-profile'), true); + assert.equal(state.has('ccu.quota.migratedCodexIndex.v2'), false); +}); diff --git a/src/test/quotaObservationStore.test.ts b/src/test/quotaObservationStore.test.ts new file mode 100644 index 00000000..55ee9734 --- /dev/null +++ b/src/test/quotaObservationStore.test.ts @@ -0,0 +1,368 @@ +import { test } from 'node:test'; +import * as assert from 'node:assert/strict'; +import { access, mkdir, readFile, writeFile } from 'node:fs/promises'; +import * as os from 'node:os'; +import * as path from 'node:path'; + +import { + createEmptyQuotaObservationStore, + deriveQuotaResetEvents, + fingerprintForStableIdentity, + loadQuotaObservationStore, + mergeQuotaCaptures, + QuotaCapture, + QuotaObservationRepository, + quotaStoreWeeklyObservations, + saveQuotaObservationStoreAtomic, +} from '../quotaObservationStore'; + +const HOUR = 60 * 60 * 1000; +const DAY = 24 * HOUR; +const NOW = Date.parse('2026-09-02T12:00:00.000Z'); +const RESET = Date.parse('2026-09-07T03:24:00.000Z'); +const SALT = '00112233445566778899aabbccddeeff'; + +function capture(overrides: Partial = {}): QuotaCapture { + return { + provider: 'codex', + unattributedEpochSignal: 'fixture-codex-epoch', + accountAttribution: 'unattributed', + observedAt: NOW, + periodType: 'seven-day', + usedFraction: 0.3, + resetAt: RESET, + source: 'codex-local-structured-event', + confidence: 'medium', + captureReason: 'refresh', + ...overrides, + }; +} + +test('stable fingerprints are machine-local and never retain the raw identity signal', () => { + const first = fingerprintForStableIdentity(SALT, 'claude', '/private/profile-a'); + const repeated = fingerprintForStableIdentity(SALT, 'claude', '/private/profile-a'); + const other = fingerprintForStableIdentity(SALT, 'claude', '/private/profile-b'); + + assert.equal(first, repeated); + assert.notEqual(first, other); + assert.equal(first.includes('/private/profile-a'), false); + + const merged = mergeQuotaCaptures(createEmptyQuotaObservationStore(), [capture({ + provider: 'claude', + stableIdentitySignal: '/private/profile-a', + accountAttribution: 'profile-continuity', + source: 'claude-official-api', + sourceWindowId: 'private-provider-window-canary', + })], { salt: SALT, now: NOW }); + const serialized = JSON.stringify(merged); + assert.equal(serialized.includes('/private/profile-a'), false); + assert.equal(serialized.includes('private-provider-window-canary'), false); + assert.equal(merged.observations[0].accountFingerprint, first); + assert.match( + merged.observations[0].providerWindowFingerprint ?? '', + /^pwin_[a-f0-9]{32}$/, + ); +}); + +test('exact duplicate polls are idempotent while changed utilization is retained', () => { + const first = capture({ observedAt: NOW - HOUR, usedFraction: 0.2 }); + let store = mergeQuotaCaptures(createEmptyQuotaObservationStore(), [first, first], { + salt: SALT, + now: NOW, + }); + store = mergeQuotaCaptures(store, [ + first, + capture({ observedAt: NOW, usedFraction: 0.3 }), + ], { salt: SALT, now: NOW }); + + assert.equal(store.observations.length, 2); + assert.deepEqual(store.observations.map((item) => item.usedFraction), [0.2, 0.3]); + assert.equal(new Set(store.observations.map((item) => item.windowId)).size, 1); +}); + +test('structured windows remain deterministic when observations arrive out of order', () => { + const early = capture({ observedAt: NOW - HOUR, usedFraction: 0.2 }); + const late = capture({ observedAt: NOW, usedFraction: 0.3 }); + const ordered = mergeQuotaCaptures(createEmptyQuotaObservationStore(), [early, late], { + salt: SALT, + now: NOW, + }); + let reversed = mergeQuotaCaptures(createEmptyQuotaObservationStore(), [late], { + salt: SALT, + now: NOW, + }); + reversed = mergeQuotaCaptures(reversed, [early], { salt: SALT, now: NOW }); + + assert.deepEqual(reversed, ordered); + assert.equal(new Set(ordered.observations.map((item) => item.windowId)).size, 1); +}); + +test('reset projections remain deterministic when the post-reset fact arrives first', () => { + const before = capture({ + observedAt: NOW - HOUR, + resetAt: NOW + HOUR, + usedFraction: 0.9, + unattributedEpochSignal: 'before-reset-epoch', + }); + const after = capture({ + observedAt: NOW, + resetAt: NOW + 8 * HOUR, + usedFraction: 0.05, + unattributedEpochSignal: 'after-reset-epoch', + }); + const ordered = mergeQuotaCaptures(createEmptyQuotaObservationStore(), [before, after], { + salt: SALT, + now: NOW, + }); + let reversed = mergeQuotaCaptures(createEmptyQuotaObservationStore(), [after], { + salt: SALT, + now: NOW, + }); + reversed = mergeQuotaCaptures(reversed, [before], { salt: SALT, now: NOW }); + + assert.deepEqual(reversed, ordered); + assert.deepEqual(deriveQuotaResetEvents(reversed), deriveQuotaResetEvents(ordered)); + assert.equal(deriveQuotaResetEvents(reversed)[0].evidence, 'reset-at-change'); +}); + +test('resetAt changes retain consecutive and same-day reset events', () => { + const firstReset = NOW + 2 * HOUR; + const secondReset = NOW + 8 * HOUR; + const thirdReset = NOW + 14 * HOUR; + const store = mergeQuotaCaptures(createEmptyQuotaObservationStore(), [ + capture({ observedAt: NOW - HOUR, resetAt: firstReset, usedFraction: 0.92 }), + capture({ observedAt: NOW, resetAt: secondReset, usedFraction: 0.03 }), + capture({ observedAt: NOW + HOUR, resetAt: thirdReset, usedFraction: 0.02 }), + ], { salt: SALT, now: NOW + 2 * HOUR }); + + const resetEvents = deriveQuotaResetEvents(store); + assert.equal(resetEvents.length, 2); + assert.deepEqual(resetEvents.map((event) => event.evidence), [ + 'reset-at-change', + 'reset-at-change', + ]); + assert.equal(new Set(store.observations.map((item) => item.windowId)).size, 3); + assert.equal(new Set(store.observations.map((item) => item.accountFingerprint)).size, 1); +}); + +test('unattributed reset epochs expose boundary evidence without merging account epochs', () => { + const store = mergeQuotaCaptures(createEmptyQuotaObservationStore(), [ + capture({ + observedAt: NOW - HOUR, + resetAt: NOW + HOUR, + usedFraction: 0.91, + unattributedEpochSignal: 'first-unattributed-window', + }), + capture({ + observedAt: NOW, + resetAt: NOW + 8 * HOUR, + usedFraction: 0.04, + unattributedEpochSignal: 'second-unattributed-window', + }), + ], { salt: SALT, now: NOW }); + + assert.equal(new Set(store.observations.map((item) => item.accountFingerprint)).size, 2); + assert.equal(new Set(store.observations.map((item) => item.windowId)).size, 2); + assert.ok(store.observations[1].flags.includes('account-ambiguous')); + const events = deriveQuotaResetEvents(store); + assert.equal(events.length, 1); + assert.equal(events[0].evidence, 'reset-at-change'); + assert.notEqual(events[0].previousAccountFingerprint, events[0].nextAccountFingerprint); +}); + +test('small percentage rollback is noise but a significant rollback creates a new window', () => { + const noisy = mergeQuotaCaptures(createEmptyQuotaObservationStore(), [ + capture({ observedAt: NOW - 2 * HOUR, usedFraction: 0.5 }), + capture({ observedAt: NOW - HOUR, usedFraction: 0.48 }), + ], { salt: SALT, now: NOW }); + assert.equal(deriveQuotaResetEvents(noisy).length, 0); + assert.equal(new Set(noisy.observations.map((item) => item.windowId)).size, 1); + + const reset = mergeQuotaCaptures(noisy, [ + capture({ observedAt: NOW, usedFraction: 0.08 }), + ], { salt: SALT, now: NOW }); + const resetEvents = deriveQuotaResetEvents(reset); + assert.equal(resetEvents.length, 1); + assert.equal(resetEvents[0].evidence, 'usage-drop'); + assert.notEqual(reset.observations[1].windowId, reset.observations[2].windowId); +}); + +test('different stable account signals never share a fingerprint or window', () => { + const store = mergeQuotaCaptures(createEmptyQuotaObservationStore(), [ + capture({ + provider: 'claude', + stableIdentitySignal: '/profile/a', + accountAttribution: 'profile-continuity', + source: 'claude-official-api', + }), + capture({ + provider: 'claude', + stableIdentitySignal: '/profile/b', + accountAttribution: 'profile-continuity', + source: 'claude-official-api', + }), + ], { salt: SALT, now: NOW }); + + assert.equal(new Set(store.observations.map((item) => item.accountFingerprint)).size, 2); + assert.equal(new Set(store.observations.map((item) => item.windowId)).size, 2); +}); + +test('an account epoch change alone does not invent an official reset event', () => { + const store = mergeQuotaCaptures(createEmptyQuotaObservationStore(), [ + capture({ + provider: 'claude', + stableIdentitySignal: '/profile/a', + accountAttribution: 'profile-continuity', + source: 'claude-official-api', + observedAt: NOW - HOUR, + usedFraction: 0.2, + }), + capture({ + provider: 'claude', + stableIdentitySignal: '/profile/b', + accountAttribution: 'profile-continuity', + source: 'claude-official-api', + observedAt: NOW, + usedFraction: 0.3, + }), + ], { salt: SALT, now: NOW }); + + assert.equal(new Set(store.observations.map((item) => item.accountFingerprint)).size, 2); + assert.equal(new Set(store.observations.map((item) => item.windowId)).size, 2); + assert.deepEqual(deriveQuotaResetEvents(store), []); +}); + +test('invalid and far-future captures are rejected without manufacturing values', () => { + const store = mergeQuotaCaptures(createEmptyQuotaObservationStore(), [ + capture({ usedFraction: Number.NaN }), + capture({ usedFraction: 1.01 }), + capture({ observedAt: NOW + 2 * DAY }), + capture({ resetAt: null, usedFraction: 0 }), + ], { salt: SALT, now: NOW }); + + assert.equal(store.observations.length, 1); + assert.equal(store.observations[0].usedFraction, 0); + assert.equal(store.observations[0].remainingFraction, 1); + assert.equal(store.observations[0].resetAt, null); +}); + +test('weekly projection exposes only seven-day facts and retains attribution metadata', () => { + const store = mergeQuotaCaptures(createEmptyQuotaObservationStore(), [ + capture({ periodType: 'five-hour', usedFraction: 0.4 }), + capture({ periodType: 'seven-day', usedFraction: 0.3 }), + ], { salt: SALT, now: NOW }); + const rows = quotaStoreWeeklyObservations(store, 'codex'); + + assert.equal(rows.length, 1); + assert.equal(rows[0].usedPercent, 30); + assert.equal(rows[0].accountAttribution, 'unattributed'); + assert.equal(rows[0].seriesKey, 'codex-epoch-1'); + assert.notEqual(rows[0].seriesKey, store.observations[1].accountFingerprint); + assert.equal(rows[0].windowId, 'codex-window-1'); + assert.doesNotMatch(JSON.stringify(rows), /(?:acct|anon|win|pwin)_[a-f0-9]{32}/); +}); + +test('retention and compaction preserve a live window first and last observation', () => { + const additions = Array.from({ length: 10 }, (_unused, index) => capture({ + observedAt: NOW - (9 - index) * HOUR, + usedFraction: 0.1 + index * 0.01, + })); + const compacted = mergeQuotaCaptures(createEmptyQuotaObservationStore(), additions, { + salt: SALT, + now: NOW, + limitPerSeries: 4, + }); + assert.equal(compacted.observations.length, 4); + assert.ok(compacted.observations.some((item) => item.observedAt === NOW - 9 * HOUR)); + assert.ok(compacted.observations.some((item) => item.observedAt === NOW)); + + const oldNow = NOW - 190 * DAY; + const old = mergeQuotaCaptures(createEmptyQuotaObservationStore(), [capture({ + observedAt: oldNow, + resetAt: oldNow + DAY, + })], { salt: SALT, now: oldNow }); + const retained = mergeQuotaCaptures(old, [], { salt: SALT, now: NOW }); + assert.equal(retained.observations.length, 0); +}); + +test('quota store writes atomically and reloads a strict schema-2 document', async () => { + const root = path.join(os.tmpdir(), `ccu-quota-store-${process.pid}-${Math.random().toString(16).slice(2)}`); + await mkdir(root, { recursive: true }); + const file = path.join(root, 'quota-observations-v2.json'); + const store = mergeQuotaCaptures(createEmptyQuotaObservationStore(), [capture()], { + salt: SALT, + now: NOW, + }); + await saveQuotaObservationStoreAtomic(file, store); + const loaded = await loadQuotaObservationStore(file, { now: NOW }); + + assert.equal(loaded.disposition, 'valid'); + assert.deepEqual(loaded.store, store); + const persisted = JSON.parse(await readFile(file, 'utf8')) as Record; + assert.equal(persisted.schemaVersion, 2); + assert.deepEqual(Object.keys(persisted).sort(), [ + 'fingerprintAlgorithm', + 'observations', + 'schemaVersion', + ]); + assert.deepEqual(Object.keys((persisted.observations as Record[])[0]).sort(), [ + 'accountAttribution', + 'accountFingerprint', + 'captureReason', + 'confidence', + 'flags', + 'observedAt', + 'periodType', + 'provider', + 'providerWindowFingerprint', + 'remainingFraction', + 'resetAt', + 'schemaVersion', + 'source', + 'usedFraction', + 'windowId', + ]); + assert.equal(root.startsWith(os.tmpdir()), true); +}); + +test('corrupt and future-schema files are quarantined without exposing their contents', async () => { + const root = path.join(os.tmpdir(), `ccu-quota-quarantine-${process.pid}-${Math.random().toString(16).slice(2)}`); + await mkdir(root, { recursive: true }); + for (const [name, body] of [ + ['corrupt', '{secret-cookie'], + ['future', JSON.stringify({ schemaVersion: 99, secret: 'token-canary' })], + ] as const) { + const file = path.join(root, `${name}.json`); + await writeFile(file, body, 'utf8'); + const loaded = await loadQuotaObservationStore(file, { now: NOW }); + assert.equal(loaded.disposition, 'quarantined'); + assert.equal(loaded.store.observations.length, 0); + assert.ok(loaded.quarantinePath); + assert.equal(path.basename(loaded.quarantinePath ?? '').includes('secret'), false); + await access(loaded.quarantinePath as string); + } +}); + +test('repository serializes cross-instance concurrent appends without losing either observation', async () => { + const root = path.join(os.tmpdir(), `ccu-quota-repository-${process.pid}-${Math.random().toString(16).slice(2)}`); + await mkdir(root, { recursive: true }); + const file = path.join(root, 'quota-observations-v2.json'); + const repository = new QuotaObservationRepository( + file, + SALT, + () => NOW, + ); + const secondRepository = new QuotaObservationRepository( + file, + SALT, + () => NOW, + ); + await Promise.all([ + repository.append([capture({ observedAt: NOW - HOUR, usedFraction: 0.2 })]), + secondRepository.append([capture({ observedAt: NOW, usedFraction: 0.3 })]), + ]); + const loaded = await repository.load(); + + assert.equal(loaded.observations.length, 2); + assert.deepEqual(loaded.observations.map((item) => item.usedFraction), [0.2, 0.3]); +}); diff --git a/src/test/weeklyValue.test.ts b/src/test/weeklyValue.test.ts index a1eaf992..a9ca8fe1 100644 --- a/src/test/weeklyValue.test.ts +++ b/src/test/weeklyValue.test.ts @@ -74,6 +74,21 @@ test('current window is provisional and never presents unused allowance as final assert.equal(points[0].unusedEquivalentUsd, null); }); +test('legacy trend never claims unused value for explicit unattributed or ambiguous evidence', () => { + for (const overrides of [ + { accountAttribution: 'unattributed' as const }, + { flags: ['account-ambiguous' as const] }, + { flags: ['approximate-boundary' as const] }, + ]) { + const points = buildWeeklyValueTrend({ + observations: [observation(overrides)], + usage: [usage(45, RESET - 2 * HOUR)], + }, RESET + HOUR); + assert.equal(points[0].fullEquivalentUsd, 60); + assert.equal(points[0].unusedEquivalentUsd, null); + } +}); + test('usage observed after a stale quota sample cannot exceed the displayed full allowance', () => { const points = buildWeeklyValueTrend({ observations: [observation({ usedPercent: 50 })], @@ -89,13 +104,13 @@ test('usage observed after a stale quota sample cannot exceed the displayed full assert.equal(points[0].confidence, 'low'); }); -test('tiny utilization and poor price coverage do not manufacture a full allowance value', () => { +test('valid tiny utilization and partial pricing still produce a low-confidence total estimate', () => { const lowUtilization = buildWeeklyValueTrend({ observations: [observation({ usedPercent: 2 })], usage: [usage(20, RESET - 2 * HOUR)], }, RESET + HOUR); - assert.equal(lowUtilization[0].fullEquivalentUsd, null); - assert.equal(lowUtilization[0].confidence, 'usage-only'); + assert.equal(lowUtilization[0].fullEquivalentUsd, 1_000); + assert.equal(lowUtilization[0].confidence, 'low'); const partialPricing = buildWeeklyValueTrend({ observations: [observation()], @@ -106,10 +121,12 @@ test('tiny utilization and poor price coverage do not manufacture a full allowan totalTokens: 1_000, }], }, RESET + HOUR); - assert.equal(partialPricing[0].fullEquivalentUsd, null); + assert.equal(partialPricing[0].fullEquivalentUsd, 20 / 0.75); + assert.equal(partialPricing[0].unusedEquivalentUsd, null); + assert.equal(partialPricing[0].confidence, 'low'); }); -test('pricing coverage after the observation cannot qualify an underpriced prefix', () => { +test('pricing coverage after the observation cannot inflate confidence for an underpriced prefix', () => { const points = buildWeeklyValueTimeline('codex', { observations: [observation({ usedPercent: 50 })], usage: [ @@ -130,9 +147,9 @@ test('pricing coverage after the observation cannot qualify an underpriced prefi assert.ok(points[0].pricingCoverage > 0.8); assert.equal(points[0].utilizationPercent, 50); - assert.equal(points[0].fullEquivalentUsd, null); + assert.equal(points[0].fullEquivalentUsd, 100); assert.equal(points[0].unusedEquivalentUsd, null); - assert.equal(points[0].confidence, 'usage-only'); + assert.equal(points[0].confidence, 'low'); }); test('source keys keep overlapping Codex account windows from sharing usage', () => { @@ -523,7 +540,7 @@ test('an irregular Codex reset is mapped to the fixed period containing its obse assert.equal(previous?.usedEquivalentUsd, 30); assert.equal(previous?.utilizationPercent, 40); assert.equal(previous?.fullEquivalentUsd, 75); - assert.equal(previous?.unusedEquivalentUsd, 45); + assert.equal(previous?.unusedEquivalentUsd, null); assert.equal(previous?.confidence, 'low'); assert.equal(current?.usedEquivalentUsd, 40); assert.equal(current?.utilizationPercent, 20); @@ -625,3 +642,107 @@ test('future-dated quota observations cannot anchor weekly history', () => { assert.equal(points[0].utilizationPercent, null); assert.equal(points[0].fullEquivalentUsd, null); }); + +test('multiple same-window observations use a robust candidate aggregate instead of the latest outlier', () => { + const points = buildWeeklyValueTimeline('claude', { + observations: [ + observation({ + provider: 'claude', + seriesKey: 'active-claude', + windowId: 'window-one', + accountAttribution: 'profile-continuity', + observedAt: RESET - 4 * HOUR, + usedPercent: 20, + }), + observation({ + provider: 'claude', + seriesKey: 'active-claude', + windowId: 'window-one', + accountAttribution: 'profile-continuity', + observedAt: RESET - 3 * HOUR, + usedPercent: 40, + }), + observation({ + provider: 'claude', + seriesKey: 'active-claude', + windowId: 'window-one', + accountAttribution: 'profile-continuity', + observedAt: RESET - 2 * HOUR, + usedPercent: 10, + }), + ], + usage: [ + usage(20, RESET - 5 * HOUR), + usage(20, RESET - 3.5 * HOUR), + usage(20, RESET - 2.5 * HOUR), + ], + }, { now: RESET + HOUR }); + + assert.equal(points[0].fullEquivalentUsd, 100); + assert.equal(points[0].usedEquivalentUsd, 60); + assert.equal(points[0].unusedEquivalentUsd, 40); +}); + +test('low priced coverage lowers confidence but does not hide a mathematically valid total', () => { + const points = buildWeeklyValueTimeline('claude', { + observations: [observation({ + provider: 'claude', + seriesKey: 'active-claude', + accountAttribution: 'profile-continuity', + observationConfidence: 'medium', + usedPercent: 50, + })], + usage: [{ + timestamp: RESET - 2 * HOUR, + equivalentUsd: 20, + pricedTokens: 100, + totalTokens: 1_000, + }], + }, { now: RESET + HOUR }); + + assert.equal(points[0].fullEquivalentUsd, 40); + assert.equal(points[0].unusedEquivalentUsd, null); + assert.equal(points[0].confidence, 'low'); +}); + +test('one unattributed Codex window may show a low-confidence total but never an unused claim', () => { + const points = buildWeeklyValueTimeline('codex', { + observations: [observation({ + seriesKey: 'codex-epoch-1', + windowId: 'window-one', + accountAttribution: 'unattributed', + usedPercent: 50, + })], + usage: [usage(40, RESET - 2 * HOUR)], + }, { now: RESET + HOUR }); + + assert.equal(points[0].fullEquivalentUsd, 80); + assert.equal(points[0].unusedEquivalentUsd, null); + assert.equal(points[0].confidence, 'low'); +}); + +test('overlapping unattributed Codex epochs never share usage for an allowance estimate', () => { + const points = buildWeeklyValueTimeline('codex', { + observations: [ + observation({ + seriesKey: 'codex-epoch-1', + windowId: 'window-one', + accountAttribution: 'unattributed', + observedAt: RESET - 2 * HOUR, + usedPercent: 50, + }), + observation({ + seriesKey: 'codex-epoch-2', + windowId: 'window-two', + accountAttribution: 'unattributed', + observedAt: RESET - HOUR, + usedPercent: 25, + }), + ], + usage: [usage(40, RESET - 3 * HOUR)], + }, { now: RESET + HOUR }); + + assert.equal(points[0].fullEquivalentUsd, null); + assert.equal(points[0].unusedEquivalentUsd, null); + assert.equal(points[0].confidence, 'usage-only'); +}); diff --git a/src/weeklyValue.ts b/src/weeklyValue.ts index b75ca499..98f7bdf6 100644 --- a/src/weeklyValue.ts +++ b/src/weeklyValue.ts @@ -5,7 +5,6 @@ import { ProviderTokenCounts, UsageProvider } from './providers/providerTypes'; const WEEK_MS = 7 * 24 * 60 * 60 * 1000; const RESET_CLUSTER_MS = 5 * 60 * 1000; -const MIN_EXTRAPOLATION_PERCENT = 5; const MIN_PRICED_SHARE = 0.8; export type WeeklyValueConfidence = 'high' | 'medium' | 'low' | 'usage-only'; @@ -22,6 +21,19 @@ export interface WeeklyQuotaObservation { resetAt: number; usedPercent: number; sourceKey?: string; + /** Account continuity carried by the schema-2 local observation store. Old + * fixtures and migration rows may omit these fields and are treated as + * low-information legacy evidence. */ + accountAttribution?: 'verified-local-signal' | 'profile-continuity' | 'unattributed'; + windowId?: string; + observationConfidence?: 'high' | 'medium' | 'low'; + flags?: Array< + | 'approximate-boundary' + | 'low-log-coverage' + | 'low-price-coverage' + | 'account-ambiguous' + | 'clock-anomaly' + >; } export interface WeeklyEquivalentUsage { @@ -279,10 +291,15 @@ function clusterObservations(observations: WeeklyQuotaObservation[]): Observatio ); for (const observation of ordered) { const previous = clusters[clusters.length - 1]; + const previousWindowId = previous?.observations[0]?.windowId; + const sameExplicitWindow = previousWindowId !== undefined || observation.windowId !== undefined + ? previousWindowId !== undefined && previousWindowId === observation.windowId + : true; if ( previous && previous.provider === observation.provider && previous.seriesKey === observation.seriesKey && + sameExplicitWindow && Math.abs(previous.resetAt - observation.resetAt) <= RESET_CLUSTER_MS ) { previous.observations.push(observation); @@ -511,6 +528,120 @@ function alignedResetAt(resetAt: number, anchorResetAt: number): number | null { return Math.abs(aligned - resetAt) <= RESET_CLUSTER_MS ? aligned : null; } +interface AllowanceCandidateSummary { + value: number; + pricingCoverage: number; + count: number; + dispersed: boolean; +} + +function evidenceWeight( + observation: WeeklyQuotaObservation, + pricingCoverage: number, + boundaryApproximate: boolean, + windowStart: number, + periodEnd: number, +): number { + const confidenceWeight = observation.observationConfidence === 'high' + ? 1 + : observation.observationConfidence === 'medium' + ? 0.8 + : observation.observationConfidence === 'low' + ? 0.55 + : 0.7; + const attributionWeight = observation.accountAttribution === 'verified-local-signal' + ? 1 + : observation.accountAttribution === 'profile-continuity' + ? 0.9 + : observation.accountAttribution === 'unattributed' + ? 0.55 + : 0.7; + const flags = new Set(observation.flags ?? []); + const flagWeight = + (flags.has('low-log-coverage') ? 0.6 : 1) * + (flags.has('low-price-coverage') ? 0.7 : 1) * + (flags.has('account-ambiguous') ? 0.4 : 1) * + (flags.has('clock-anomaly') ? 0.5 : 1); + const boundaryWeight = boundaryApproximate || flags.has('approximate-boundary') + ? 0.55 + : 1; + const span = Math.max(1, periodEnd - windowStart); + const progress = Math.max( + 0, + Math.min(1, (observation.observedAt - windowStart) / span), + ); + const recencyWeight = 0.65 + 0.35 * progress; + // Low coverage must lower confidence, not erase a mathematically valid + // observation. Keep a small non-zero floor for partially priced prefixes. + const coverageWeight = Math.max(0.05, Math.min(1, pricingCoverage)); + return confidenceWeight * attributionWeight * flagWeight * + boundaryWeight * recencyWeight * coverageWeight; +} + +function robustAllowanceCandidate( + observations: WeeklyQuotaObservation[], + rows: WeeklyEquivalentUsage[], + windowStart: number, + periodEnd: number, + boundaryApproximate: boolean, +): AllowanceCandidateSummary | null { + const candidates = observations.flatMap((observation) => { + if ( + observation.observedAt < windowStart || + observation.observedAt > periodEnd || + observation.usedPercent <= 0 || + observation.usedPercent > 100 + ) { + return []; + } + const observed = totals(rows.filter((row) => row.timestamp <= observation.observedAt)); + if (observed.equivalentUsd <= 0) { + return []; + } + const pricingCoverage = observed.totalTokens > 0 + ? Math.min(1, observed.pricedTokens / observed.totalTokens) + : 0; + const value = observed.equivalentUsd / (observation.usedPercent / 100); + if (!Number.isFinite(value) || value <= 0) { + return []; + } + return [{ + value, + pricingCoverage, + weight: evidenceWeight( + observation, + pricingCoverage, + boundaryApproximate, + windowStart, + periodEnd, + ), + }]; + }).sort((left, right) => left.value - right.value); + if (candidates.length === 0) return null; + const totalWeight = candidates.reduce((sum, item) => sum + item.weight, 0); + let accumulated = 0; + let median = candidates[candidates.length - 1].value; + for (const candidate of candidates) { + accumulated += candidate.weight; + if (accumulated >= totalWeight / 2) { + median = candidate.value; + break; + } + } + const weightedCoverage = candidates.reduce( + (sum, item) => sum + item.pricingCoverage * item.weight, + 0, + ) / Math.max(Number.EPSILON, totalWeight); + const minimum = candidates[0].value; + const maximum = candidates[candidates.length - 1].value; + return { + value: median, + pricingCoverage: weightedCoverage, + count: candidates.length, + dispersed: minimum > 0 && maximum / minimum > 2, + }; +} + /** * A quota sample can outlive the seven-day interval it describes. When that * is the only evidence left after a reset, falling back to Monday buckets would @@ -646,11 +777,8 @@ export function buildWeeklyValueTimeline( resetAligned: boolean; }; const historyByReset = new Map(history.map((point) => [point.resetAt, point])); - const mappedObservations = new Map(); + const mappedObservations = new Map(); for (const entry of validClusters) { - if (entry.cluster.seriesKey !== anchorEntry.cluster.seriesKey) { - continue; - } const alignedReset = alignedResetAt(entry.cluster.resetAt, anchorResetAt); const target = alignedReset === null ? history.find((point) => @@ -665,18 +793,9 @@ export function buildWeeklyValueTimeline( entry, resetAligned: alignedReset !== null, }; - const previous = mappedObservations.get(target.resetAt); - if ( - !previous || - entry.latest.observedAt > previous.entry.latest.observedAt || - ( - entry.latest.observedAt === previous.entry.latest.observedAt && - candidate.resetAligned && - !previous.resetAligned - ) - ) { - mappedObservations.set(target.resetAt, candidate); - } + const mapped = mappedObservations.get(target.resetAt) ?? []; + mapped.push(candidate); + mappedObservations.set(target.resetAt, mapped); } const ambiguousCurrentCodexReset = provider === 'codex' && validClusters.some((entry) => { if ( @@ -693,10 +812,16 @@ export function buildWeeklyValueTimeline( }); return history.map((point): WeeklyValuePoint => { - const mapped = mappedObservations.get(point.resetAt); - if (!mapped) { + const mappedEntries = mappedObservations.get(point.resetAt); + if (!mappedEntries || mappedEntries.length === 0) { return point; } + const seriesKeys = new Set(mappedEntries.map((mapped) => mapped.entry.cluster.seriesKey)); + const accountAmbiguous = provider === 'codex' && seriesKeys.size > 1; + const mapped = [...mappedEntries].sort((left, right) => + left.entry.latest.observedAt - right.entry.latest.observedAt || + Number(left.resetAligned) - Number(right.resetAligned), + )[mappedEntries.length - 1]; const { entry, resetAligned } = mapped; const latest = entry.latest; const sourceKeys = new Set( @@ -736,32 +861,37 @@ export function buildWeeklyValueTimeline( [...bucketSourceKeys].every((sourceKey) => sourceKeys.has(sourceKey)) ) ); - const observed = totals(seriesRows.filter((row) => row.timestamp <= latest.observedAt)); - const observedPricingCoverage = observed.totalTokens > 0 - ? Math.min(1, observed.pricedTokens / observed.totalTokens) - : 0; + const candidateSummary = robustAllowanceCandidate( + entry.cluster.observations, + seriesRows, + point.windowStart, + Math.min(now, point.resetAt), + !resetAligned || point.boundaryUncertain === true, + ); + const observedPricingCoverage = candidateSummary?.pricingCoverage ?? 0; const observationGapMs = Math.max( 0, (point.current ? now : point.resetAt) - latest.observedAt, ); const withholdInference = provider === 'codex' && ( + accountAmbiguous || + latest.flags?.includes('account-ambiguous') === true || (point.current && ambiguousCurrentCodexReset) ); const approximateInference = provider === 'codex' && ( !resetAligned || !codexSourceAttributionExact || - point.boundaryUncertain === true + point.boundaryUncertain === true || + latest.accountAttribution === 'unattributed' ); let fullEquivalentUsd: number | null = null; let observationOverrun = false; if ( !withholdInference && point.usageAvailable !== false && - latest.usedPercent >= MIN_EXTRAPOLATION_PERCENT && - observed.equivalentUsd > 0 && - observedPricingCoverage >= MIN_PRICED_SHARE + candidateSummary !== null ) { - fullEquivalentUsd = observed.equivalentUsd / (latest.usedPercent / 100); + fullEquivalentUsd = candidateSummary.value; if (!Number.isFinite(fullEquivalentUsd) || fullEquivalentUsd <= 0) { fullEquivalentUsd = null; } else if (point.usedEquivalentUsd > fullEquivalentUsd) { @@ -776,13 +906,22 @@ export function buildWeeklyValueTimeline( utilizationPercent: latest.usedPercent, fullEquivalentUsd, unusedEquivalentUsd: - !point.current && fullEquivalentUsd !== null + !point.current && + fullEquivalentUsd !== null && + !accountAmbiguous && + resetAligned && + point.boundaryUncertain !== true && + latest.accountAttribution !== 'unattributed' && + observedPricingCoverage >= MIN_PRICED_SHARE ? Math.max(0, fullEquivalentUsd - point.usedEquivalentUsd) : null, observationGapMs, confidence: fullEquivalentUsd === null ? 'usage-only' - : observationOverrun || approximateInference + : observationOverrun || + approximateInference || + observedPricingCoverage < MIN_PRICED_SHARE || + candidateSummary?.dispersed === true ? 'low' : confidenceFor(point.current, observationGapMs, observedPricingCoverage), basis: 'quota-observation', @@ -857,10 +996,17 @@ export function buildWeeklyValueTrend( continue; } const observed = totals(matchedUsage.filter((row) => row.timestamp <= latest.observedAt)); + const observedPricingCoverage = observed.totalTokens > 0 + ? Math.min(1, observed.pricedTokens / observed.totalTokens) + : 0; const pricingCoverage = used.totalTokens > 0 ? Math.min(1, used.pricedTokens / used.totalTokens) : 0; const current = cluster.resetAt > now; + const unusedAttributionSafe = latest.accountAttribution !== 'unattributed'; + const unusedBoundarySafe = !(latest.flags ?? []).some((flag) => + flag === 'approximate-boundary' || flag === 'account-ambiguous', + ); const observationGapMs = Math.max( 0, (current ? now : cluster.resetAt) - latest.observedAt, @@ -868,9 +1014,9 @@ export function buildWeeklyValueTrend( let fullEquivalentUsd: number | null = null; let observationOverrun = false; if ( - latest.usedPercent >= MIN_EXTRAPOLATION_PERCENT && + latest.usedPercent > 0 && observed.equivalentUsd > 0 && - pricingCoverage >= MIN_PRICED_SHARE + latest.usedPercent <= 100 ) { fullEquivalentUsd = observed.equivalentUsd / (latest.usedPercent / 100); if (!Number.isFinite(fullEquivalentUsd) || fullEquivalentUsd <= 0) { @@ -893,7 +1039,11 @@ export function buildWeeklyValueTrend( usedEquivalentUsd: used.equivalentUsd, fullEquivalentUsd, unusedEquivalentUsd: - !current && fullEquivalentUsd !== null + !current && + fullEquivalentUsd !== null && + observedPricingCoverage >= MIN_PRICED_SHARE && + unusedAttributionSafe && + unusedBoundarySafe ? Math.max(0, fullEquivalentUsd - used.equivalentUsd) : null, utilizationPercent: latest.usedPercent, @@ -901,7 +1051,9 @@ export function buildWeeklyValueTrend( observationGapMs, confidence: fullEquivalentUsd === null ? 'usage-only' - : observationOverrun + : observationOverrun || + latest.usedPercent < 5 || + observedPricingCoverage < MIN_PRICED_SHARE ? 'low' : confidenceFor(current, observationGapMs, pricingCoverage), basis: 'quota-observation', diff --git a/tests/ui/codex-visual.spec.mjs b/tests/ui/codex-visual.spec.mjs index 37de3f97..5a05fb00 100644 --- a/tests/ui/codex-visual.spec.mjs +++ b/tests/ui/codex-visual.spec.mjs @@ -295,8 +295,9 @@ test('weekly allowance value uses the shared all-time chart and exposes uncertai await expect(panel).toBeVisible(); await expect(panel).toContainText('Estimate, not a bill'); await expect(panel).toContainText('Historical used equivalents come directly from local token logs'); - await expect(panel).toContainText('When usage cannot be reliably attributed to a single quota observation'); - await expect(panel).toContainText('no account split or allowance estimate is invented'); + await expect(panel).toContainText('One coherent unattributed quota window can show a low-confidence total estimate'); + await expect(panel).toContainText('If epochs overlap or attribution conflicts, only used equivalent is shown'); + await expect(panel).toContainText('no account split is invented'); await expect(panel).toContainText('If an official reset falls within a recorded day'); await expect(panel.locator('tbody')).toContainText('Low · Boundary approx.'); await expect(panel.locator('.hc-col')).toHaveCount(2); From 046c64b1ef11985572b0cddda1b203d1bff651e3 Mon Sep 17 00:00:00 2001 From: Carl <74130028+Carl723000@users.noreply.github.com> Date: Wed, 2 Sep 2026 01:43:05 +0800 Subject: [PATCH 25/72] docs: close G3 checkpoint Co-authored-by: OpenAI Codex <215057067+openai-codex[bot]@users.noreply.github.com> --- .../2026-09-01-v2.3.1-goal-ledger.en.md | 35 +++++++++++++++++-- .../2026-09-01-v2.3.1-goal-ledger.zh-CN.md | 35 +++++++++++++++++-- 2 files changed, 66 insertions(+), 4 deletions(-) diff --git a/docs/releases/2026-09-01-v2.3.1-goal-ledger.en.md b/docs/releases/2026-09-01-v2.3.1-goal-ledger.en.md index 7ed0ff7a..846920fc 100644 --- a/docs/releases/2026-09-01-v2.3.1-goal-ledger.en.md +++ b/docs/releases/2026-09-01-v2.3.1-goal-ledger.en.md @@ -17,8 +17,8 @@ This tracked ledger records G0–G6 stage status, reproducible acceptance eviden | G0 baseline and fixtures | Complete | `f0af506` | Node 795 passed / 1 skipped; UI 90 passed; targeted 29 passed | N/A | No blocking risk | | G1 data contract | Complete | `86d95df` | Focused 17 passed; Node 806 passed / 1 skipped; compile passed | Conditional GO, then revision GO | Blocked legacy workspace plaintext remains until explicit recovery; host SecretStorage backing varies; unified quota store is a G3 deliverable | | G2 statistics/order/effort | Complete | `49f2041` | Node 815 passed / 1 skipped; UI 90 passed; all focused regressions passed | N/A | Unified observation/account attribution remains a G3 concern | -| G3 quota observations/estimates | In progress | — | Architecture and fixture audit underway | Pending | — | -| G4A visual/interaction/accessibility | Not started | — | — | N/A | — | +| G3 quota observations/estimates | Complete | `a04ab04` | Node 841 passed / 1 skipped; UI 90 passed; compile passed | GO, then revision GO | Quarantined corrupt/future files remain on disk until the data-clear path or a future cleanup policy removes them | +| G4A visual/interaction/accessibility | In progress | — | Shared-token, narrow-width, and accessibility audit underway | N/A | — | | G4B combined heatmap/share | Not started | — | — | Pending | — | | G5 README/screenshots | Not started | — | — | Pending | — | | G6 release candidate | Not started | — | — | N/A | — | @@ -121,6 +121,37 @@ Additional guards cover the complete 30-day definition in the configured timezon - [x] Explicit reasoning-effort variants normalize; missing/invalid evidence becomes `unknown`; zero unknown is hidden. - [x] Full Node and browser gates pass. +## G3 quota-observation and weekly-estimate evidence + +### Persisted observation boundary + +- Added a schema-versioned, provider/account/window-partitioned observation store with machine-local HMAC fingerprints. Raw credentials and identity signals are discarded after fingerprinting; Webview projections receive only per-run ephemeral aliases. +- Observation writes use a same-directory `0600` temporary file, `fsync`, and atomic rename. A cross-process lease, locked re-read, canonical deduplication, 180-day retention, and bounded compaction make repeated refreshes stable without merging account epochs. +- Strict validation quarantines corrupt or future-schema files instead of trusting or deleting them. Legacy P3/P4 global-state observations and P5 Codex index history migrate P2-first; source keys and migration markers advance only after the destination append succeeds, so an interrupted migration retries safely. +- Claude account continuity is derived only after a successful quota response from a one-way refresh-token HMAC. The token itself is never returned, persisted, logged, or exposed to the Webview. Codex observations use the same store and privacy projection. + +### Reset detection and allowance inference + +- Reset events are derived at runtime from provider window identity, changed `resetAt`, or a significant used-percentage rollback. Account changes alone do not invent an official reset; same-day and consecutive-day resets remain distinct. +- Every valid observation with `0 < usedFraction ≤ 1` contributes `priced usage through observedAt ÷ usedFraction` as a candidate total. A weighted robust median combines candidates while pricing coverage, log coverage, boundary precision, and attribution determine confidence. +- Estimated total is never lower than confirmed used value. Unused value is clamped non-negative and appears only for a closed window with reliable account attribution, boundary continuity, and sufficient pricing coverage. +- A single coherent unattributed Codex window may expose a low-confidence total but never unused value. Overlapping or multi-account ambiguity degrades to used-only instead of inventing an account split. + +### Verification and read-only review + +- Focused fixtures cover current/legacy adapters, atomic/deduplicated persistence, concurrent append, deterministic out-of-order identities, corrupt/future quarantine, retention, interrupted migration, token rotation, stale in-flight responses, same-day and consecutive resets, rollback noise, reset changes, duplicate observations, multi-account ambiguity, low pricing coverage, current windows, and non-negative floors. +- Final full Node suite: 842 tests, 841 passed, 0 failed, 1 filesystem-capability skip. Final Playwright suite: 90/90 passed. TypeScript compilation passed. +- Claude CLI checkpoint review: GO with no blocker. It requested legacy-trend hardening, a real migration-write-failure fixture, and explicit account-transition semantics; all were added. The single permitted revision review returned GO for G4A with no blocker, followed by a tighter exact-error assertion. + +### G3 exit gate + +- [x] Persist immutable, anonymous, per-account quota observations without credentials or stable identifiers in UI state. +- [x] Preserve irregular, consecutive, and same-day reset evidence without treating noise or account switches as official resets. +- [x] Produce a total allowance estimate whenever a valid same-window utilization observation exists, with robust aggregation and explicit confidence. +- [x] Suppress unsafe unused estimates and prevent negative or below-used values. +- [x] Make migration, repeated refresh, and concurrent append idempotent and recoverable. +- [x] Pass full Node/browser/compile gates and both allowed Claude review passes. + ## Update rule Each stage must finish its automated verification before this ledger records its commit, evidence, Claude decision, and risk. The G6 acceptance matrix must be derived from these records; screenshots or copy cannot substitute for a failed gate. diff --git a/docs/releases/2026-09-01-v2.3.1-goal-ledger.zh-CN.md b/docs/releases/2026-09-01-v2.3.1-goal-ledger.zh-CN.md index 6d81e9d0..fd02c2b5 100644 --- a/docs/releases/2026-09-01-v2.3.1-goal-ledger.zh-CN.md +++ b/docs/releases/2026-09-01-v2.3.1-goal-ledger.zh-CN.md @@ -17,8 +17,8 @@ | G0 基线与夹具 | 已完成 | `f0af506` | Node 795 passed / 1 skipped;UI 90 passed;定向 29 passed | 不适用 | 无阻塞风险 | | G1 数据契约 | 已完成 | `86d95df` | 定向 17 passed;Node 806 passed / 1 skipped;compile passed | 首轮有条件 GO,修订复核 GO | 受阻 workspace 明文需用户显式恢复;SecretStorage 底层保护依赖宿主;统一额度存储属于 G3 | | G2 统计/排序/effort | 已完成 | `49f2041` | Node 815 passed / 1 skipped;UI 90 passed;全部定向回归通过 | 不适用 | 统一观测与账号归属仍由 G3 处理 | -| G3 额度观测与估算 | 进行中 | — | 正在审计架构与验收夹具 | 待审阅 | — | -| G4A 视觉/交互/可访问性 | 未开始 | — | — | 不适用 | — | +| G3 额度观测与估算 | 已完成 | `a04ab04` | Node 841 passed / 1 skipped;UI 90 passed;compile passed | 首轮 GO,修订复核 GO | 损坏/未来 schema 的隔离文件会保留至“清除数据”或未来清理策略处理 | +| G4A 视觉/交互/可访问性 | 进行中 | — | 正在审计共享 token、窄宽度和可访问性 | 不适用 | — | | G4B 综合热力图与分享 | 未开始 | — | — | 待审阅 | — | | G5 README/截图 | 未开始 | — | — | 待审阅 | — | | G6 发布候选 | 未开始 | — | — | 不适用 | — | @@ -121,6 +121,37 @@ - [x] 显式 reasoning-effort 新旧变体可规范化;缺失/非法证据进入 `unknown`;零 unknown 隐藏。 - [x] 完整 Node 与浏览器门均通过。 +## G3 额度观测与周额度估算证据 + +### 持久化观测边界 + +- 新增带 schema 版本、按 provider/账号/窗口分区的观测存储,账号与窗口仅使用机器本地 HMAC fingerprint。原始凭证和身份信号完成摘要后立即丢弃;Webview 只能收到本次运行内的临时别名。 +- 写入采用同目录 `0600` 临时文件、`fsync` 和原子 rename;跨进程 lease、锁内重读、规范化去重、180 天保留期及有界压缩保证重复刷新稳定且不合并账号 epoch。 +- 严格校验会隔离损坏或未来 schema 文件,而不是信任或删除它们。P3/P4 globalState 旧观测及 P5 Codex index history 均先写入 P2,只有目标 append 成功后才清理源键或推进迁移 marker,因此中断后可安全重试。 +- Claude 的账号连续性只在额度请求成功后由 refresh token 的单向 HMAC 得到;token 本身不返回、不持久化、不记录日志、也不进入 Webview。Codex 观测使用同一存储及隐私投影。 + +### 重置检测与额度推断 + +- reset 事件在运行时依据 provider window identity、`resetAt` 变化或 used 百分比显著回落推导。单纯账号切换不会伪造官方重置;同日多次及连续两日重置均保留为独立事件。 +- 每个满足 `0 < usedFraction ≤ 1` 的有效观测都会形成“截至 observedAt 的已定价用量 ÷ usedFraction”候选总额度;多个候选用加权稳健中位数合并,并由定价覆盖、日志覆盖、边界精度和账号归属共同决定可信度。 +- 总额度估算绝不低于窗口已确认使用值;未用值强制非负,且仅在窗口已闭合、账号归属可靠、边界连续且定价覆盖充分时显示。 +- 单一且连贯、但无法归属账号的 Codex 窗口可以显示低可信总额度,但不显示未用值;窗口重叠或多账号歧义时只显示已用值,不虚构账号拆分。 + +### 验证与只读审阅 + +- 定向 fixture 覆盖新旧 adapter、原子/去重持久化、并发 append、乱序窗口身份、损坏/未来 schema 隔离、保留期、中断迁移、token 轮换、过期 in-flight 响应、同日/连续 reset、回落噪声、reset 变化、重复观测、多账号歧义、低定价覆盖、当前窗口及非负下限。 +- 最终完整 Node:842 项,841 passed,0 failed,1 项因文件系统能力 skipped;最终 Playwright:90/90 passed;TypeScript compile passed。 +- Claude CLI 检查点首轮无阻塞 GO,并要求加固旧 trend、加入真实迁移写失败 fixture、明确账号切换语义;均已完成。允许的一次修订复核返回“可进入 G4A、无阻塞”,随后又收紧了精确错误断言。 + +### G3 通过条件 + +- [x] 以不可变、匿名、按账号隔离的形式保存额度观测,不保存凭证,不向 UI 暴露稳定标识。 +- [x] 保留不规则、连续及同日多次 reset 证据,且不把噪声或账号切换误判为官方重置。 +- [x] 只要同窗存在有效利用率观测即给出总额度估算,并采用稳健聚合及显式可信度。 +- [x] 抑制不安全的未用额度,且任何估算不得为负或低于已确认使用值。 +- [x] 迁移、重复刷新和并发 append 均幂等且可恢复。 +- [x] 通过完整 Node、浏览器、compile 门及两次允许的 Claude 审阅。 + ## 后续更新规则 每阶段必须先完成对应自动化验证,再更新本表中的 commit、测试证据、Claude 结论和风险。G6 的最终验收矩阵应由本台账中的证据生成;任何未通过项不得用截图或文案替代。 From fca21da5dca2609c2624574f07ddcd45248efd4b Mon Sep 17 00:00:00 2001 From: Carl <74130028+Carl723000@users.noreply.github.com> Date: Wed, 2 Sep 2026 02:08:36 +0800 Subject: [PATCH 26/72] feat: align dashboard interaction and accessibility Co-authored-by: OpenAI Codex <215057067+openai-codex[bot]@users.noreply.github.com> --- src/webview.ts | 580 +++++++++++++++++--------- tests/ui/codex-accessibility.spec.mjs | 200 ++++++++- tests/ui/codex-responsive.spec.mjs | 46 +- tests/ui/codex-visual.spec.mjs | 342 ++++++++++++++- 4 files changed, 962 insertions(+), 206 deletions(-) diff --git a/src/webview.ts b/src/webview.ts index 6318f953..155a9bbd 100644 --- a/src/webview.ts +++ b/src/webview.ts @@ -1566,7 +1566,7 @@ export class UsageWebviewProvider {
    -
    +

    ${I18n.t.statusBar.loading}

    @@ -1588,7 +1588,7 @@ export class UsageWebviewProvider {
    -
    +