Skip to content

feat: track step runtime_ms on chat messages#23219

Merged
kylecarbs merged 1 commit into
mainfrom
stevenmasley/chat-runtime-ms
Mar 18, 2026
Merged

feat: track step runtime_ms on chat messages#23219
kylecarbs merged 1 commit into
mainfrom
stevenmasley/chat-runtime-ms

Conversation

@kylecarbs
Copy link
Copy Markdown
Member

Summary

Adds a runtime_ms column to chat_messages that records the wall-clock duration (in milliseconds) of each LLM step. This covers LLM streaming, tool execution, and retries — the full time the agent is "alive" for a step.

This is the foundation for billing by agent alive time. The column follows the same pattern as total_cost_micros: stored per assistant message, aggregatable with SUM() over time periods by user.

Changes

  • Migration: adds nullable runtime_ms bigint to chat_messages.
  • chatloop: adds Runtime time.Duration field to PersistedStep, measures time.Since(stepStart) at the beginning of each step (covering stream + tool execution + retries).
  • chatd: passes step.Runtime.Milliseconds() to the assistant message InsertChatMessage call; all other message types (system, user, tool) get NULL.
  • Tests: adds runtime > 0 assertion in chatloop tests.

Billing query pattern

Once ready, aggregation mirrors the existing cost queries:

SELECT COALESCE(SUM(cm.runtime_ms), 0)::bigint AS total_runtime_ms
FROM chat_messages cm
JOIN chats c ON c.id = cm.chat_id
WHERE c.owner_id = @user_id
  AND cm.created_at >= @start_time
  AND cm.created_at < @end_time
  AND cm.runtime_ms IS NOT NULL;

Adds a runtime_ms column to chat_messages that records the
wall-clock duration (in milliseconds) of each LLM step. This
covers LLM streaming, tool execution, and retries — the full
time the agent is 'alive' for a step.

The value is persisted on assistant-role messages only (tool
and system messages get NULL). This aligns with the existing
billing pattern where total_cost_micros is stored per assistant
message and aggregated with SUM() over time periods.

Changes:
- Migration: adds nullable runtime_ms bigint to chat_messages.
- chatloop: adds Runtime field to PersistedStep, measures
  time.Since(stepStart) covering stream + tool execution.
- chatd: passes step.Runtime.Milliseconds() to the assistant
  message InsertChatMessage call; all other message types get
  NULL.
- Tests: adds runtime > 0 assertion in chatloop tests.
@kylecarbs kylecarbs force-pushed the stevenmasley/chat-runtime-ms branch from 5ce6d93 to 7435e51 Compare March 18, 2026 14:26
@kylecarbs kylecarbs requested a review from johnstcn March 18, 2026 14:43
@kylecarbs kylecarbs changed the title feat(chatd): track step runtime_ms on chat messages feat: track step runtime_ms on chat messages Mar 18, 2026
@kylecarbs kylecarbs merged commit 4dd8531 into main Mar 18, 2026
29 of 30 checks passed
@kylecarbs kylecarbs deleted the stevenmasley/chat-runtime-ms branch March 18, 2026 14:57
@github-actions github-actions Bot locked and limited conversation to collaborators Mar 18, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants