The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment

Project	Deployment	Actions	Updated (UTC)
docs	Skipped		Jun 9, 2026 12:16am

PR Summary

Medium Risk
Changes production observability and billing-adjacent cost/throttle signals; failures are swallowed so dashboards may gap, but request paths stay safe and AWS creds gate emission.

Overview
Replaces the OpenTelemetry meter backend for hosted-key metrics with direct CloudWatch PutMetricData under the Sim/HostedKey namespace, while keeping the same hostedKeyMetrics API so telemetry.ts and tool call sites are unchanged.

The new implementation buffers metric points in-process and flushes them asynchronously (5s interval, 1000-point threshold, and awaited drains on SIGTERM/SIGINT/beforeExit). It is a no-op when AWS_ACCESS_KEY_ID is missing (local dev). Dimensions are renamed to CloudWatch-style PascalCase (Provider, Tool, Key, Reason, plus Environment). Queue wait moves from an OTel histogram to individual QueueWaitDuration millisecond datapoints. A 10k buffer cap drops oldest points if flushing stalls; failed batches are logged and dropped so telemetry never blocks requests. flushHostedKeyMetrics() is exported for tests and explicit drains.

^{Reviewed by Cursor Bugbot for commit bbe5f42. Bugbot is set up for automated code reviews on this repo. Configure here.}

Greptile Summary

This PR replaces the OTel/Prometheus backend for hosted-key metrics with a CloudWatch push-based buffer (Sim/HostedKey namespace), keeping the hostedKeyMetrics facade unchanged so no call-site edits are needed. The motivation is sound: CloudWatch server-side aggregation eliminates the per-process counter-reset problem that broke cumulative Prometheus metrics in ephemeral trigger.dev workers.

• The buffer/batch/flush design (5 s timer, FLUSH_THRESHOLD-triggered drain, beforeExit/SIGTERM/SIGINT exit handlers, MAX_BUFFER hard cap with oldest-drop) is well-structured; the synchronous buffer swap in flushHostedKeyMetrics prevents double-sends.
• QueueWaitDuration was previously a histogram; individual MetricDatum pushes only give CloudWatch Sum/Avg/Min/Max, so percentile-based SLOs or alerts that relied on the histogram distribution will silently degrade to average-only data.
• ENABLED is gated solely on AWS_ACCESS_KEY_ID being present, which returns false under IAM instance-profile or task-role auth; any future migration from static keys to role-based credentials would silently disable all metrics.

Confidence Score: 5/5

Safe to merge — the change is well-scoped, call sites are untouched, and the buffering logic correctly prevents data loss under normal operation.

The architectural switch from OTel to CloudWatch is sound and the implementation handles the core concerns (batching, exit flushing, memory bounding, no-op without creds). The two flagged items are observational concerns: one around histogram precision being lost for QueueWaitDuration, and one around the ENABLED guard not covering IAM role credentials. Neither causes incorrect behavior today given the documented deployment model with static AWS creds.

apps/sim/lib/monitoring/metrics.ts — specifically the ENABLED guard and QueueWaitDuration metric type.

Important Files Changed

Sequence Diagram

sequenceDiagram
    participant Caller as Caller
    participant Enqueue as enqueue
    participant Buffer as Buffer
    participant Flush as flushHostedKeyMetrics
    participant CW as CloudWatch

    Caller->>Enqueue: recordUsed / recordFailed / etc.
    Enqueue->>Buffer: push MetricDatum
    Enqueue->>Enqueue: ensureBackground registers 5s timer and exit handlers
    alt buffer reaches FLUSH_THRESHOLD 1000
        Enqueue-->>Flush: fire-and-forget flush
    end

    Flush->>Buffer: swap pending buffer to empty
    Flush->>CW: PutMetricDataCommand batched
    alt success
        CW-->>Flush: OK
    else error
        Flush->>Flush: log warn and drop batch
    end

    Note over Enqueue,Flush: SIGTERM or SIGINT or beforeExit triggers await flush

_{Reviews (2): Last reviewed commit: "fix(metrics): await metric flush on shut..." | Re-trigger Greptile}

Addressed the Bugbot findings in bbe5f4265:

1. Shutdown handlers don't await flush — the SIGTERM/SIGINT/beforeExit handler is now async and awaits flushHostedKeyMetrics(), so the final batch drains before exit. This is also what makes ephemeral trigger.dev workers reliably flush on teardown.
2. Buffer grows without cap — added a hard MAX_BUFFER (10k) cap separate from the flush threshold (1k). If flushing stalls, the oldest points are dropped to bound memory, and the dropped count is logged on the next flush.
3. Worker flush never invoked — flushing is automatic via the timer + the now-awaited exit handlers (which call the exported flushHostedKeyMetrics()), so ephemeral workers flush on beforeExit/SIGTERM without needing an explicit per-run call. Updated the module doc to describe this accurately; the export remains for explicit/early draining (e.g. tests).

@greptile review