The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment

Project	Deployment	Actions	Updated (UTC)
docs	Skipped		Jun 4, 2026 9:45pm

PR Summary

Medium Risk
Touches the hosted-key execution and billing-adjacent telemetry path; changes are additive observability but mis-labeled metrics could skew operational dashboards.

Overview
Hosted-key observability moves from sampled trace events to unsampled OpenTelemetry metrics, so usage, cost, throttles, and queue behavior show up reliably in Grafana instead of disappearing when traces are sampled.

The Next.js OTel bootstrap now exports metrics on the same OTLP endpoint as traces (normalizeOtlpMetricsUrl + PeriodicExportingMetricReader at 60s). A new hostedKeyMetrics module defines low-cardinality counters/histograms (hosted_key.used, hosted_key.cost_charged, hosted_key.failed, hosted_key.throttled, hosted_key.upstream_rate_limited, hosted_key.queue_wait_duration, etc.) with a key label for the env var name only.

PlatformEvents hosted-key helpers now call these recorders instead of trackPlatformEvent spans. executeTool records success (used + cost_charged), non-success paths (failed with rate_limited / auth / other), and updates the key label after re-acquire. Adds @opentelemetry/sdk-metrics.

^{Reviewed by Cursor Bugbot for commit e8dae53. Bugbot is set up for automated code reviews on this repo. Configure here.}

Greptile Summary

This PR replaces span-based platform.hosted_key.* events (which were silently dropped by the trace sampler) with proper OTel metrics, wiring a PeriodicExportingMetricReader into the Next.js NodeSDK and emitting counters/histograms for usage, cost, failures, throttles, and queue waits.

• apps/sim/lib/monitoring/metrics.ts — new file defining lazy-initialized, no-op-safe OTel instruments (hosted_key.used, hosted_key.cost_charged, hosted_key.failed, hosted_key.throttled, hosted_key.upstream_rate_limited, hosted_key.queue_wait_duration, hosted_key.queue_wait_exceeded) with low-cardinality labels (provider, tool, reason, key env-var name).
• apps/sim/instrumentation-node.ts — adds PeriodicExportingMetricReader (60 s flush) sharing the existing OTLP endpoint/headers; normalizeOtlpMetricsUrl swaps the /v1/traces suffix for /v1/metrics.
• apps/sim/tools/index.ts — threads envVarName through applyHostedKeyCostToResult, adds classifyHostedKeyFailure, and records used/cost_charged on success and failed on both throwing and non-throwing failure paths; extends the retry-requeue path to update metric labels when a fresh key is acquired.

Confidence Score: 4/5

Safe to merge; the new metric plumbing is correct for all common paths and the OTel no-op fallback ensures nothing breaks without a provider configured.

The metric recording logic is sound across the normal success, direct-failure, and retry-exhaustion paths. The one concrete data-quality gap is in the requeue edge case: when retries are exhausted, a fresh key is acquired, and the requeued attempt itself fails — recordFailed fires with reason=rate_limited (from the original 429 error object) but attributed to the freshly acquired key, which never actually hit the rate limit.

apps/sim/tools/index.ts — specifically the requeue path around line 1237 where hostedKeyForMetrics.key is mutated before the requeued call executes.

Important Files Changed

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[executeTool called] --> B{isUsingHostedKey?}
    B -- No --> Z[execute normally, no metrics]
    B -- Yes --> C[set hostedKeyForMetrics]
    C --> D{directExecution?}

    D -- Yes --> E[run directExecution]
    E --> F{finalResult.success?}
    F -- Yes --> G[applyHostedKeyCostToResult\nrecordUsed + recordCostCharged]
    F -- No --> H[recordFailed reason=other]

    D -- No --> I[executeWithRetry]
    I --> J{throws?}
    J -- No --> K{finalResult.success?}
    K -- Yes --> G
    K -- No --> H

    J -- Yes: non-429 --> L[outer catch\nrecordFailed classifyHostedKeyFailure]
    J -- Yes: 429 retries exhausted --> M{reacquireAfterRetriesExhausted?}
    M -- requeue succeeds --> N[return result\nrecordUsed + recordCostCharged]
    M -- requeue fails or requeued call throws --> O[recordThrottled\nouter catch recordFailed reason=rate_limited\nkey=FRESH_KEY ⚠️]
    M -- no requeue --> P[recordThrottled\nouter catch recordFailed]

_{Reviews (3): Last reviewed commit: "fix(metrics): parse OTLP metrics URL via..." | Re-trigger Greptile}

@greptile review

@greptile review