Skip to content

v2.0.0 dedup logic discards real token data when proxy writes two records per message (tokens=0 placeholder + real values) #18

Closed

Description

@zhaoxiao9302

Title: v2.0.0 dedup logic discards real token data when proxy writes two records per message (tokens=0 placeholder + real values)

Body:

Bug Description

v2.0.0 introduced deduplication via createUniqueHash() (messageId + requestId). When using a proxy (e.g. mimo, ccswitch) that writes two records per message to the .jsonl files — a placeholder with tokens=0 first, then a second record with real token counts — the dedup keeps the zero-value record and discards the real one.

This causes the status bar and webview to show drastically incorrect usage (e.g. \$0.20 instead of \$7.98 for today).

Root Cause

Proxies like mimo write records to jsonl in a streaming fashion:

// Record 1 — written immediately when streaming starts (placeholder)
{"message":{"id":"msg_xxx","usage":{"input_tokens":0,"output_tokens":0}}, "timestamp":"2026-06-01T03:43:44.657Z"}

// Record 2 — written after response completes (real values)
{"message":{"id":"msg_xxx","usage":{"input_tokens":271,"output_tokens":5774}}, "timestamp":"2026-06-01T03:43:46.182Z"}

Both records share the same messageId (and both have no requestId), so createUniqueHash() generates the same hash for both. The dedup in loadUsageRecords() keeps the first occurrence and skips the rest:

// dataLoader.js line ~320
const uniqueHash = this.createUniqueHash(data);
if (uniqueHash && processedHashes.has(uniqueHash)) {
continue; // ← skips the real-value record
}

Impact

Metric Before dedup (raw) After dedup (v2.0.0)
Today non-zero records 191 4
Today cost ~$7.98 $0.20
All-time cost ~$79.85 ~$0.20

Out of 188 unique message hashes for today, 185 had the first record with tokens=0 and the second with real values.

Suggested Fix

When a duplicate hash is encountered, compare token sums and keep the record with higher values instead of blindly skipping:

// In loadUsageRecords(), replace the simple skip with:
if (uniqueHash && processedHashes.has(uniqueHash)) {
    const newUsage = data.message.usage;
    const newSum = (newUsage.input_tokens || 0) + (newUsage.output_tokens || 0)
        + (newUsage.cache_creation_input_tokens || 0) + (newUsage.cache_read_input_tokens || 0);
    const existing = records[hashToIndex.get(uniqueHash)];
    const exUsage = existing.message.usage;
    const exSum = (exUsage.input_tokens || 0) + (exUsage.output_tokens || 0)
        + (exUsage.cache_creation_input_tokens || 0) + (exUsage.cache_read_input_tokens || 0);
    if (newSum > exSum) {
        records[hashToIndex.get(uniqueHash)] = data;
    }
    continue;
}

This also requires changing processedHashes from a Set to a Map (hash → array index).

Environment

  • Extension version: 2.0.0
  • Claude Code version: 2.1.159
  • Proxy: mimo (ccswitch)
  • OS: Linux (Manjaro, kernel 6.12)
  • VS Code: Remote SSH (vscode-server)

Verification

After applying the fix locally, today's data correctly shows 233 records / $7.98 instead of 4 records / $0.20.

Activity

  1. Carl723000 commented on Jun 2, 2026

    @Carl723000
    Collaborator

    Hi @zhaoxiao9302 — this is a fantastic report. The root cause analysis is correct, the suggested fix is the right shape, and the 185/188 stat is exactly the kind of evidence that makes a bug investigation go fast.

    Thank you!

    I'm planning the fix essentially as you proposed: switch processedHashes from Set<string> to Map<string, number> (hash →
    record index), and on a duplicate hash keep whichever record has the higher total token sum. That naturally handles proxy-induced "placeholder + real" pairs without needing to detect the proxy itself.

    I'll ship this in v2.0.1 this week. CHANGELOG will reference your issue with proper attribution.

    Some users (myself included) have been seeing under-reported DeepSeek and CC Switch usage that I now believe is the same bug — your report explains it. Doubly appreciated.

  2. Carl723000 commented on Jun 3, 2026

    @Carl723000
    Collaborator

    Hi @zhaoxiao9302 — great report, great fix suggestion. v2.0.1 ships exactly the approach you described: processedHashes is now a Map<string, number> (hash → record index) and on a duplicate hash we keep whichever record has the higher total token sum.

    Your 185/188 stat was very helpful for verifying the fix. CHANGELOG credits you.

    Closing. Let me know if you see anything unexpected.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions