Skip to content

mergeResults silently overwrites duplicate output custom_id values #1

Open

Description

@yoffe8-droid

When multiple result files are concatenated, two records can share a custom_id. mergeResults silently keeps only the last one. A success followed by an error becomes retryable; reversing the same records removes that ID from failures. The caller receives no duplicate indication.

This is a synthetic reproduction against the current src/merge.js inspected on main (file commit 0e5b77cf232b0053823805ee5b77e09110c026b3), run locally with Node v24.22.1; no provider calls or production data.

import { mergeResults } from "./src/merge.js";
const inputs = [{ id: "q1", prompt: "Synthetic test" }];
const ok = { custom_id: "q1", response: {
  status_code: 200,
  body: { choices: [{ message: { content: "ok" } }] }
}};
const error = { custom_id: "q1", error: { message: "synthetic failure" } };
console.log(mergeResults(inputs, [ok, error], { provider: "openai" }).failures);
// ["q1"]
console.log(mergeResults(inputs, [error, ok], { provider: "openai" }).failures);
// []

Expected: reject ambiguous duplicate result IDs, or expose them explicitly with a documented resolution policy. Silent last-record-wins can hide an accidentally concatenated retry attempt. The existing unique, unordered-results behavior should remain unchanged.

A minimal fail-fast option before parsed.set is:

if (parsed.has(line.custom_id)) {
  throw new Error(`duplicate result custom_id: ${line.custom_id}`);
}

I tested that proposed guard in eight targeted local tests across both supported providers: conflicting duplicates in either order, identical duplicates, duplicate unexpected IDs, and preservation of unique unordered results plus failed/missing/unexpected classifications. All eight passed. These are targeted checks, not a claim that the upstream suite was run. A structured duplicate report may be preferable if combining attempts is intended.

Prepared with AI assistance; the reproduction and targeted test results were checked locally.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions