Skip to content

Flaky test: nostr-cid-vm.test.js JWK-y corruption can yield a valid curve point #438

@melvincarvalho

Description

@melvincarvalho

Symptom

test/nostr-cid-vm.test.js:223 (it('rejects a JWK with the right x but wrong y (curve-point integrity)', ...)) intermittently fails with:

ERR_ASSERTION
  actual: 'https://alice.example.com/profile/card.jsonld#me'
  expected: 'did:nostr:<hex pubkey>'
  operator: 'strictEqual'

The test expects auth to fall back to did:nostr: when the JWK presents a corrupted y coordinate — but on some runs the WebID upgrade succeeds anyway, meaning the corrupted JWK was accepted as a valid curve point.

Root cause hypothesis

Each test run generates a fresh secp256k1 keypair (generateSecretKey() in the before/beforeEach). The corruption applied to y is:

const badJwk = {
  ...goodJwk,
  y: goodJwk.y.slice(0, -1) + (goodJwk.y.endsWith('A') ? 'B' : 'A')
};

— flipping the last base64url character of y. For a random base point P = (x, y), that produces a value y' that's almost always not a valid curve point and therefore correctly rejected. But not always: each x has exactly two valid y solutions (y and p − y); if the single-character flip happens to round-trip to the negation −y mod p, the corrupted JWK is still on the curve and the test fails as observed.

The probability is tiny per run, but reproducible over many CI runs / local re-runs. This makes it a true flake — same code, same test, different outcome.

Reproduction

Run the suite repeatedly:

for i in {1..50}; do npm test 2>&1 | grep -E "^ℹ fail" | grep -v "fail 0" && echo "FLAKED ON RUN $i"; done

Expect at least one failure within tens of runs.

Fix options

  1. Deterministic keypair. Hardcode a known-good secretKey/pubkey pair for this test — same behaviour every run, no probabilistic anything. Simplest. Loses a tiny bit of "tries lots of values" coverage but the test isn't fuzzing the curve impl.
  2. Provably-invalid corruption. Replace y with a value that cannot be on the curve (e.g. all-zeroes — (x, 0) is only on secp256k1 for x = 0). Or set y to MAX_FIELD_VAL + 1. Removes the y == −y collision path entirely.
  3. Pre-check the corruption. After producing badJwk, run a curve-point validation against it; if it happens to be valid, regenerate. Effectively rejection sampling. Works but feels like working around the symptom.

(1) is the smallest, clearest change. (2) is more thorough — corruption stays semantically equivalent to "not on the curve" without depending on the random keypair.

Severity

Low — test only, doesn't affect runtime behaviour. Annoying because it can fail any unrelated PR's CI run, eroding signal.

Out of scope

  • A general audit of other tests that use generateSecretKey() for similar randomness assumptions (test/nostr-event.test.js etc.). Worth a follow-up sweep but not urgent.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions