Skip to content

fix(arborist): skip legacy dependency generation for v3 lockfiles - #10130

Open
jeffwidman wants to merge 1 commit into
npm:latestfrom
jeffwidman:perf/skip-v3-legacy-metadata
Open

jeffwidman wants to merge 1 commit into
npm:latestfrom
jeffwidman:perf/skip-v3-legacy-metadata

Conversation

@jeffwidman

Copy link
Copy Markdown
Member

Avoid recursively constructing legacy dependency metadata that v3 lockfile serialization immediately discards.

Compatibility feedback requested

This is intentionally a draft: serialized v3 output matches in the differential checks, but the in-memory behavior is observably different.

  • After committing a v3 tree, tree.meta.data.dependencies is absent instead of containing a synthesized legacy dependency tree.
  • meta.get() can no longer recover a missing modern entry from that synthesized legacy tree. This includes inert/versionless nodes omitted from modern metadata, not just callers manually removing an entry.
  • Legacy input remains available before committing a complete tree. Reading older lockfiles and generating v1/v2 output are retained.

Six existing snapshots inspect the raw metadata object. Their legacy dependency blocks are deliberately removed to represent the proposed behavior; updating those snapshots is not evidence that the compatibility question is settled.

Is retaining synthesized legacy metadata after a v3 commit part of Arborist's supported API contract? If so, this needs an appropriate compatibility/release plan or a different implementation boundary. The README documents node.meta, so I do not want to assume its intermediate state is unobservable.

Implementation

Keep the root/header work in the legacy builder, then stop before traversing descendants when committing a v3 tree. Delete any existing legacy dependency tree as well: skipping regeneration alone can leave stale v2 metadata after an update.

The change is limited to v3. V1/v2 generation, older-format input handling, and hidden-lockfile behavior remain on their existing paths. The README points callers to commit() / toJSON() for data in the selected lockfile format.

Tests cover the skipped descendant work, root metadata, equivalent v1/v2 output, v2-to-v3 updates without stale legacy data, legacy-input lookup before committing, and the changed inert-node lookup behavior. Explicit legacy-format tests also preserve coverage of git metadata, aliases, mixed dependency flags, and cyclic links.

Performance evidence

In an offline, warmed large-workspace --package-lock-only writer benchmark, adding only this change to a baseline already containing adaptations of #10127, #10128, and npm/hosted-git-info#347 produced:

Median Baseline With this change Reduction
CPU time 6.898 s 6.130 s 11.1%
Wall time 5.677 s 5.124 s 9.7%

These are three measured alternating pairs after one warmup per arm. All 4,686 manifests/lockfiles plus the npm configuration file were byte-identical across all eight runs, and normalized command output matched. Separate instrumentation showed 4,358 -> 0 generated legacy entries. A public synthetic 128-dependency graph also reduced descendant bundle-membership reads from 256 -> 0 for v3, while v1/v2 retained their traversal.

The benchmark runtime was Node 24.22.0, npm 11.17.0, and Arborist 9.8.0. This PR is independently based on current latest (npm 12.2.0 / Arborist 10.0.3), not stacked on the other PRs. These measurements describe the incremental native writer benefit after the earlier optimizations, not this PR alone versus unmodified npm, nor end-to-end Dependabot job savings. A dry-run checker does not serialize a lockfile and is not expected to benefit.

A tiny control over 21 pairs had medians of +2.9% CPU and +0.4% wall time; paired bootstrap intervals crossed zero. That result is inconclusive, not proof of regression-free performance.

Validation

On the current upstream-based source, with the existing npm development dependencies available:

  • node bin/npm-cli.js test --workspace=@npmcli/arborist --ignore-scripts -- --jobs=4 — passed all 59 test files; 100% statements, branches, functions, and lines.
  • eslint workspaces/arborist/lib/shrinkwrap.js workspaces/arborist/test/shrinkwrap.js — passed.
  • git diff --check — passed.

Additional differential checks matched 72 core cases, with raw-state/fallback differences recorded separately rather than hidden. An offline native-command matrix matched complete files and normalized logs across 24 commands covering workspace changes and lockfile versions. The final publication source was rechecked against the 72-case results, including the explicitly changed fallback observations.

The full npm CLI suite was not run.

References

Preserve root metadata while avoiding discarded legacy descendant work. Clear stale legacy input after committing a v3 tree and explicitly test the proposed in-memory compatibility boundary.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant