Skip to content

[WIP] [NV] Add a dsr1 B300 trt agg configuration - #641

Closed
camiloamoreno wants to merge 1 commit into
mainfrom
nv/dsr1-b300-trt-agg
Closed

[WIP] [NV] Add a dsr1 B300 trt agg configuration#641
camiloamoreno wants to merge 1 commit into
mainfrom
nv/dsr1-b300-trt-agg

Conversation

@camiloamoreno

Copy link
Copy Markdown
Collaborator

No description provided.

@camiloamoreno
camiloamoreno requested a review from a team as a code owner February 5, 2026 01:28
@camiloamoreno
camiloamoreno marked this pull request as draft February 5, 2026 01:29
@camiloamoreno
camiloamoreno deleted the nv/dsr1-b300-trt-agg branch February 5, 2026 18:03
Oseltamivir added a commit that referenced this pull request Aug 5, 2026
…lamp

Fixes the defect instead of stepping around it. b368eae clamped the low-latency ladder to 128
so the corrupt 256 rung was not measured; the corruption itself was still in the kernels we
publish numbers from, at roughly 1.5-3.3% per invocation on every Blackwell SKU.

The fix was already upstream. DeepEP PR #642 adds a CTA-scope fence.proxy.async.shared::cta
before mbarrier_arrive(empty_barriers[stage_idx]) in LOW_LATENCY_COMBINE_RECV, so the combine
consumer's shared-memory reads retire before the stage is declared empty and the producer's next
TMA load refills it -- signalling empty too early is what let one output row be assembled from
two tiles, which matches the observed signature exactly (norm preserved to 4 s.f., 16-40% of
elements wrong). It closed #621, the same race found independently on NVL72.

COLLX_DEEPEP_V2_COMMIT was fa8a9b16, the head of the pre-merge PR #605 branch, cut before #642
landed. Its one unique commit was the #630 single-node V2 init fix, which main carries as
56169594e, so moving to main loses nothing and also picks up #715 (system-scope release before
the GIN barrier when scale-up spans NVLink and RDMA), #688 (NCCL Device API: runtime version for
ncclDevCommCreate), #178 (SM90), #641, and #640/#627 upstream's own libnccl and SO-name handling.
Verified before bumping that main still exposes every API this adapter calls: legacy Buffer
kwargs incl. allow_mnnvl, get_low_latency_rdma_size_hint, low_latency_dispatch/combine,
ElasticBuffer's full kwarg set, topk_idx_t, and the fp8 cast helpers.

rewrite_deepep_v2 now succeeds when the source already matches, because main contains #640 and
the old count(old) == 1 assertion would have aborted every leg at repository-stage. Verified
against all four source states: old form rewrites, fixed form is a no-op, absent and duplicated
forms still fail.

_LL_LADDER_CAP returns to 256. The two constants stay separate: the receive must not be sized
from max(ladder), or clamping the ladder would change the footprint that drives transport traffic
and fp8 dequant volume. The cap test now asserts ladder <= buffer -- strict inequality encoded
the workaround, not an invariant. The backend cache key includes the pin, so this forces a
rebuild rather than reusing the old build.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

Development

Successfully merging this pull request may close these issues.

1 participant