Skip to content

fix(core): count blocks, not clusters, in the cooperative-launch check - #2578

Open
LeSingh1 wants to merge 1 commit into
NVIDIA:mainfrom
LeSingh1:launcher-cooperative-cluster-units
Open

fix(core): count blocks, not clusters, in the cooperative-launch check#2578
LeSingh1 wants to merge 1 commit into
NVIDIA:mainfrom
LeSingh1:launcher-cooperative-cluster-units

Conversation

@LeSingh1

@LeSingh1 LeSingh1 commented Aug 9, 2026

Copy link
Copy Markdown
Contributor

Problem

_check_cooperative_launch (cuda_core/cuda/core/_launcher.pyx:76-83) compares the requested grid against a residency limit expressed in thread blocks:

    max_grid_size = (
        kernel.occupancy.max_active_blocks_per_multiprocessor(prod(config.block), config.shmem_size) * num_sm
    )
    if prod(config.grid) > max_grid_size:
        ...
        raise ValueError(f"The specified grid size ({x} * {y} * {z}) exceeds the limit ({max_grid_size})")

But config.grid counts clusters, not blocks, whenever cluster is set. LaunchConfig says so itself (_launch_config.pyx:33-37, 42-44):

When cluster is specified, the grid parameter represents the number of clusters (not blocks). The hierarchy is: grid (clusters) -> cluster (blocks) -> block (threads).

and _to_native_launch_config implements exactly that (_launch_config.pyx:146-150):

        if self.cluster is not None:
            drv_cfg.gridDimX = self.grid[0] * self.cluster[0]
            drv_cfg.gridDimY = self.grid[1] * self.cluster[1]
            drv_cfg.gridDimZ = self.grid[2] * self.cluster[2]

So for LaunchConfig(grid=g, cluster=c, block=b, is_cooperative=True) the driver is asked for prod(g) * prod(c) blocks while the guard only inspects prod(g) — with cluster=(2, 2, 1) it under-counts by 4x. A cooperative launch that genuinely over-subscribes the device passes the check and then deadlocks or fails inside the driver, instead of getting the clean up-front ValueError this function exists to raise. The message compounds it: it prints cluster counts labelled "grid size" and compares them against a block limit.

Fix

Route the comparison through _cooperative_block_count(config), which applies the same cluster multiplication _to_native_launch_config does, and name the block count in the error message. No behaviour change when cluster is None.

Tests

test_cooperative_block_count_counts_blocks_not_clusters asserts _cooperative_block_count returns 6 for grid=(2, 3, 1) with no cluster and 24 for the same grid with cluster=(2, 2, 1), and that config.grid is still stored in cluster units. Device is mocked in cuda.core._launch_config exactly as the two neighbouring tests (test_launch_config_cooperative_unsupported, test_to_native_launch_config_cooperative) already do, so it runs on any GPU rather than needing Hopper+ and cooperative-launch support.

What I ran

Environment: macOS, no CUDA driver and no CUDA toolkit, so cuda.core cannot be built or imported here.

  • Did not run: the new test or the rest of cuda_core/tests/test_launcher.py — they need a built cuda.core and a GPU.
  • Ran: python -m py_compile, ruff check, ruff format --check on cuda_core/tests/test_launcher.py. ruff check reports the same single pre-existing I001 on the file's top import block as it does on main; no new findings.
  • Checked: no existing test asserts the old error message — "exceeds the limit" occurs exactly once in the tree, at the raise site itself.
  • Checked: config.cluster is always either None or a 3-tuple (cast_to_3_tuple in LaunchConfig.__init__), so prod(config.cluster) is always well defined on the branch that uses it.

Overlap note: #2066 also touches _launcher.pyx (adding a separate cluster-support check) and removes the Device() calls from LaunchConfig.__init__ that the neighbouring tests — and this new one — monkeypatch. It does not change the block/cluster accounting. If #2066 lands first this needs a trivial rebase; happy to do that.

`_check_cooperative_launch` compares the requested grid against a residency
limit expressed in thread blocks:

    max_grid_size = (
        kernel.occupancy.max_active_blocks_per_multiprocessor(...) * num_sm
    )
    if prod(config.grid) > max_grid_size:

but `config.grid` counts CLUSTERS, not blocks, whenever `cluster` is set.
LaunchConfig says so in its own docstring ("When cluster is specified, the
grid parameter represents the number of clusters (not blocks)"), and
`_to_native_launch_config` implements exactly that, multiplying grid by
cluster before filling in gridDimX/Y/Z.

So with e.g. cluster=(2, 2, 1) the guard under-counts by 4x: the driver is
asked for prod(grid) * prod(cluster) blocks while the check only looks at
prod(grid). A cooperative launch that genuinely over-subscribes the device
sails past the guard and fails (or deadlocks) inside the driver instead of
getting the clean ValueError this function exists to raise. The message was
misleading too -- it printed cluster counts as "grid size" and compared them
against a block limit.

Route the comparison through a small `_cooperative_block_count(config)`
helper that applies the same cluster multiplication as
`_to_native_launch_config`, and spell the block count out in the error.
@copy-pr-bot

copy-pr-bot Bot commented Aug 9, 2026

Copy link
Copy Markdown
Contributor

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@github-actions github-actions Bot added the cuda.core Everything related to the cuda.core module label Aug 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cuda.core Everything related to the cuda.core module

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant