Skip to content

Generalize runtime memory sampling for non-CUDA backend models #126

Closed

Description

@Silas-Asamoah

Summary

Stormlog should generalize its runtime memory-sampling layer so non-CUDA backend models can participate in tracking and telemetry without pretending they expose CUDA-style allocator counters.

Why

Today the PyTorch-side collector and tracker path is still CUDA/MPS-shaped:

  • stormlog/device_collectors.py requires normalized samples with allocated_bytes, reserved_bytes, active_bytes, and inactive_bytes.
  • stormlog/tracker.py only accepts cuda or mps devices for GPU tracking.
  • stormlog/profiler.py is hard-wired to torch.cuda.* for bounded profiling.
  • stormlog/telemetry.py assumes allocator-centric event fields are always available.

That works for CUDA/ROCm/MPS, but it is too narrow for runtimes such as TorchTPU/XLA or JAX, where the stable signals may be closer to device used/free/total memory than allocator-reserved/active/inactive counters.

Proposed direction

Introduce a backend capability model that separates:

  • allocator-level counters that are present only on backends that actually expose them
  • device-level capacity/usage counters that are available across more runtimes
  • feature flags for unsupported capabilities such as native allocator history, fragmentation analysis, or allocator attribution

This should keep the shared session/telemetry/TUI artifact layer intact while making the runtime-facing collectors and trackers honest about what they can measure.

Scope

  • Generalize DeviceMemoryCollector so backends can report partial capability sets without faking CUDA allocator semantics.
  • Update tracker export paths to tolerate missing allocator-only fields where the backend does not expose them.
  • Preserve session-aware telemetry, distributed identity, and TUI loading.
  • Add explicit capability metadata so CLI/TUI/reporting can say when allocator-centric analysis is unavailable.

Non-goals

  • Full feature parity for every backend in one change.
  • Rewriting CUDA-native attribution/history to work on non-CUDA runtimes.
  • Adding TorchTPU or JAX support directly in this issue.

Acceptance criteria

  • The collector/tracker contract can represent backends that only expose device-level memory info.
  • Telemetry exports and loaders remain stable for existing CUDA/ROCm/MPS workflows.
  • CLI/TUI surfaces degrade explicitly when allocator-native features are unavailable.
  • This work leaves a clean path for follow-up runtime issues such as TorchTPU and JAX.

Activity

  1. self-assigned this
    on Jul 13, 2026
  2. Silas-Asamoah commented on Oct 2, 2026

    @Silas-Asamoah
    OwnerAuthor

    Closing as completed: #229 delivered this and shipped in v0.3.10. That PR said "Closes #126", but it merged into release/dev, so the keyword didn't fire.

    What it covered against the acceptance criteria:

    • Backends that expose only device-level memory: DeviceMemoryCapabilities declares per-counter and allocator-feature support, DeviceMemorySample counters are nullable, and MemoryTracker(..., collector=...) accepts an injected collector.
    • Stable telemetry for existing workflows: canonical TelemetryEventV4 exports with capability metadata, plus conservative v2/v3 and legacy upgrades. CUDA, ROCm and MPS sampling is unchanged.
    • Explicit degradation in the CLI and TUI: device-only runs plot device-used memory, show allocator fields as N/A, and report a reason instead of a finding for unsupported fragmentation, gap and attribution analyses.
    • A clean path for follow-ups: TorchTPU remains open as Add TorchTPU support once the public runtime is released #125.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions