Three FP64 square matrix multiply kernels over the same N×N problem, with intentionally different performance:
| Kernel | Role |
|---|---|
matmul_naive |
One output per thread; inner k loop reads A and B from global memory every iteration. |
matmul_mid |
Same structure as naive, but __restrict__ pointers and partial inner-loop unrolling (still global-memory bound). |
matmul_fast |
16×16 LDS-tiled GEMM: staging through __shared__, __syncthreads, padded tiles to reduce bank conflicts. |
Constants live at the top of main.hip: kN, kBlock (naive/mid), kTile (LDS tiled kernel). Correctness uses a small CPU spot-check on random matrix elements.
The project is built with -g -gdwarf-4 so rocprofv3 ATT / stats_*.csv can map instructions back to source lines (see AMD doc below).
module load rocm/7.14.0-gfx94X # or your ROCm 7.14 gfx94X module
make clean && makeOverride GPU arch: make ARCH=gfx942 (default in Makefile).
make runOfficial guide: Using thread trace (develop branch; your installed ROCm may differ). A short excerpt is in docs/using-thread-trace.md.
Default-style collection (run matmul under Slurm; example inside salloc):
rocprofv3 --att -d -- srun -n 1 ./matmulAMD Instinct (SQ activity streamed into ATT buffer):
rocprofv3 --att --att-activity 8 -d -- srun -n 1 ./matmulFrom login without an interactive shell on the node, pass your allocation:
rocprofv3 --att --att-activity 8 -d -- srun -n 1 --jobid="${myjob}" ./matmulTrace one kernel (HIP names are mangled; substrings like the following usually work—confirm with one --kernel-trace run if needed):
rocprofv3 --att --att-activity 8 -d --kernel-include-regex matmul_fast -- srun -n 1 ./matmulReplace matmul_fast with matmul_naive or matmul_mid to compare instruction/stall profiles.
After the run, rocprofv3 typically leaves:
stats_*.csv— per-instruction summary (latency, stall, idle; Source column when built with-g).ui_output_agent_*_dispatch_*— JSON for ROCprof Compute Viewer.- Raw
.att/.out— SQ thread trace and code objects (see AMD doc).
Add these paths to .gitignore (already listed) so they are not committed by mistake.
Open the generated ui_output_agent_* directory in ROCprof Compute Viewer (see ROCm / rocprof-compute documentation for the version that matches your install).