GPU Projects

CUDA benchmarks and numerical linear-algebra solvers with reproducible CMake builds.

This repository collects GPU implementations of sparse matrix-vector multiplication, sparse matrix-matrix multiplication, stationary and Krylov iterative methods, FP64 GEMM, and adapters for four sparse direct solvers. Most subprojects share a single top-level CMake build with only CUDA Toolkit dependencies; the SpGEMM benchmark and the four sparse direct-solver adapters are standalone subprojects built separately.

Quick start

Prerequisites: CMake 3.24+, a C++17 compiler, the CUDA Toolkit, and an NVIDIA GPU.

git clone https://github.com/2002WYT/GPU_projects.git
cd GPU_projects
cmake -S . -B build -DCMAKE_BUILD_TYPE=Release
cmake --build build -j
ctest --test-dir build --output-on-failure

The default architecture is sm_70 for the NVIDIA V100 used in the published benchmarks. Override it for another GPU, for example:

cmake -S . -B build \
  -DCMAKE_BUILD_TYPE=Release \
  -DCMAKE_CUDA_ARCHITECTURES=80

CTest runs small GPU smoke tests for SpMV, Jacobi, multicolor Gauss–Seidel, and CG. A CUDA-capable GPU must be visible when the tests run.

Projects

Project	What it demonstrates	Default target
SpMV	Five CSR SpMV kernels, correctness checks, timing, GFLOP/s, and effective bandwidth	spmv_compare
Jacobi	Cooperative Groups, device-side convergence, and double buffering	jacobi_solver
Gauss–Seidel	Graph coloring, SOR, CUB reduction, and CUDA Graph replay	gauss_seidel_solver
Conjugate Gradient	CSR SpMV, dot products, AXPY/AXPBY, and true-residual verification	cg_solver
GEMM	FP64 kernel autotuning, selected kernels, Matrix Market input, and cuBLAS comparison	gemm_autotune, gemm_selected, gemm_mtx
SpGEMM	Hand-written CUDA SpGEMM vs NVIDIA AMGX, dual-backend timing and in-process CPU verification	bench_custom, bench_amgx (standalone)
Sparse solver benchmarks	SuperLU_DIST, PanguLU, cuDSS, and STRUMPACK adapters	opt-in

Detailed implementation notes and benchmark methodology live inside each project directory:

• SpMV implementation notes and V100 results
• Jacobi solver notes
• Gauss–Seidel/SOR notes
• Conjugate Gradient notes
• SpGEMM benchmark notes and V100 results
• Four-solver benchmark report

Build options

Every first-party project has a top-level switch:

Option	Default
GPU_PROJECTS_BUILD_SPMV	ON
GPU_PROJECTS_BUILD_JACOBI	ON
GPU_PROJECTS_BUILD_GAUSS_SEIDEL	ON
GPU_PROJECTS_BUILD_CG	ON
GPU_PROJECTS_BUILD_GEMM	ON
GPU_PROJECTS_BUILD_SOLVERS	OFF

For example, build only SpMV and CG:

cmake -S . -B build \
  -DGPU_PROJECTS_BUILD_SPMV=ON \
  -DGPU_PROJECTS_BUILD_JACOBI=OFF \
  -DGPU_PROJECTS_BUILD_GAUSS_SEIDEL=OFF \
  -DGPU_PROJECTS_BUILD_CG=ON \
  -DGPU_PROJECTS_BUILD_GEMM=OFF
cmake --build build -j

SpGEMM is the exception: it has no top-level GPU_PROJECTS_BUILD_SPGEMM switch and is not part of the top-level build. Build it standalone from its own directory — see SpGEMM (standalone build) below.

Run examples

./build/spmv_compare 200000 20 3 1 256
./build/jacobi_solver 10000
./build/gauss_seidel_solver 100000
./build/cg_solver 100000
./build/GEMM/gemm_autotune

Run ./build/spmv_compare --help for the SpMV matrix modes and benchmark arguments. The GEMM executables document their accepted dimensions and Matrix Market inputs in their --help output.

SpGEMM (standalone build)

The SpGEMM benchmark compares a hand-written CUDA sparse matrix-matrix multiplication kernel (custom) against NVIDIA's AMGX library (amgx) for C = A·Bᵀ, with dual-backend timing and in-process CPU verification (no .csr dumps). It is not part of the top-level CMake build — it has its own CMakeLists.txt and build script. The custom backend needs only the CUDA Toolkit; the amgx backend additionally requires NVIDIA AMGX 2.5.0.

cd SpGEMM
# build.sh builds AMGX first, then this project (when AMGX_ROOT is set)
AMGX_ROOT=/path/to/AMGX-2.5.0 CUDA_ARCHITECTURES=70 bash scripts/build.sh

# ...or invoke cmake directly; bench_custom builds with no AMGX
cmake -S . -B build -DCMAKE_BUILD_TYPE=Release -DCMAKE_CUDA_ARCHITECTURES=70
cmake --build build --parallel

Run a single matrix:

./build/bench_custom --a path/to/A.mtx --b-mode transpose --warmup 2 --repeat 5
./build/bench_amgx   --a path/to/A.mtx --b-mode transpose --warmup 2 --repeat 5

See the SpGEMM README for the bin-classified hash/sort kernel design, the dual-backend verification procedure, and the full 48-matrix V100 benchmark. Note that scripts/build.sh and run_bench.sh ship with absolute paths from the author's local environment; override the AMGX paths via environment variables and point --matrix-dir at your own matrices to reproduce.

Optional sparse direct solvers

The four external adapters are disabled by default because they require separate installations. Enable the group and pass installation prefixes as CMake cache variables instead of editing repository files:

cmake -S . -B build \
  -DCMAKE_BUILD_TYPE=Release \
  -DGPU_PROJECTS_BUILD_SOLVERS=ON \
  -DSUPERLU_DIST_ROOT=/path/to/superlu_dist \
  -DPANGULU_ROOT=/path/to/pangulu \
  -DCUDSS_ROOT=/path/to/cudss \
  -DSTRUMPACK_ROOT=/path/to/strumpack \
  -DSCALAPACK_ROOT=/path/to/scalapack \
  -DOPENBLAS_ROOT=/path/to/openblas
cmake --build build -j

Individual adapters can be disabled when only part of the dependency stack is installed:

cmake -S . -B build \
  -DGPU_PROJECTS_BUILD_SOLVERS=ON \
  -DGPU_SOLVERS_BUILD_SUPERLU_DIST=OFF \
  -DGPU_SOLVERS_BUILD_PANGULU=OFF \
  -DGPU_SOLVERS_BUILD_CUDSS=ON \
  -DGPU_SOLVERS_BUILD_STRUMPACK=OFF \
  -DCUDSS_ROOT=/path/to/cudss

The resulting executables are placed in build/bin. A small Matrix Market input is included at Four sparse solvers/files/AA1.mtx.

mpirun -n 1 ./build/bin/test_slu "Four sparse solvers/files/AA1.mtx"
mpirun -n 1 ./build/bin/test_pangulu "Four sparse solvers/files/AA1.mtx"
mpirun -n 1 ./build/bin/test_cudss "Four sparse solvers/files/AA1.mtx"
mpirun -n 1 ./build/bin/test_strumpack "Four sparse solvers/files/AA1.mtx"

The adapters share the repository's mtx_reader.h, which accepts real, integer, and pattern coordinate matrices and expands Matrix Market symmetric storage into zero-based full CSR.

Performance highlights

The following results were measured on an NVIDIA Tesla V100-SXM2-16GB with n = 2,000,000. See the full SpMV report for the test procedure and raw comparisons.

Regular row-length distribution

Kernel	Best block size	Average time	GFLOP/s	Speedup over CSR Scalar
CSR Vector	512	5.13 ms	32.61	3.04×
CSR Adaptive	512	5.13 ms	32.61	3.04×
LightSpMV	256	5.20 ms	32.18	3.00×
PCSR	128	7.71 ms	21.72	2.02×
CSR Scalar	128	15.61 ms	10.72	1.00×

Heavy-tailed row-length distribution

Kernel	Best block size	Average time	GFLOP/s	Speedup over CSR Scalar
CSR Adaptive	512	7.60 ms	28.86	2.00×
CSR Vector	512	7.86 ms	27.92	1.94×
LightSpMV	256	7.94 ms	27.65	1.92×
PCSR	512	11.85 ms	18.52	1.28×
CSR Scalar	128	15.22 ms	14.42	1.00×

These measurements show why the repository reports both matrix structure and kernel throughput: CSR Adaptive becomes the best option when a small number of rows contain much more work.

Benchmarking conventions

• Kernels run warm-up iterations before timing.
• CUDA Events measure GPU work.
• Repeated runs report average execution time.
• Numerical programs report correctness alongside performance.
• Iterative solvers use the relative residual ||b-Ax||₂ / ||b||₂.
• Direct-solver reports distinguish reorder, factorization, solve, and external wall-clock time where the APIs expose those phases.

Example profiling commands:

nsys profile --trace=cuda,nvtx,osrt --stats=true \
  -o spmv_nsys ./build/spmv_compare 200000 10 2 1 256

ncu --section full -o spmv_ncu \
  ./build/spmv_compare 200000 1 0 1 256

Repository layout

GPU_projects/
├── CMakeLists.txt          # top-level build (SpMV, Jacobi, Gauss–Seidel, CG, GEMM)
├── spmv/
├── jacobi/
├── gauss-seidel/
├── CG/
├── GEMM/
├── SpGEMM/                  # standalone build — its own CMakeLists.txt + build.sh
└── Four sparse solvers/     # standalone build — opt-in, external dependencies

Scope and limitations

• The default examples generate synthetic matrices; only selected programs accept Matrix Market input.
• The iterative methods are educational and benchmarking implementations, not a replacement for a production solver package.
• The published performance numbers describe the recorded V100 software/hardware setup; re-run the benchmarks before comparing another GPU or CUDA version.
• Third-party sparse solvers retain their own build requirements and licenses.

License

No repository-wide license has been selected yet. Until a license is added, the source is publicly visible but reuse rights are not granted automatically.