ring-allreduce

Here are 5 public repositories matching this topic...

czhao-dev / ring-allreduce-perf-study

From-scratch Ring-AllReduce on CPU (C++20) and GPU (CUDA, multi-device): real multi-GPU benchmarks, Nsight profiling, and a bandwidth-optimal ring implementation built without NCCL.

benchmarking hpc parallel-computing distributed-computing cuda gpu-computing multi-gpu cpp20 nvlink collective-communication nsight-systems ring-allreduce

Updated Jul 31, 2026
Cuda

abho7 / distributed-training-framework

Star

From-scratch data-parallel training framework with fault injection and continuous correctness verification. Two sync strategies, two execution engines cross-validated bitwise, 600-run chaos sweep.

python distributed-systems numpy sgd fault-injection data-parallelism chaos-engineering distributed-training correctness-verification ring-allreduce

Updated Aug 30, 2026
Python

asp53826 / dist-train

Star

Distributed training from scratch: ring all-reduce built from send/recv, data/tensor/pipeline parallelism, and honest scaling efficiency measurements

pytorch distributed-training machine-learning-systems pipeline-parallelism collective-communication tensor-parallelism ring-allreduce

Updated Aug 13, 2026
Python

MrMiniMIze / mini-nccl

Star

Distributed training from first principles: ring, tree and halving-doubling all-reduce over raw TCP, with data-parallel, fully-sharded, tensor and pipeline parallelism composed into 2D and 3D meshes

gpu pytorch high-performance-computing distributed-training nccl machine-learning-systems collective-communication ring-allreduce

Updated Aug 5, 2026
Python

Olajide-Badejo / Ring-AllReduce

Star

Ring-allreduce built from scratch in C++20 over raw MPI point-to-point primitives, benchmarked against MPI_Allreduce across 25 message sizes and 15 process counts, with a fitted alpha-beta cost model explaining the measured bandwidth gap. Includes a reproducible LaTeX report and an engineering postmortem.

benchmarking cmake cpp hpc mpi parallel-computing distributed-computing scientific-computing high-performance-computing performance-analysis openmpi allreduce cpp20 collective-communication ring-allreduce

Updated Aug 5, 2026
C++

Add this topic to your repo

To associate your repository with the ring-allreduce topic, visit your repo's landing page and select "manage topics."

Learn more