Repository navigation
Set the memory barrier scope of SASS MEMBAR instructions - #580
Draft
karthikbhata97 wants to merge 1 commit into
Draft
karthikbhata97 wants to merge 1 commit into
karthikbhata97 wants to merge 1 commit into
Conversation
GPGPU-Sim now makes a GPU- or system-scope memory barrier wait for the warp's store acks, keeps the old register-only wait for a CTA-scope one, and treats a barrier of unknown scope as GPU scope. Parse the scope from the opcode so trace-driven MEMBAR.CTA / MEMBAR.SC.CTA / MEMBAR.ALL.CTA keep the cheap CTA-scope wait, and MEMBAR.GL / MEMBAR.SC.GPU and MEMBAR.SYS / MEMBAR.SC.SYS get GPU and system scope. Depends on the gpgpu-sim_distribution change that adds inst_t::set_membar_scope(). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Companion to accel-sim/gpgpu-sim_distribution#152, which must be merged first: this change uses
inst_t::set_membar_scope(), which that PR adds. The accel-sim-framework CI builds the same-named branch of thegpgpu-sim_distributionfork,membar-wait-for-store-acks.Problem
GPGPU-Sim's
shader_core_ctx::warp_waiting_at_mem_barrier()never waited for a warp's outstanding stores to be acknowledged, so GPU- and system-scope fences were modeled as too cheap. The gpgpu-sim PR fixes this:In trace mode nothing sets the scope today. Without this PR, every SASS
MEMBAR, includingMEMBAR.CTA, would get the conservative wait.Fix
trace_warp_inst_t::parseSASSInstruction()sets the scope from the opcode tokens:MEMBAR.CTA,MEMBAR.SC.CTA,MEMBAR.ALL.CTAMEMBAR.GL,MEMBAR.SC.GPU,MEMBAR.ALL.GPUMEMBAR.SYS,MEMBAR.SC.SYS,MEMBAR.ALL.SYSThe spelling is
MEMBAR.{CTA,GL,SYS}before Volta andMEMBAR.{SC,ALL}.{CTA,GPU,SYS}from Volta on. CUDA 12.4 emitsMEMBAR.SC.CTA/MEMBAR.SC.GPU/MEMBAR.SC.SYSfor__threadfence_block()/__threadfence()/__threadfence_system()on sm_70 and sm_86.Tests
Built with CMake as in CI (CUDA 12.4), against
gpgpu-sim_distributiondevplus the companion PR.gpu-simulator/format-code.sh(clang-format 16.0.6) leaves the tree unchanged.Microbenchmark. 80 CTAs x 256 threads, each doing 16 iterations of a global store followed by a fence. It was traced with the NVBit tracer on an L40S as an sm_86 binary and run on SM86_RTX3070. The table shows
gpu_tot_sim_cycle:devMEMBAR.SC.CTAMEMBAR.SC.GPUMEMBAR.SC.SYSWith both PRs, the no-fence and
MEMBAR.SC.CTAruns give the same output asdev, line for line.rodinia_2.0-ftV100 traces (the CI set) on SM7_QV100: the output is identical todevin all 10 apps. These traces contain noMEMBAR.Measured effect on real workloads. On a research fork of Accel-Sim, a prototype of this change changed the simulated time of fence-heavy kernels by up to 24%.
🤖 Generated with Claude Code