Skip to content

01_introduction.adoc

Latest commit

 

History

History
33 lines (19 loc) · 2.36 KB

01_introduction.adoc

File metadata and controls

33 lines (19 loc) · 2.36 KB

Subgroup Operations: The Hidden Power

Introduction

In the previous chapters, we looked at how to share data between hundreds or even thousands of threads in a workgroup using Shared Memory (LDS) and explicit barriers. While powerful, this approach has a significant cost: every barrier forces the GPU to pause and wait, and every access to shared memory consumes precious bandwidth.

What if you could share data even faster? What if you could exchange values without ever touching VRAM or even the LDS? This is where Subgroup Operations come in. They are the "secret sauce" behind many of the most highly optimized GPU algorithms in existence today.

Why Subgroups Matter

A Subgroup is a hardware-level bundle of threads (typically 32 on NVIDIA/Intel or 32/64 on AMD) that execute in perfect lockstep on the same SIMD unit. Because the hardware already physically synchronizes these threads, they can communicate with each other using specialized instructions that are often as fast as a single clock cycle.

In this chapter, we’ll explore the hidden power of subgroups:

  1. Cross-Invocation Communication: Utilizing Subgroup Shuffles, Broadcasts, and Arithmetic to exchange data directly through registers, bypassing memory entirely.

  2. Subgroup Partitioning: Implementing "Ballot" and "Match" operations to perform complex branching and data filtering across the entire bundle.

  3. Non-Uniform Indexing: Leveraging modern Vulkan features to safely access arrays of resources that might be different for every thread in the subgroup.

Moving Beyond Barriers

Subgroup operations allow you to write "barrier-free" kernels for small-scale data exchange. Instead of having every thread in a workgroup wait at a barrier just to share a single float, you can use a subgroup shuffle to pass that value instantly.

This leads to:

  • Higher Performance: No pipeline stalls from waiting threads.

  • Lower Latency: Data exchange happens at register speeds.

  • Greater Flexibility: Algorithms can be more "wave-aware," adapting to the hardware’s native execution width.

We’ll start by looking at the fundamental building blocks of subgroup communication: Shuffles and Broadcasts.