GPU Glossary

TABLE OF CONTENTS

Device Hardware

CUDA (Device Architecture)

Streaming Multiprocessor

Special Function Unit

Load/Store Unit

Tensor Memory Accelerator

Streaming Multiprocessor Architecture

Texture Processing Cluster

Graphics/GPU Processing Cluster

Device Software

CUDA (Programming Model)

Streaming ASSembler

Parallel Thread eXecution

Compute Capability

Cooperative Thread Array

Thread Block Grid

Thread Hierarchy

Memory Hierarchy

CUDA (Software Platform)

CUDA C++ (programming language)

NVIDIA GPU Drivers

CUDA Driver API

NVIDIA Management Library

CUDA Runtime API

NVIDIA CUDA Compiler Driver

NVIDIA Runtime Compiler

NVIDIA CUDA Profiling Tools Interface

NVIDIA Nsight Systems

CUDA Binary Utilities

Performance Bottleneck

Arithmetic Intensity

Memory Bandwidth

Arithmetic Bandwidth

Warp Execution State

Pipe Utilization

Issue Efficiency

Streaming Multiprocessor Utilization

Warp Divergence

Branch Efficiency

Memory Coalescing

Register Pressure

/device-hardware/special-function-unit

What is a Special Function Unit?

SFU

The Special Function Units (SFUs) in Streaming Multiprocessors (SMs) accelerate certain arithmetic operations.

Notable for neural network workloads are transcendental mathematical operations, like exp, sin, and cos.

The Streaming Assembler (SASS) instructions associated with the SFUs generally begin with MUFU: MUFU.SQRT, MUFU.EX2. See this Godbolt link for sample assembly using the MUFU.EX2 instruction to implement the expf intrinsic in CUDA C++ .

Building on GPUs? We know a thing or two about it.

Modal is an ergonomic Python SDK wrapped around a global GPU fleet. Deploy serverless AI workloads instantly without worrying about quota requests, driver compatibility issues, or managing bulky ML dependencies. Deploy on GPUs

Something seem wrong?
Or want to contribute?

Click this button to
let us know on GitHub.

Load/Store Unit ?