What are Registers?
Registers are the memory of the memory hierarchy associated with individual threads (left). Modified from diagrams in NVIDIA's CUDA Refresher: The CUDA Programming Model and the NVIDIA CUDA C++ Programming Guide .
At the lowest level of the memory hierarchy are the registers, which store information manipulated by a single thread .
The values in registers are generally stored in the register file of the Streaming Multiprocessor (SM) , but they can also spill to the global memory in the GPU RAM at a substantial performance penalty.
As when programming CPUs, these registers are not directly manipulated by high-level languages like CUDA C . They are only visible to lower-level languages like Parallel Thread Execution (PTX) or Streaming Assembler (SASS) and so are typically managed by a compiler like nvcc . Among the compiler's goals is to limit the register space used by each thread so that more thread blocks can be simultaneously scheduled into a single SM .
The registers used in the PTX instruction set architecture are documented here . The registers used in SASS are not, to our knowledge, documented.