Learn how imageblocks allow you to define and manipulate custom per-pixel data structures in high-bandwidth tile memory.
Imageblocks are tiles of structured image data stored in local memory that allow you to describe image data in tile memory that the A11 graphics processing unit (GPU) can manipulate efficiently. They’re deeply integrated with fragment processing and the A11’s tile shading stage, and are also available to compute kernels. Metal has always rendered to imageblocks on iOS devices, but Metal 2 on A11 expands on this functionality by exposing imageblocks as data structures that you have complete control over.
Imageblocks for Passing Data Between Fragment and Tile Stages
Figure 1 shows the A11 GPU architecture and how an imageblock can pass data between the fragment and tile stages of a render pass. Although threadgroup memory is suitable for unstructured data, an imageblock is recommended for image data.
An imageblock is a 2D data structure with width, height, and pixel depth. Each pixel in an imageblock can consist of multiple components, and each component can be addressed as its own image slice. Figure 2 shows an imageblock composed of three image slices representing albedo, specular, and normal components.
Imageblocks are available to both kernel and fragment functions and persist for the lifetime of a tile, across draws and dispatches. Imageblock persistence means that you can mix render and compute operations in a single rendering pass, with both accessing the same local memory. By keeping multiple operations within a tile, you can create sophisticated algorithms that remain in local GPU memory.
Your existing code automatically creates imageblocks that match your render attachment formats. However, you can also define your own imageblocks completely within your shader. Imageblocks that you define can be far more sophisticated than those created by render attachments; for example, they can include additional channels, arrays, and nested structures. Furthermore, imageblocks you define can be reused for different purposes across different phases of your computation.
Within a fragment shader, the current fragment only has access to the imageblock data associated with that fragment’s position in the tile. In a compute function, a thread can access all of the imageblock data. When using rendering with attachments, load and store actions continue as the means by which slices of data are read into and written from tile memory. However, if you’re using explicit imageblocks, you use compute functions to explicitly read from and write to device memory. Writes can often be performed as block transfers, taking advantage of memory hardware.