Article

Transferring Data Between Connected GPUs

Use high-speed connections between GPUs to transfer data quickly.

Overview

Starting with macOS 10.15, some Mac systems directly connect GPUs to each other, allowing you to quickly transfer data between them. These connections are not only faster, but they also avoid using the memory bus between the CPU and GPUs, leaving it available for other tasks. If your app uses multiple GPUs, test to see if they're connected, and when they are, use the transfer mechanism described here.

When GPUs are connected to each other, they're said to be in the same peer group. You determine whether a GPU is in a peer group by reading the device object’s peerGroupID property. A nonzero value indicates that the GPU is in a peer group.

func isMemberOfAnyPeerGroup(_ device: MTLDevice ) -> Bool
{
    return device.peerGroupID != 0
}

GPUs in the same peer group share the same peer group ID. 

func areMembersOfSamePeerGroup(_ deviceA:MTLDevice,_ deviceB: MTLDevice) -> Bool
{
    return isMemberOfAnyPeerGroup(deviceA) &&
           deviceA.peerGroupID == deviceB.peerGroupID
}

You can get the list of all devices associated with a peer group by filtering on this ID.

func devicesInPeerGroup(_ peerGroupID: UInt64) -> [MTLDevice]
{
    if peerGroupID == 0
    {
        return []
    }
    let allDevices = MTLCopyAllDevices()
    return allDevices.filter({$0.peerGroupID == peerGroupID})
}

Copy Resources to the GPU that Needs to Access Them

In Metal, resources are created by device objects, and are always associated with the device object that created them. Peer groups don't change that association. If a resource is associated with a device object, and you want to access it on another device object, you need to copy the data to a resource associated with the second device object.

To copy data between members of a peer group, make a remote view on the second GPU that’s connected to the resource you want to copy. A remote view is a resource object that contains no storage of its own; it references the storage on the original GPU. You can only use remote views to copy data; using them in other Metal commands results in an error.

Create a Remote View of a Resource

To create a remote view of a resource, the device object you make the resource view on must share the same peerGroupID as the device object that created the resource. In addition, the resource must use the MTLStorageMode.private storage mode or be backed by an IOSurface.

To create a buffer view, call the makeRemoteBufferView(_:) method:

let remoteBufferView = sourceBuffer.makeRemoteBufferView(otherDevice)

Similarly, to create a texture view, call the makeRemoteTextureView(_:) method.

let remoteTextureView = sourceTexture.makeRemoteTextureView(otherDevice)

Copy Data Between Connected GPUs

Create a MTLBlitCommandEncoder and encode a copy command. The source for this copy command is the remote view object:

if let blitEncoder = commandBuffer.makeBlitCommandEncoder()
{
    blitEncoder.copy(from: remoteBufferView, sourceOffset: 0,
                       to: destinationBuffer, destinationOffset: 0,
                     size: remoteBufferView.allocatedSize)
        
    blitEncoder.copy(from: remoteTextureView,
                       to: destinationTexture)
        
    blitEncoder.endEncoding()
}

As shown in the following illustration, there are three resource objects: the original resource that contains the data, a remote view that references the data, and a resource that receives the data.

A figure showing two devices. The source buffer or texture is created by the first device. The second device has a remote view of the source texture and a destination texture.

Synchronize Access to Resources

Blit commands used in peer-to-peer transfers follow all of the usual synchronization rules on the GPU they're performed on. However, they don't automatically synchronize with any commands running on the source GPU. If you encode commands that modify the source resource, ensure that those commands are complete before executing the blit command to transfer the data to the other GPU. This is the same as what you do when transferring resources between GPUs through system memory.

To synchronize commands between different device objects, use shared events. See Synchronizing Events Across Multiple Devices and Image Filter Graph with Heaps and Events.

See Also

Resources

Setting Resource Storage Modes

Set a storage mode that defines the memory location and access permissions of a resource.

Copying Data to a Private Resource

Use a blit command encoder to copy buffer or texture data to a private resource.

Synchronizing a Managed Resource

Synchronize the contents of a managed resource for the CPU or GPU.

Reducing the Memory Footprint of Metal Apps

Learn best practices for using memory efficiently in iOS and tvOS.

protocol MTLResource

An allocation of memory that is accessible to a GPU.

protocol MTLBlitCommandEncoder

An encoder that encodes memory copying, filtering, and fill commands.

protocol MTLResourceStateCommandEncoder

An encoder that encodes commands that modify resource configurations.

Buffers

Create and manipulate unstructured GPU resources.

Textures

Create and manipulate structured GPU resources.

Indirect Command Buffers

Recoup encoding time by reusing commands, or create a GPU-driven rendering pipeline by generating commands on the GPU.

Heaps

Create a single allocation of memory from which you can suballocate resources.

Synchronization

Manage access to resources in your app to avoid data hazards.