PCI dext: scatter-gather DMA to application buffer

Hello,

I am porting a PCI driver written with IOKit to the new PCIDriverKit framework.

I am able to perform DMA with a contiguous buffer allocated inside the dext (with IOBufferMemoryDescriptor::Create).

But I would also like to perform DMA to and from a buffer allocated by an application.

What I exactly want to do is:

  • Allocate an aligned buffer in the app (with e.g. posix_memalign)
  • Send the pointer to this buffer to the dext
  • In the dext, retrieve the list of pages descriptors to be able to perform DMA without copy into (or from) this buffer.

In IOKit, we can use methods such IOMemoryDescriptor::withAddressRange and IODMACommand::gen64IOVMSegments to map and retrieve the scatter gather list but I cannot find any information on how to do this in a dext with the PCIDriverKit framework.

Can anybody help me on how to do that?

Replies

    /*
    **********************************************************************************
    **          create a memory descriptor and map its address
    **********************************************************************************
    */
    kern_return_t arcsas_userclient_create_memory_descriptor_and_map_address(const void* address, size_t length, IOMemoryDescriptor** memory_descriptor) 
    {
        IOBufferMemoryDescriptor *buffer_memory_descriptor = nullptr;
        uint64_t buffer_address;
        uint64_t len;

    #if ARCSAS_DEBUG_IO_USER_CLIENT
        arcsas_debug_print("ArcSASIOUserClient: *******************************************************\n");
        arcsas_debug_print("ArcSASIOUserClient: ** IOUserClient IOMemoryDescriptor create_with_bytes   \n");
        arcsas_debug_print("ArcSASIOUserClient: *******************************************************\n");
    #endif
        if (!address || !memory_descriptor) 
        {
            return kIOReturnBadArgument;
        }
        if (IOBufferMemoryDescriptor::Create(kIOMemoryDirectionInOut, length, 0, &buffer_memory_descriptor) != kIOReturnSuccess) 
        {
            if (buffer_memory_descriptor) 
            {
                OSSafeReleaseNULL(buffer_memory_descriptor);
            }
            return kIOReturnError;
        }
        if (buffer_memory_descriptor->Map(0, 0, 0, 0, &buffer_address, &len) != kIOReturnSuccess) 
        {
            if (buffer_memory_descriptor) 
            {
                OSSafeReleaseNULL(buffer_memory_descriptor);
            }
            return kIOReturnError;
        }
        if (length != len) 
        {
            if (buffer_memory_descriptor) 
            {
                OSSafeReleaseNULL(buffer_memory_descriptor);
            }
            return kIOReturnNoMemory;
        }
        memcpy(reinterpret_cast<void*>(buffer_address), address, length);
        *memory_descriptor = buffer_memory_descriptor;
        return kIOReturnSuccess;
    }
  IODMACommandSpecification dmaSpecification;
IOAddressSegment dma_physical_address_segment = {0};
IOAddressSegment dma_virtual_address_segment;
uint64_t dma_resource_phsical, dmaFlags = kIOMemoryDirectionInOut;
uint32_t dmaSegmentCount = 1;/* for get Physical Address, we need only one large Segment */

.....

bzero(&dmaSpecification, sizeof(dmaSpecification)); dmaSpecification.options = kIODMACommandSpecificationNoOptions; dmaSpecification.maxAddressBits = 64; if(IODMACommand::Create(ivars->pciDevice, kIODMACommandCreateNoOptions, &dmaSpecification, &ivars->dma_resource_iodmacommand) != kIOReturnSuccess) { arcsas_debug_print("ArcSASUserSpaceDriver %d: ************************************************************* \n", ivars->adapter_index); arcsas_debug_print("ArcSASUserSpaceDriver %d: ** pciDevice IODMACommand Create Failed \n", ivars->adapter_index); arcsas_debug_print("ArcSASUserSpaceDriver %d: ************************************************************* \n", ivars->adapter_index); return false; } if(ivars->dma_resource_iodmacommand->PrepareForDMA(kIODMACommandPrepareForDMANoOptions, ivars->dma_resource_descriptor, 0/*offset /, dma_resource_size/length/, &dmaFlags, &dmaSegmentCount, &dma_physical_address_segment) != kIOReturnSuccess) { arcsas_debug_print("ArcSASUserSpaceDriver %d: ************************************************************************** \n", ivars->adapter_index); arcsas_debug_print("ArcSASUserSpaceDriver %d: ** IODMACommand PrepareForDMA try to get Memory Physical Address Failed \n", ivars->adapter_index); arcsas_debug_print("ArcSASUserSpaceDriver %d: ************************************************************************** \n", ivars->adapter_index); return false; } / physical / dma_resource_phsical = (uint64_t)dma_physical_address_segment.address; dma_resource->Physical_Address.parts.low = (uint32_t)(dma_resource_phsical & 0xFFFFFFFF); dma_resource->Physical_Address.parts.high = (uint32_t)(dma_resource_phsical>> 32); / virtual */ dma_resource->Virtual_Address = reinterpret_cast < uint8_t *> (dma_virtual_address_segment.address); memset(dma_resource->Virtual_Address, 0, dma_resource_size);

Hello, I also have the same question as you, asked if it has been solved, can you share the solution?

Hi,

I had similar struggles when I was porting my KEXT PCIE driver to DEXT driver. I especially had a hard time porting DMA functionality. The problem was DriverKit has some restrictions about memory management API. And I also couldn't useIOMemoryDescriptor::withAddressRange like @Benoit55 said and I didn't want to copy the user space buffer like @erich-areca so I came up with two solutions.

  • You can choose to allocate a buffer in DEXT address space using IOBufferMemoryDescriptor::Create and map it to the user space using functions IOConnectMapMemory64and CopyClientMemoryForTypefunctions
  • You can use the feature of IOConnectCallStructMethod that creates IOMemoryDescriptor when the structure size (or buffer size ) exceeds the size of a page ( 4096 byte ) and you may need to map that descriptor to DEXT space using CreateMapping instance method.
  • And rest is same as usual ( usingIODMACommand to prepare and create bus address segments )