Hello everyone,
I am migrating a KEXT for a SCSI PCI RAID controller (LSI 3108 RoC) to DriverKit (DEXT). While the DEXT loads successfully, I'm facing a DMA issue: an INQUIRY command results in a 0-byte disk because the data buffer received by the DEXT is all zeros, despite our firmware logs confirming that the correct data was prepared and sent.
We have gathered detailed forensic evidence and would appreciate any insights from the community.
Detailed Trace of a Failing INQUIRY Command:
1, DEXT Dispatches the Command:
Our UserProcessParallelTask implementation correctly receives the INQUIRY task. Logs show the requested transfer size is 6 bytes, and the DEXT obtains the IOVA (0x801c0000) to pass to the hardware.
DEXT Log:
[UserProcessParallelTask_Impl] --- FORENSIC ANALYSIS ---
[UserProcessParallelTask_Impl] fBufferIOVMAddr = 0x801c0000
[UserProcessParallelTask_Impl] fRequestedTransferCount = 6
2, Firmware Receives IOVA and Prepares Correct Data:
A probe in our firmware confirms that the hardware successfully received the correct IOVA and the 6-byte length requirement. The firmware then prepares the correct 6-byte INQUIRY response in its internal staging buffer.
Firmware Logs:
-- [FIRMWARE PROBE: INCOMING DMA DUMP] --
Host IOVA (High:Low) = 0x00000000801c0000
DataLength in Header = 6 (0x6)
--- [Firmware Outgoing Data Dump from go_inquiry] ---
Source Address: 0x228BB800, Length: 6 bytes
0x0000: 00 00 05 12 1F 00
3, Hardware Reports a Successful Transfer, but Data is Lost:
After the firmware initiates the DMA write to the Host IOVA, the hardware reports a successful transfer of 6 bytes back to our DEXT.
DEXT Completion Log:
[AME_Host_Normal_Handler_SCSI_Request] [TaskID: 200] COMPLETING...
[AME_Host_Normal_Handler_SCSI_Request] Hardware Transferred = 6 bytes
[AME_Host_Normal_Handler_SCSI_Request] - ReplyStatus = SUCCESS (0x0)
[AME_Host_Normal_Handler_SCSI_Request] - SCSIStatus = SUCCESS (0x0)
The Core Contradiction:
Despite the firmware preparing the correct data and the hardware reporting a successful DMA transfer, the fDataBuffer in our DEXT remains filled with zeros. The 6 bytes of data are lost somewhere between the PCIe bus and host memory.
This "data-in-firmware, zeros-in-DEXT" phenomenon leads us to believe the issue lies in memory address translation or a system security policy, as our legacy KEXT works perfectly on the same hardware.
Compared to a KEXT, are there any known, stricter IOMMU/security policies for a DEXT that could cause this kind of "silent write failure" (even with a correct IOVA)?
Alternatively, what is the correct and complete expected workflow in DriverKit for preparing an IOMemoryDescriptor* fDataBuffer (received in UserProcessParallelTask) for a PCI hardware device to use as a DMA write target?
Any official documentation, examples, or advice on the IOMemoryDescriptor to PCI Bus Address workflow would be immensely helpful.
Thank you.
Charles
While the DEXT loads successfully, I'm facing a DMA issue: an INQUIRY command results in a 0-byte disk because the data buffer received by the DEXT is all zeros, despite our firmware logs confirming that the correct data was prepared and sent.
How is your DEXT getting access to the transfer data? SCSIControllerDriverKit was intentionally architected to make that very difficult.
Compared to a KEXT, are there any known, stricter IOMMU/security policies for a DEXT that could cause this kind of "silent write failure" (even with a correct IOVA)?
No, not really.
However, I think you've confused things a bit here:
Alternatively, what is the correct and complete expected workflow in DriverKit for preparing an IOMemoryDescriptor* fDataBuffer (received in UserProcessParallelTask) for a PCI hardware device to use as a DMA write target?
My concern here is where you say:
...for preparing an IOMemoryDescriptor* fDataBuffer
The problem is that the whole "goal" of UserProcessParallelTask is to AVOID giving you an IOMemoryDescriptor. What you're actually given is "fBufferIOVMAddr", which is a "raw" physical address ready to be passed over to the PCI bus. The way this is supposed to work is that you pass that address to your card (doing whatever is necessary to break up the transfer), it does the DMA, and you then tell the kernel "done now". That completes the entire transfer without your DEXT ever having access to the actual data.
NOW, you do have UserGetDataBuffer, but we don't really want you to use it. Quoting the header documentation:
"The dext class can call this method inside UserProcessParallelTask to get the data buffer associated with that IO request. Only call this method if access to the data buffer is required. Calling this method can have a significant impact on performance. The caller will have to prepare new DMA mappings for this buffer and can no longer use the mappings in fBufferIOVMAddr."
If you call UserGetDataBuffer, then you're taking over the entire DMA process and it's now your responsibility to "get" the data into the memory descriptor you're given. That also means that "fBufferIOVMAddr" is no longer useful.
That leads to here:
Despite the firmware preparing the correct data and the hardware reporting a successful DMA transfer, the fDataBuffer in our DEXT remains filled with zeros.
This is just a guess, but I suspect what you actually did in UserProcessParallelTask was:
-
Call UserGetDataBuffer, thinking that would let you "see" the data.
-
Did your DMA transfer using fBufferIOVMAddr.
However, that meant what actually happened was:
-
The DMA to fBufferIOVMAddr completed as expected.
-
You called UserCompleteParallelTask.
-
The kernel copied the contents of the IOMD returned by UserGetDataBuffer into the fBufferIOVMAddr.
...zeroing out the data that was just transferred.
__
Kevin Elliott
DTS Engineer, CoreOS/Hardware
