DEXT receives zero-filled buffer from DMA, despite firmware confirming data write

While the DEXT loads successfully, I'm facing a DMA issue: an INQUIRY command results in a 0-byte disk because the data buffer received by the DEXT is all zeros, despite our firmware logs confirming that the correct data was prepared and sent.

How is your DEXT getting access to the transfer data? SCSIControllerDriverKit was intentionally architected to make that very difficult.

Compared to a KEXT, are there any known, stricter IOMMU/security policies for a DEXT that could cause this kind of "silent write failure" (even with a correct IOVA)?

No, not really.

However, I think you've confused things a bit here:

Alternatively, what is the correct and complete expected workflow in DriverKit for preparing an IOMemoryDescriptor* fDataBuffer (received in UserProcessParallelTask) for a PCI hardware device to use as a DMA write target?

My concern here is where you say:

...for preparing an IOMemoryDescriptor* fDataBuffer

The problem is that the whole "goal" of UserProcessParallelTask is to AVOID giving you an IOMemoryDescriptor. What you're actually given is "fBufferIOVMAddr", which is a "raw" physical address ready to be passed over to the PCI bus. The way this is supposed to work is that you pass that address to your card (doing whatever is necessary to break up the transfer), it does the DMA, and you then tell the kernel "done now". That completes the entire transfer without your DEXT ever having access to the actual data.

NOW, you do have UserGetDataBuffer, but we don't really want you to use it. Quoting the header documentation:

"The dext class can call this method inside UserProcessParallelTask to get the data buffer associated with that IO request. Only call this method if access to the data buffer is required. Calling this method can have a significant impact on performance. The caller will have to prepare new DMA mappings for this buffer and can no longer use the mappings in fBufferIOVMAddr."

If you call UserGetDataBuffer, then you're taking over the entire DMA process and it's now your responsibility to "get" the data into the memory descriptor you're given. That also means that "fBufferIOVMAddr" is no longer useful.

That leads to here:

Despite the firmware preparing the correct data and the hardware reporting a successful DMA transfer, the fDataBuffer in our DEXT remains filled with zeros.

This is just a guess, but I suspect what you actually did in UserProcessParallelTask was:

• Call UserGetDataBuffer, thinking that would let you "see" the data.

• Did your DMA transfer using fBufferIOVMAddr.

However, that meant what actually happened was:

• The DMA to fBufferIOVMAddr completed as expected.

• You called UserCompleteParallelTask.

• The kernel copied the contents of the IOMD returned by UserGetDataBuffer into the fBufferIOVMAddr.

...zeroing out the data that was just transferred. __
Kevin Elliott
DTS Engineer, CoreOS/Hardware

Hello Kevin,

Thank you for your analysis. We have reviewed our implementation and can confirm that we are not calling UserGetDataBuffer in our UserProcessParallelTask_Impl. Our DEXT fully adheres to the official recommendation you described, using the framework-provided fBufferIOVMAddr for the DMA operation.

Since that was not the issue, we performed a deeper comparison between our working KEXT and the DEXT. We discovered that in its InitializeController() method, the KEXT explicitly declares its DMA capabilities to the I/O Kit storage stack by setting several properties. The code is as follows:

// In KEXT's InitializeController()
setProperty(kIOMaximumSegmentCountReadKey,  (UInt64)MAX_IO_SG, 64);
setProperty(kIOMaximumSegmentCountWriteKey, (UInt64)MAX_IO_SG, 64);
setProperty(kIOMaximumByteCountReadKey,     (UInt64)MAX_IO_LENGTH, 64);
setProperty(kIOMaximumByteCountWriteKey,    (UInt64)MAX_IO_LENGTH, 64);
setProperty(kIOPropertyPhysicalInterconnectLocationKey, kIOPropertyExternalKey);

It's possible that the absence of these property settings in our DEXT is what causes the SCSI framework to fail when preparing the IOMMU mapping for the INQUIRY command's buffer, leading to the silent DMA failure.

Our question now is what the correct method is for implementing this in DriverKit. We found the IOPCIDevice::SetProperties(OSDictionary *) method, and we have also seen these property keys mentioned in DriverKit-related documentation. However, the key constants themselves (e.g., kIOMaximumSegmentCountReadKey) do not appear to be defined in any available DriverKit SDK header, which prevents our code from compiling.

Could you please advise us on the official, recommended way for a DEXT to set these storage-specific properties for the framework? Are we missing a specific header file, or is the expected approach to manually define these key strings in our source code?

Thank you again for your valuable assistance.

Charles

The correct approach for setting these properties is documented in the SCSIControllerDriverKit.iig file.

Instead of calling IOPCIDevice::SetProperties, the intended method is to override the UserReportHBAConstraints virtual function in our DEXT's main class.

This also answers the second part of my question: we found that all the necessary property keys (e.g., kIOMaximumSegmentCountReadKey) are officially defined in the <DriverKit/IOKitKeys.h> header, which we were able to include directly.

The solution is to:

1 - Declare the override for UserReportHBAConstraints in the .iig file.

2 - #include <DriverKit/IOKitKeys.h> in the .cpp file.

3 - Implement the function to populate the dictionary with the hardware's DMA constraints using the official keys.

Best regards, Charles

We've implemented the override for UserReportHBAConstraints. However, our logs confirm it is never called by the framework.

We are confused by the official documentation for UserReportHBAConstraints

which states:

Subclasses must call this method from the dext before UserInitializeController returns.

What does this mean in practice? Are we expected to call this method manually?

We've implemented the override for UserReportHBAConstraints. However, our logs confirm it is never called by the framework.

Correct. Methods in DriverKit fall into one of two categories:

• Methods you're responsible for implementing, which are typically marked as pure virtual:

 virtual kern_return_t
    UserInitializeController () = 0;

• Methods you'll be calling, which have concrete implementations like:

virtual kern_return_t
UserReportHBAConstraints ( OSDictionary * constraints );

We are confused by the official documentation for UserReportHBAConstraints.

Yes, the header doc for that is a bit confused about its audience and purpose. The comments say:

Subclasses must set the required keys if they override this method. If a subclass does not provide the required keys, the system will panic.

...but that's basically nonsense as far as DEXT is concerned. The only way you have to set those dictionary keys is to call "UserReportHBAConstraints", so if you were going to override it, you'd need to call super::UserReportHBAConstraints. More to the point, since “you“ are responsible for calling it there's no reason you'd ever override it. Indeed, our own controller DEXT defines its own "Populate" method, which configures all the keys, then calls "UserReportHBAConstraints". I'd do exactly the same thing.

In a similar vein, our DEXT meets this requirement:

which states: Subclasses must call this method from the dext before UserInitializeController returns.

...by calling its "Populate" method inside UserInitializeController.

What does this mean in practice? Are we expected to call this method manually?

Yes, that's exactly what it means.

__
Kevin Elliott
DTS Engineer, CoreOS/Hardware