Metal CIKernel instances with arbitrarily structured data arguments

Hello,

Can you provide the host code that declares and passes the array as an argument?

Likewise, can you provide the kernel declaration and the code that accesses the array?

If you could provide both in an Xcode project that would be ideal.

Please link the other forum posts for context.

Thanks for your reply. Strangely I didn't get an email notification of your reply.

Here are some links to other forum posts:

https://developer.apple.com/forums/thread/650405

https://developer.apple.com/forums/thread/125431

There is also a discussion here: https://stackoverflow.com/questions/57751660/metal-shading-language-for-core-image-color-kernel-how-to-pass-an-array-of-floa

On the Metal side I am using the extern "C" method.

I've tried multiple setups using both an array of structs and a simple array of floats:

struct MyStruct {
    
	float value1;
	float value2;
	float value3;
	float value4;
	float value5;
	float value6;
};

extern "C" float4 myKernel(coreimage::sample_t pixel, uint count, constant MyStruct *myArray) {

	for (uint i = 0; i < count; ++i) {

        	MyStruct myStruct = myArray[I];

		float valueToProcess = myStruct.value1;

Or

extern "C" float4 myKernel(coreimage::sample_t pixel, uint count, constant float myValues[]) {

On the Swift side the data is sent into the CIColorKernel like:

struct MyStruct {
        
	let value1: Float
	let value2: Float
	let value3: Float
 	let value4: Float
 	let value5: Float
 	let value6: Float
}

let myArray: [MyStruct] = ... 

let data = Data(bytes: myArray, count: MemoryLayout<MyStruct>.stride * myArray.count)

or

let data = myArray.withUnsafeBufferPointer { Data(buffer: $0) }

kernel.apply(
	extent: inputImage.extent,
       	roiCallback: { _, rect in rect },
        arguments: [
         	inputImage,
        	UInt(myArray.count),
        	data
         ]
)

Or

let myArray: [Float] = ... 
let data = myArray.withUnsafeBufferPointer { Data(buffer: $0) }

kernel.apply(
	extent: inputImage.extent,
       	roiCallback: { _, rect in rect },
        arguments: [
         	inputImage,
        	UInt(myArray.count),
        	data
         ]
)

I tried multiple combinations including using SIMD types but couldn’t get any to work reliably, so I’ve reverted back to a fixed set of CIVectors for now. Ideally I need an arbitrary array size that works similarly to a MTLArgumentEncoder.

As I already mentioned, I get the expected behavior up to a certain data size (1-2 structs or around 4 floats in the array), afterwards excess data is discarded. As the size grows beyond the threshold the instability randomly increases with flickering and other unexpected results.

Thanks! Michael

Thank you for the reply and citing other posts.

The notifications feature has been buggy and hopefully that's sorted out now.

Re: "Ideally I need an arbitrary array size that works similarly to a MTLArgumentEncoder."

Please send an enhancement request.

In the meantime, I'm asking framework engineers for any additional suggestions they may have and will let you know if they do.

Re: "I get the expected behavior up to a certain data size (1-2 structs or around 4 floats in the array), afterwards excess data is discarded. As the size grows beyond the threshold the instability randomly increases with flickering and other unexpected results."

This is expected as there isn't a way to specify threadgroup or gird sizes in a CIKernel. Exceeding those bounds introduces random data in kernel calculations, and possibly memory corruption on write.

According to framework engineers CoreImage doesn't support variable sized arrays as input to a CIKernel, the supported input arguments to a CIKernel are all explained in https://developer.apple.com/metal/MetalCIKLReference6.pdf

So again, please consider a feature request.

Thank you for your fast reply!

According to framework engineers CoreImage doesn't support variable sized arrays as input to a CIKernel, the supported input arguments to a CIKernel are all explained in https://developer.apple.com/metal/MetalCIKLReference6.pdf

Are you referring to arrays as input arguments or an array that is sent as data for the input argument? I might be missing something but I don’t see which input arguments are supported on the Swift side in the linked PDF.

https://developer.apple.com/documentation/coreimage/cikernel/apply(extent:roicallback:arguments:) only says that "each object in the array must be compatible with the corresponding parameter declared in the kernel routine source code” with a link to the documentation archive which indicates that the arguments must be NSNumber or CIVector.

Exceeding those bounds introduces random data in kernel calculations, and possibly memory corruption on write.

The interpretation of the feature added in iOS13 and macOS Catalina seems to be that Data type is also supported? It would be great to have some clarification on how the feature is expected to work and what its limitations are, as there seems to be some confusion surrounding this. The linked PDF is dated before the release of iOS13.