I've been struggling with a bug with a Metal CIKernel call.
The kernel looks like this:
extern "C" float4 FeedbackIn(coreimage::sampler src, coreimage::sampler gradient, coreimage::sampler feedback, float time, float threshold, float optionValue, coreimage::destination dest )
{
float4 input = src.sample(src.coord());
return input;
}
The call from Swift looks like this:
let result = ciKernel!.apply(extent: self.inputImage!.extent,
roiCallback: { [self]
(index, rect) -> CGRect in
let roiRect = rect.insetBy(dx: -1 * range, dy: -1 * range )
return roiRect
},
arguments: [self.inputImage!, gradient!, feedback!, time, threshold, scaledOption])
// CIImage, CIImage, CIImage, Float, Float, Float
When I remove the 'feedback' from the Kernel and corresponding argument from the list in the Swift code, there is no issue. With the code as displayed, the export function will gradually slow down and memory size increase in Instruments Leaks. Instruments Leaks does not show any leaks.
I am running Xcode 13.3, macOS 12.1.1 on an iMac (Retina 5K, 27-inch, Late 2015).
Any insight on this is greatly appreciated! I don't know if passing the third CIImage is allowed or this is a bug.
Thanks to all in advance!
-
—
Boneoh
-
—
Boneoh
-
—
Boneoh
Add a CommentA little more detail about this. I am testing using a five-minute video 1920x1080.
This filter receives a CIImage that has been queued from a previous frame. So the feedback image is different for each frame. I'm guessing that somehow a reference to the feedback image is being retained but I don't know how I can prove it. The memory usage is steadily increasing over time. The memory use gradually increases to more than 6 GB.
I've save the .trace file, but unfortunately the forum won't allow me to post a link to DropBox.
I've also tried changing the call in the Swift to use weak vars and pass them. It did not help.
// test weak var feedback2 = feedback weak var gradient2 = gradient weak var image2 = self.inputImage // performance issue arises when passing in the 'feedback' image! let result = ciKernel!.apply(extent: self.inputImage!.extent, roiCallback: { [self] (index, rect) -> CGRect in let roiRect = rect.insetBy(dx: -1 * range, dy: -1 * range ) return roiRect }, arguments: [ image2!, gradient2!, feedback2!, time, threshold, scaledOption]) // CIImage, CIImage, CIImage, Float, Float, Float // FeedbackIn(coreimage::sampler src, coreimage::sampler gradient, coreimage::sampler feedback, float time, float threshold, float optionValue, coreimage::destination dest )Apologies for the ****** formatting above. Adding a comment seems limited, and I don't want to post my comments as an answer.
In addition to the memory issue, I've found a performance issue.
In my original post, I passed three CIImage instances to the CIKernel. This caused a memory issue.
I removed one of the CIImage instances for testing, and the memory issue went away.
Next I found that the export processing speed continued to degrade. Exporting the first minute measured about 20 frames per second. Exporting the first two minutes degraded to about 9 frames per second. I stopped the third attempt after around five minutes.
I did another comparison. Passing the input source image and a gradient image for a two minute test took was about 28 frames per second. But when I pass the input image and a feedback image, it degraded to about 9 frames per second for a two minute export.
The difference is that the gradient CIImage is static throughout the processing. The feedback CIImage is a queued image delayed from ten frames previously. That's the only clue I have for the performance. The performance degrades quickly over time. It starts out relatively fast and quickly degrades.
I'm open to more debugging but am at a loss of how to dig any deeper on this.