Visualization Kernel for Optical Flow

Hey there,

I am currently working on a project that can heavily benefit from optical flow. So I am really happy Apple provided an API to generate it. But before I will put further work into it I want to investigate how accurate the results are.
In there WWDC20 presentation Apple had a very cool visualization of the optical flow using a custom CIFilter. There kernel code wasn't in the presentation but they stated they will publish it in the slide attachments. Sadly, I can't happen to find these attachments. At least they are not listed in the "Resources" segment for that session.

Does anyone know where I might find the kernel code instead?

Thanks and best regards,
Max

PS:
Here is the link to the session I was talking about. The requested visualization can be found at 23 minutes.

https://developer.apple.com/videos/play/wwdc2020/10673

Replies

Did you get any further on this? I am interested in using optical flow to interpolated between to images. I go the basic to work and got a VNPixelBufferObservation back. As I understand it it is supposed to contain two float values for each pixel describing how it moves. But while I can print out the values they don't make much sense. Did you get it to work?

Sten
You can visualize some form of the result by reading the pixelBuffer from each VNPixelBufferObservation returned by the VNGenerateOpticalFlowRequest. For me, those results aren't super helpful, so I would also love to see the custom kernel that shows the magnitude and direction of the motion.

Has anyone been able to track it down?
Hey all,

You can actually find the mentioned kernel in the "Code" section for this video, but only in the Apple Developer App, not on the web.

If you navigate to this video in the Developer App, click the "Code" tab, and scroll down to the 23:05 mark, you will see the kernel!