Hello,
This week I've had a miserable time trying to get video frames from a Quicktime movie and display them in a Metal texture. It has been far harder than I anticipated. I just want to show frame 0, and then step back and forth through the video frames with the arrow keys. After failing with the AVPlayererItemVideoOutput.hasNewPixelBuffer(forItemTime: time) function, I'm now trying to use the seek() function. This almost works. With the video paused I call the seek function for the time I want, and it says it succeeds.
However, every few frames, the hasNewPixelBuffer() function returns 'false' and the copyPixelBuffer() function just returns the same data as the currently displayed frame. At first, thought it might have something to do with the fact that the video (h264) has frame-reordering. But I get the same results when I use a video with all its frames in the correct order. I've tried the seek tolerance set to 0, and then also various values like half a frame or less, in case there was some floating point roundoff issue with the times I was calculating. Nothing seems to work.
Anyone know how to do this? It ought to be so simple and yet I've tried everything I can think of. I've also tried AVAssetImageGenerator and it did the same thing. AVAssetReader will report all the frames sequentially, but that doesn't really help if you just want random access to the frames.
I hate AVFoundation after this week -- why does it have to be so difficult to do this simplest of video tasks...
Any suggestions appreciated.