ProRes encoding on M1 Max fails for high bit depth buffers

I have code that has worked for many years for writing ProRes files, and it is now failing on the new M1 Max MacBook. Specifically, if I construct buffers with the pixel type "kCVPixelFormatType_64ARGB", after a few frames of writing, the pixel buffer pool becomes nil. This code works just fine on non Max processors (Intel and base M1 natively).

Here's a sample main that demonstrates the problem. Am I doing something wrong here?

//  main.m
//  TestProresWriting
//

#import <Foundation/Foundation.h>
#import <AVFoundation/AVFoundation.h>

int main(int argc, const char * argv[]) {
    @autoreleasepool {
        int timescale = 24;
        int width = 1920;
        int height = 1080;

        NSURL *url = [NSURL URLWithString:@"file:///Users/diftil/TempData/testfile.mov"];

        NSLog(@"Output file = %@", [url absoluteURL]);
        NSFileManager *fileManager = [NSFileManager defaultManager];
        NSError *error = nil;
        [fileManager removeItemAtURL:url error:&error];

        // Set up the writer
        AVAssetWriter *trackWriter = [[AVAssetWriter alloc] initWithURL:url
                                                    fileType:AVFileTypeQuickTimeMovie
                                                        error:&error];

        // Set up the track
        NSDictionary *videoSettings = [NSDictionary dictionaryWithObjectsAndKeys:
                                       AVVideoCodecTypeAppleProRes4444, AVVideoCodecKey,
                                       [NSNumber numberWithInt:width], AVVideoWidthKey,
                                       [NSNumber numberWithInt:height], AVVideoHeightKey,
                                       nil];

        

        AVAssetWriterInput *track = [AVAssetWriterInput assetWriterInputWithMediaType:AVMediaTypeVideo
                                                        outputSettings:videoSettings];

        // Set up the adapter

        NSDictionary *attributes = [NSDictionary
                                    dictionaryWithObjects:
                                    [NSArray arrayWithObjects:[NSNumber numberWithUnsignedInt:kCVPixelFormatType_64ARGB], // This pixel type causes problems on M1 Max, but works on everything else
                                     [NSNumber numberWithUnsignedInt:width],[NSNumber numberWithUnsignedInt:height],
                                     nil]
                                    forKeys:
                                    [NSArray arrayWithObjects:(NSString *)kCVPixelBufferPixelFormatTypeKey,
                                     (NSString*)kCVPixelBufferWidthKey, (NSString*)kCVPixelBufferHeightKey,
                                     nil]];

        /*
        NSDictionary *attributes = [NSDictionary
                                    dictionaryWithObjects:
                                    [NSArray arrayWithObjects:[NSNumber numberWithUnsignedInt:kCVPixelFormatType_32ARGB], // This pixel type works on M1 Max
                                     [NSNumber numberWithUnsignedInt:width],[NSNumber numberWithUnsignedInt:height],
                                     nil]
                                    forKeys:
                                    [NSArray arrayWithObjects:(NSString *)kCVPixelBufferPixelFormatTypeKey,
                                     (NSString*)kCVPixelBufferWidthKey, (NSString*)kCVPixelBufferHeightKey,
                                     nil]];
        */

        AVAssetWriterInputPixelBufferAdaptor *pixelBufferAdaptor = [AVAssetWriterInputPixelBufferAdaptor
                            assetWriterInputPixelBufferAdaptorWithAssetWriterInput:track
                            sourcePixelBufferAttributes:attributes];

        // Add the track and start writing

        [trackWriter addInput:track];
        [trackWriter startWriting];

        CMTime startTime = CMTimeMake(0, timescale);
        [trackWriter startSessionAtSourceTime:startTime];

        while(!track.readyForMoreMediaData);

        int frameTime = 0;

        CVPixelBufferRef frameBuffer = NULL;

        for (int i = 0; i < 100; i++)
        {
            NSLog(@"Frame %@", [NSString stringWithFormat:@"%d", i]);
            CVPixelBufferPoolRef PixelBufferPool = pixelBufferAdaptor.pixelBufferPool;
            if (PixelBufferPool == nil)
            {
                NSLog(@"PixelBufferPool is invalid.");
                exit(1);
            }

            CVReturn ret = CVPixelBufferPoolCreatePixelBuffer(nil, PixelBufferPool, &frameBuffer);

            if (ret != kCVReturnSuccess)
            {
                NSLog(@"Error creating framebuffer from pool");
                exit(1);
            }

            CVPixelBufferLockBaseAddress(frameBuffer, 0);
            // This is where we would put image data into the buffer.  Nothing right now.
            CVPixelBufferUnlockBaseAddress(frameBuffer, 0);

            while(!track.readyForMoreMediaData);

            CMTime presentationTime = CMTimeMake(frameTime+(i*timescale), timescale);
            BOOL result = [pixelBufferAdaptor appendPixelBuffer:frameBuffer
                                           withPresentationTime:presentationTime];
            if (result == NO)
            {
                NSLog(@"Error appending to track.");
                exit(1);
            }

            CVPixelBufferRelease(frameBuffer);
        }

        // Close everything
        if ( trackWriter.status == AVAssetWriterStatusWriting)
            [track markAsFinished];

        NSLog(@"Completed.");
    }
    return 0;
}
  • It's been a couple of months since I posted this, and I wanted to try to summarize where we are with this issue. First, I think it's established that this is a bug (my report is FB9757381), and it's related to M1 devices with ProRes hardware acceleration (the Pro and Max variants). The specific bug is that the M1 Pro and Max implementations of the AVFoundation architecture will crash unexpectedly (and without prior warning) when buffers with pixel types kCVPixelFormatType_64ARGB are written. Other pixel types may also fail or behave unexpectedly (there are reports of alpha channel problems), but I haven't exhaustively tested them.

    A workaround proposed by the helpful @MirrorMan indicates that on all M1 chips (including Max and Pro), the pixel type kCVPixelFormatType_64RGBALE works without error, but it fails on Intel architectures. As a workaround, it's suboptimal because it requires querying the underlying silicon type and choosing a different pixel type, which is contrary to how we ought to write code on this platform. Also, it's not obvious that if we switched to the workaround pixel format that it would keep working in a future macOS update. In the absence of guidance from Apple, this workaround isn't viable.

    This is complicated by the API itself. In the documentation for the CoreVideo type constants, it is indicated that not all types are supported, but there's no way to know which ones are or aren't. 

    As a developer, I'm stuck. I have an application I support that runs everywhere except on the new MacBook Pro computers, where it crashes. I'm not able to give my clients an estimate on when the program will work. It would be very helpful to me if I could get an idea of Apple's thinking about this issue. For instance, does Apple agree this is a bug, and if so, is there a timeline for a fix? If it's not a bug and it's something I'm doing wrong, I'd like to know that too. If we are to use a workaround, that would be good to know as well. 

    I know it's normal policy that bugs are not acknowledged or commented upon, but I am asking for some help here. This is a case of new hardware and a somewhat vague API creating a stickier than normal situation, and I'd be very grateful for some feedback.

  • Hi, at the moment, your feedback item is classified as a bug. However, the team still needs to investigate the issue to confirm this. Unfortunately, we don't give ETAs/timelines for bug fixes. However, your bug hasn't been forgotten about and it's been assigned to an engineer to investigate.

  • I really appreciate the response. I'll keep watching the releases and keep my fingers crossed. If you get any more info that you're able to share, that'd be great too. Thanks!

Replies

Hi, this sounds like a bug, especially if you aren’t using any deprecated APIs. Please report this on Feedback Assistant, and attach this code sample and a sample video file if possible. Also, post the feedback number here for reference.

  • Thanks! I have done so. The feedback number is FB9757381. Thanks very much again!

  • Perfect! Looks like the bug report has been sent to the AVFoundation team already.

  • I tested this on macOS 12.1 Beta 3, and it's still broken. Wondering if there's been any update.

I may be running into a very similar issue on the M1 Max, where our transparent ProRes 4444 videos are fine on all other devices but with the M1 Max they appear black.

  • @kirton same as above, file a bug on feedback assistant and put the feedback number here.

Add a Comment

Is there some public bug that's already been logged for this ? I think I might be hitting a similar issue with my own ProRes integration.

  • @Clusty yes, the OP filed FB9757381. However, I would encourage you to file another one yourself. More bug reports can sometimes help with prioritizing the order in which bugs are fixed. If you do file one, post the feedback id here to make the bug easier to find for other apple engineers who see this post in the future.

  • Did my bug report also: FB9840778 Would love to know if this is "expected behaviour" or bug that will be fixed.

  • Thank you! Since OP says this is only an issue on M1 Max and not M1/Intel, I would assume this is a bug. But the team will have to investigate to know for sure.

Add a Comment

I've just run through some analysis on this. It's all about pixel formats --- which are all over the place for ProRes and M1's. I encode from RGB, so here are the kCVPixelFormatTypes that I have working on different machines:

MACHINE / 64ARGB / 64RGBALE Intel / Works / fails M1 Mac Mini (2020) / Works! / Works MacBook M1 Pro / fails! / Works

From this I conclude to use 64ARGB on Intel, 64RGBALE on all Apple Silicon. It would be nice to official sign-off on that. And it would be nice to have little-endian encoding and decoding "everywhere".

  • Thanks! That's interesting. Does anyone know a way to query the supported kCVPixelFormats at runtime? I thought I ran across that in the docs once, but it's lost to my searches now. Probably just dreamed it.

Add a Comment

Hello, I am not really a developer, but I seem to be running into a problem similar to this. I've been exporting Apple ProRes 4444 files + alpha perfectly for a long time, but on the Silicon M1 Max Macbooks it doesn't seem to work anymore. Mov-files exported with this codec appear black instead of transparant. How can I raise this issue/bug with Apple? It's a pretty important thing for editors / motion designers who are in need of exports + an alpha layer for their clients. Thanks!

  • @fefferlef you can report this bug on Feedback Assistant or on apple.com/feedback . If you use Feedback assistant, please post the feedback ID here.

Add a Comment

I am not a dev, but a video Postproduction operator. I use prores4444 with alpha all the time. Works all on my intel iMac. But on my new M1 max MacBook Pro some files are shown black with glitches, not correct with alpha channel. I posted a bug report. FB9909638

  • @ChristianWittmer thank you

  • Do you have any ETA or timeline when the bug will be fixed?

  • We don’t provide ETAs unfortunately.

Add a Comment

Hello, I am not really a developer too, but I don't remember how many times I reported this issue to the apple support line in 2 months. I was hoping that it was fixed with the new MacOs update, but it seems that there is no solution for it again. And again, I'm afraid Apple will never be able to do this job!

Damn. Right now, this machine is totally useless for me - I can't edit my current project on this machine.

Come on Apple, this is a totally serious issue - fix it now!

I will return the MacBook to Apple until the bug is solved.

Good luck everyone!

A small finding: The issue seems to appear only with kCVPixelFormatType_64ARGB pixels and switching to kCVPixelFormatType_4444AYpCbCr16 pixels seems to fix the issue (and I think it is more performant )

  • That does work, but it's an annoying workaround. It involves the application doing the color space conversion from RGB to YCbCr, which eliminates any performance gains (it is better to have the API do this symmetrically on read and write than to implement it in the application). And if the ProRes hardware encoding of the M1 Pro/Max/Ultra can do the color space conversion, it's very unfortunate to have to offload this to the CPU.

  • That's not a great workaround. Changing to YCbCr as input requires conversion of the colorspace from RGB, which the application would have to do. So it may be faster to write, but the conversion would add overhead. Presumably it would be better to let the API do the colorspace in and out of YCbCr rather than implementing it manually, and if the coprocessor of the Max/Pro/Ultra chips can do this, there's no way I can do it faster on the CPU.

Add a Comment

Looks like MacOS Montery 12.3.1 updated the video codecs. Perhaps the problem is fixed.

  • I'm not a developer but I work in video marketing and I've encountered something similar to what's being conveyed here. However, I only noticed this issue once I updated to macOS Monterey 12.3.1 (on a 14" MBP with an M1 Max chip).

    My situation is that I'm seeing a vertical black bar on a graphic (exported as ProRes with an alpha channel), localized only on the right side of the frame.

    I see it when I'm running Quicktime, when played back in FCP X, and in Premiere Pro. But strangely, I'm ONLY seeing it on exports that are in 1x1, 9x16, or 4x5 aspect ratios. For 16x9 aspect ratios, everything is normal.

    Last month, prior to updated to 12.3.1 and using these same alpha channel graphics, I did not have any issues on my M1 MBP. I was also able to test these files with a friend's machine -- who also has a 14" M1 MacBookPro, and he got the same issue. In contrast, I have an older laptop (MPB 2019, running an Intel chip and on Big Sur 11.6.5 and I have no issues with the same graphics files.

    Very strange.

  • Sadly, it isn't fixed.

  • I’m seeing exactly the same problem on m1 ultra Mac studio with Monterey 12.4 black line on the right side of frame on prores4444 files with alpha channel. It’s in the alpha layer as it goes away if the alpha is turned off. Only on image ratios other than 16:9 hopefully they’ll fix this soon…

Add a Comment

It isn't fixed.

I have a Mac Studio with M1 Ultra that has been behaving strangely with 4444, specifically related to alpha channel transparency. When I export a project as 4444 from Final Cut Pro, it has transparency, but when I encode video to 4444 using Syphon (crucial to my workflow) the alpha channel is not transparent, but black. On my previous intel machine, this was not an issue. I've tried exporting in Syphon as HAP Q, then converting to 4444 using an application called AVF Batch Converter, but this does not work on my M1 ultra. Specifically, the background remains black, not transparent.

However, I can send files to my video editor as HAP Q, and he can use the exact same AVF Batch Converter settings to get 4444 with alpha channel on his Intel i9 iMac. I don't understand why the M1 ultra is having this issue. If I could export in 4444 with Syphon like I could on my previous intel computer, that would be amazing.

  • UPDATE: 4444 is now working as expected. All I did was update to OS 12.6 and now my specific software Syphon exports alpha channel on 4444 Pro Res

Add a Comment

I have received a notification from the Apple byg system that it is fixed in the latest beta version of osx ( I have bot verified it yet ).

I am a full time cinematographer and video editor and have run into this issue after switching from an intel-based iMac Pro to the M2 Ultra Mac Studio earlier this year and am having a hard time finding documentation of this issue and this is the closest I have come, even if it was a year ago, it is still a problem.

Apple ProRes 4444 XQ render files exported from a render artist using Houdini worked fined on my Intel Mac but exhibit a broken alpha channel on my new Mac Studio that can only be remedied by taking the files and re-exporting from an Intel based Mac within Adobe Media Encoder to MXF format. Reexporting from the Mac Studio does not remedy the issue.

The files exhibit the erroneous characteristic in Adobe programs and QuickTime Player, but not in VLC. I do not possess the wherewithal to determine the root cause of the issue, only to provide the one workaround I discovered. I tried several different settings within Premier to no avail to try to get the footage to playback correctly. Essentially it comes out blocky frame by frame, as if the alpha channel is inverting itself incorrectly in segments.

videosettings and attributes are not identical dictionaries. I solved a similar problem by using the same dictionary for both variables.