h264 Periodic Intra Refresh on iOS 8

Question

Created Jun ’15

Replies 0

Boosts 0

Views 1.1k

Participants 1

HI,

Anyone has experiance decoding a raw h264 stream with Periodic Intra Refresh using AVSampleBufferDisplayLayer ? I have an IP camera that sends h264 row stream with Periodic Intra Refresh over UDP.

I'm building a CMBlockBuffer, appending all the NALU of a frame, then create a CMSampleBuffer with DisplayImmediately attachement and enque it to the AVSampleBufferDisplayLayer as descibed in "WWDC - 2014 Direct Access to  Video Encoding and Decoding" video.

My code is:

// Called with all NALU of a frame as NSData (with start code replaced by the nalu size)
- (void)didReceiveNalu:(NSArray*)naluArray
{
        OSStatus osstatus;
        CMBlockBufferRef blockBufferRef = NULL;
        CMSampleBufferRef sampleBufferRef = NULL;


        // create a block buffer
        osstatus = CMBlockBufferCreateEmpty(CFAllocatorGetDefault(), naluArray.count, 0, &blockBufferRef);
        if (osstatus != kCMBlockBufferNoErr)
        {
            NSError* error = [NSError errorWithDomain:NSOSStatusErrorDomain code:osstatus userInfo:nil];
            NSLog(@"Error creating block buffer = %@", [error description]);
            goto exit;
        }
        for (NSData* nalu in naluArray)
        {
            osstatus = CMBlockBufferAppendMemoryBlock(blockBufferRef, (void*)nalu.bytes, nalu.length, kCFAllocatorNull,
                                                      NULL, 0, nalu.length, 0);
            if (osstatus != kCMBlockBufferNoErr)
            {
                NSError* error = [NSError errorWithDomain:NSOSStatusErrorDomain code:osstatus userInfo:nil];
                NSLog(@"Error appending block buffer = %@", [error description]);
                goto exit;
            }
        }

        // create the sample buffer
        const size_t sampleSizeArray[] = { CMBlockBufferGetDataLength(blockBufferRef) };
        osstatus = CMSampleBufferCreate(kCFAllocatorDefault, blockBufferRef, TRUE, NULL, NULL,
                                        _formatDesc, 1, 0, NULL, 0, sampleSizeArray, &sampleBufferRef);
        if (osstatus != noErr)
        {
            NSError* error = [NSError errorWithDomain:NSOSStatusErrorDomain code:osstatus userInfo:nil];
            NSLog(@"Error creating the sample buffer = %@", [error description]);
            goto exit;
        }

        // add the attachment which says that sample should be displayed immediately
        CFArrayRef attachments = CMSampleBufferGetSampleAttachmentsArray(sampleBufferRef, YES);
        CFMutableDictionaryRef dict = (CFMutableDictionaryRef)CFArrayGetValueAtIndex(attachments, 0);
        CFDictionarySetValue(dict, kCMSampleAttachmentKey_DisplayImmediately, kCFBooleanTrue);

        if ([_videoLayer status] == AVQueuedSampleBufferRenderingStatusFailed)
        {
            NSLog(@"Error _videoLayer is = %@", [_videoLayer.error description]);
            [_videoLayer flush];
        }
        else
        {
            [_videoLayer enqueueSampleBuffer:sampleBufferRef];
            // NSLog(@"done %d", [_videoLayer status]);
        }

    exit:
        if (sampleBufferRef)
        {
            CFRelease(sampleBufferRef);
        }


        if (blockBufferRef)
        {
            CFRelease(blockBufferRef);
        }
}

Everything seems to work fine until one of the Nalu (udp packet) gets drop. After that the AVSampleBufferDisplayLayer is still accepting frames (i.e status != AVQueuedSampleBufferRenderingStatusFailed) but nothing is display anymore.

Any hit?

Boost