Decompressing Non-IDR NALs

Question

Created Aug ’15

Replies 1

Boosts 0

Views 1.4k

Participants 2

I am implementing an app that receives an h264 video stream in not-NAL-packaged chunks from a hardware accessory (an abritrarily divided stream of bytes), but it's not working for me. The stream does not contain IDRs, though one is provided by the accessory manufacturer to be loaded at runtime. I have successfully decompressed the IDR by concatenating all the 0x05 (IDR image slice) NALs together and supplying an appropriate CMVideoFormatDescriptionRef generated from the SPS and PPS in the reference IDR.

I can extract the NALs received from the byte stream by hunting for 0x00000001 start codes. However, when I try to concatenate all of the following 0x01 (non-IDR video slice) NALs into a CMBlockBuffer into a CMSampleBuffer and decompress that, the decompression nearly always fails (the specific error varies, I recall most of the time with

-12911

). I am applying the same transformation to the 0x01 NALs as the 0x05 NALs to generate AVCC NALs (strip off the header, package the size as a 4-byte big endian header preceeding the payload). Is there something I'm missing? There is a 0x00 NAL every 9th NAL, which I've been using as the delimeter between CMSampleBuffer decompress attempts, though perhaps that's not the appropriate boundary to be using.

This video stream is decompressed successfully (slowly) by libav, with FLAG2_SHOW_ALL. I don't have information about the configuration used to generate the h264 stream or the specific libav compilation configuration, though it's clear there's no hardware codecs included.

Any advice would be appreciated. I've read through the WWDC 513 talk and http://stackoverflow.com/questions/29525000/how-to-use-************-to-decompress-h-264-video-stream/29525001#29525001, which got me this far, but I feel like there's something I'm missing. A reference implementation of an app that displays an h264 stream received as bytes would be invaluable, for instance.

Boost

Answer 1

roxlu OP

Sep ’15

Hey pguimby,

Seems we're working on a similar thing. I've posted something earlier today https://forums.developer.apple.com/message/66542#66542. In my case decoding works fine as longs as the encoder produces one slice. In my approach I'm parsing the h264 stream and detecting access units. For each access unit I collect the slice data which I use to create a CMBlockBufferRef. When I have the CMBlockBufferRef, I fill it with the slice data and call CMSampleBufferCreate. I pass an array with sizes into the CMSampleBufferCreate + a timestamp array. Make sure that you're using arrays of the same type here (e.g. size_t for the size array).

I've got an example application which decodes streams with one slice per frame fine. When I receive multiple slices only the top part is decoded, though I'm not receiving any errors from ************.

If you want to have a look at this code I can provide it. You can find my email through https://twitter.com/roxlu

Cheers

0