We've successfully implemented an AVAssetWriter to produce HLS streams (all code is Objective-C++ for interop with existing codebase) but are struggling to extend the operations to use tagged buffers.
We're starting to wonder if the tagged buffers required for an MV-HEVC signal are fully supported when producing HLS segments in a live-stream setting.
We generate a live stream of data using something like:
UTType *t = [UTType typeWithIdentifier:AVFileTypeMPEG4];
m_writter = [[AVAssetWriter alloc] initWithContentType:t];
// - videoHint describes HEVC and width/height
// - m_videoConfig includes compression settings and, when using MV-HEVC,
// the correct keys are added (i.e. kVTCompressionPropertyKey_MVHEVCVideoLayerIDs)
// The app was throwing an exception without these which was
// useful to know when we got the configuration right.
m_video = [[AVAssetWriterInput alloc] initWithMediaType:AVMediaTypeVideo outputSettings:m_videoConfig sourceFormatHint:videoHint];
For either path we're producing CVPixelBufferRefs that contain the raw pixel information (i.e. 32BGRA) so we use an adapter to make that as simple as possible.
If we use a single view and a AVAssetWriterInputPixelBufferAdaptor things work out very well. We produce segments and the delegate is called.
However, if we use the AVAssetWriterInputTaggedPixelBufferGroupAdaptor as exampled in the SideBySideToMVHEVC demo project, things go poorly.
We create the tagged buffers with something like:
CMTagCollectionRef collections[2];
CMTag leftTags[] = {
CMTagMakeWithSInt64Value(
kCMTagCategory_VideoLayerID, (int64_t)0),
CMTagMakeWithSInt64Value(
kCMTagCategory_StereoView, kCMStereoView_LeftEye)
};
CMTagCollectionCreate(
kCFAllocatorDefault, leftTags, 2, &(collections[0])
);
CMTag rightTags[] = {
CMTagMakeWithSInt64Value(
kCMTagCategory_VideoLayerID, (int64_t)1),
CMTagMakeWithSInt64Value(
kCMTagCategory_StereoView, kCMStereoView_RightEye)
};
CMTagCollectionCreate(
kCFAllocatorDefault, rightTags, 2, &(collections[1])
);
CFArrayRef tagCollections = CFArrayCreate(
kCFAllocatorDefault, (const void **)collections, 2, &kCFTypeArrayCallBacks
);
CVPixelBufferRef buffers[] = {*b, *alt};
CFArrayRef b = CFArrayCreate(
kCFAllocatorDefault, (const void **)buffers, 2, &kCFTypeArrayCallBacks
);
CMTaggedBufferGroupRef bufferGroup;
OSStatus res = CMTaggedBufferGroupCreate(
kCFAllocatorDefault, tagCollections, b, &bufferGroup
);
Perhaps there's something about this OBJC code that I've buggered up? Hopefully!
Anyways, when I submit this tagged bugger group to the adaptor:
if (![mvVideoAdapter appendTaggedPixelBufferGroup:bufferGroup withPresentationTime:pts]) {
// report error...
}
Appending does not raise any errors - eventually it just hangs on us and we never return from it...
Real Issue:
So either:
The delegate assigned to the AVAssetWriter doesn't fire its assetWriter callback which should produce the segments
The adapter hangs on the appendTaggedPixelBufferGroup before a segment is ready to be completed (but succeeds for a number of buffer groups before this happens).
This is the same delegate class that's assigned to the non multi view code path if MV-HEVC is turned off which works perfectly.