Technical Note TN2447

Debugging AVFoundation Compositions, Video Compositions, and Audio Mixes

This document discusses some common problems you may encounter when creating complex AVFoundation compositions, video compositions and audio mixes, and how to debug them.

Common pitfalls
Debugging Compositions, Video Compositions and Audio Mixes
Debugging Video Compositions
Additional Information
Document Revision History


The AVFoundation framework provides a feature-rich set of classes to facilitate the editing of audiovisual assets. At the heart of AVFoundation’s editing API are compositions, defined by the AVComposition object. A composition is simply a collection of tracks from one or more different media assets. The AVMutableComposition class provides an interface for inserting and removing tracks, as well as managing their temporal orderings. The tracks in an AVComposition object are fixed.

If you want to perform any custom audio or video processing on the tracks in your composition, you need to incorporate an audio mix or a video composition, respectively. An AVAudioMix object manages the input parameters for mixing audio tracks. It allows custom audio processing to be performed on audio tracks during playback or other operations. You can also set the relative volumes and ramping of audio tracks.

A video composition describes, for any time in the aggregate time range of its instructions, the number and IDs of video tracks that are to be used in order to produce a composed video frame corresponding to that time. When AV Foundation’s built-in video compositor is used, the instructions a video composition contains can specify a spatial transformation, an opacity value, and a cropping rectangle for each video source, and these can vary over time via simple linear ramping functions. An AVVideoComposition object represents an immutable video composition. The AV Foundation framework also provides a mutable subclass, AVMutableVideoComposition, that you can use to create new videos.

It’s quite possible to construct elaborate and complex compositions, video compositions, and audio mixes with entirely valid values that produce unexpected results. It's possible, for example, for a valid video composition to produce frames from which the contents of the source video tracks are entirely absent.

This document discusses some common pitfalls when constructing these, and ways to more easily debug them.

Common pitfalls

Here are some of the common types of problems you may encounter when building complex compositions, video compositions and audio mixes.


Misaligned track segments

The alignment of the tracks might be wrong. For example, this can happen as a result of CMTime value rounding errors when inserting tracks into a composition for a specific time.

Figure 1  Misaligned track segments.

Video Compositions, Audio Mixes

Gaps between segments

Don’t leave gaps between the segments in a video composition. This will almost certainly result in black frames, or a continuation of the last frame.

Figure 2  Gaps between segments.

Misaligned layer instructions

There could be a misalignment between the video composition time ranges and the composition track segments.

Figure 3  Misaligned layer instructions.

Misaligned opacity/audio ramps

You might have made a mistake in one of the ramps — by making it too long, for example.

Figure 4  Misaligned opacity/audio ramps.

Bogus layer transforms

You may have an error in the layer transformation matrix. This can cause your output to spin off beyond the boundaries of the output frame.

Figure 5  Bogus layer transforms.

Static frame for the entire duration of the source asset

You may have a video composition that produces a single static frame for the entire duration of the source asset. This can occur if you set the frameDuration of the AVMutableVideoComposition to the asset's duration. In the processing by the video compositor, only one frame would be produced, at the start time, and then "held" for the specified duration, in spite of the instructions for varying opacities, transforms, etc.

Figure 6  Static frame for the entire duration of the source asset.

Debugging Compositions, Video Compositions and Audio Mixes

What’s the best way to debug a complex composition, video composition and audio mix?

Visualize the Composition

One technique that works well is to visualize these. Rather than looking at your code, you look at a picture of your composition, video composition and audio mix.

There’s sample code that has been designed just for this purpose:



These sample applications implement a custom AVCompositionDebugView class which presents a visual description of the underlying AVComposition, AVVideoComposition and AVAudioMix objects which form the composition. See Figure 7.

Simply drop the AVCompositionDebugView into your own app. It’s a non-interactive view. Extend it to draw your own video instructions. It will help you spot any overlaps and gaps in the composition tracks, video instructions, and the audio mix.

Figure 7  The AVCompositionDebugViewer app.

Debugging Video Compositions

AVVideoCompositionValidationHandling protocol

The AVVideoCompositionValidationHandling protocol declares methods that you can implement in the delegate of an AVVideoComposition object to indicate whether validation of a video composition should continue after specific errors have been found:





In the course of validation with the isValid(for:timeRange:validationDelegate:) method, the receiver will invoke its validationDelegate with reference to any trouble spots in the video composition.

You should implement the methods defined by the AVVideoCompositionValidationHandling protocol to aid in the debugging of a video composition.

Additional Information

For more information about the editing of audiovisual assets with AVFoundation, see the Editing chapter of the AV Foundation Programming Guide.

Document Revision History




New document that discusses techniques for debugging complex AVFoundation Compositions, Video Compositions, and Audio Mixes