CMSampleBuffer

An object containing zero or more media samples of a uniform media type.

Overview

CMSampleBuffer is a Core Foundation object containing zero or more compressed (or uncompressed) samples of a particular media type (audio, video, muxed, etc), that are used to move media sample data through the media pipeline. A CMSampleBuffer can contain:

  • A CMBlockBuffer of one or more media samples, or

  • A CVImageBuffer, a reference to the format description for the stream of CMSampleBuffers, size and timing information for each of the contained media samples, and both buffer-level and sample-level attachments.

A sample buffer can contain both sample-level and buffer-level attachments. Sample-level attachments are associated with each individual sample (frame) in a buffer and include information such as timestamps and video frame dependencies. You can read and write sample-level attachments using the CMSampleBufferGetSampleAttachmentsArray(_:_:) function. Buffer-level attachments provide information about the buffer as a whole, such as playback speed and actions to be performed upon consuming the buffer. You can read and write buffer-level attachments using the APIs described in CMAttachment and the keys listed under Sample Buffer Attachment Keys.

It is possible for a CMSampleBuffer to describe samples it does not yet contain. For example, some media services may have access to sample size, timing and format information before the data is read. Such services may create CMSampleBuffers with that information and insert them into queues early, and attach (or fill) the CMBlockBuffers of media data later, when the data becomes ready. To this end, CMSampleBuffers have the concept of data-readiness, which can be tested, set, forced to become ready “now" and so on. It is also possible for a CMSampleBuffer to contain nothing but a special buffer-level attachment that describes a media stream event (for example, "discontinuity: drain and reset decoder before processing the next CMSampleBuffer”). Such a special attachment can also be attached to regular CMSampleBuffers (i.e. that contain media sample data), and if so, the event it describes is defined to occur after the samples in that CMSampleBuffer.

Topics

Creating Sample Buffers

Modifying Sample Buffers

func CMSampleBufferSetDataFailed(CMSampleBuffer, OSStatus)

Marks a sample buffer's data as "failed", to indicate that the data will not become ready.

func CMSampleBufferCopySampleBufferForRange(CFAllocator?, CMSampleBuffer, CFRange, UnsafeMutablePointer<CMSampleBuffer?>)

Creates a CMSampleBuffer containing a range of samples from an existing CMSampleBuffer.

func CMSampleBufferInvalidate(CMSampleBuffer)

Makes the sample buffer invalid, calling any installed invalidation callback.

func CMSampleBufferMakeDataReady(CMSampleBuffer)

Makes a CMSampleBuffer data ready, by calling the client's CMSampleBufferMakeDataReadyCallback.

func CMSampleBufferSetDataBuffer(CMSampleBuffer, CMBlockBuffer)

Associates a CMSampleBuffer with its CMBlockBuffer of media data.

func CMSampleBufferSetDataReady(CMSampleBuffer)

Marks a CMSampleBuffer's data as "ready".

func CMSampleBufferSetInvalidateCallback(CMSampleBuffer, CMSampleBufferInvalidateCallback, UInt64)

Sets the CMSampleBuffer’s invalidation callback, which is called during CMSampleBufferInvalidate.

func CMSampleBufferSetOutputPresentationTimeStamp(CMSampleBuffer, CMTime)

Sets an output presentation timestamp to be used in place of a calculated value.

func CMSampleBufferTrackDataReadiness(CMSampleBuffer, CMSampleBuffer)

Associates a CMSampleBuffer's data readiness with another CMSampleBuffer's data readiness.

Inspecting Sample Buffers

func CMSampleBufferGetDataBuffer(CMSampleBuffer)

Returns a CMSampleBuffer's CMBlockBuffer of media data.

func CMSampleBufferGetDecodeTimeStamp(CMSampleBuffer)

Returns the numerically earliest decode timestamp of all the samples in a CMSampleBuffer.

func CMSampleBufferGetDuration(CMSampleBuffer)

Returns the total duration of a CMSampleBuffer.

func CMSampleBufferGetFormatDescription(CMSampleBuffer)

Returns the format description of the samples in a CMSampleBuffer.

func CMSampleBufferGetImageBuffer(CMSampleBuffer)

Returns a sample buffer's CVImageBuffer of media data.

func CMSampleBufferGetNumSamples(CMSampleBuffer)

Returns the number of media samples in a CMSampleBuffer.

func CMSampleBufferGetOutputDecodeTimeStamp(CMSampleBuffer)

Returns the output decode timestamp of the CMSampleBuffer.

func CMSampleBufferGetOutputDuration(CMSampleBuffer)

Returns the output duration of a CMSampleBuffer.

func CMSampleBufferGetOutputPresentationTimeStamp(CMSampleBuffer)

Returns the output presentation timestamp of the CMSampleBuffer.

func CMSampleBufferGetPresentationTimeStamp(CMSampleBuffer)

Returns the numerically earliest presentation timestamp of all the samples in a CMSampleBuffer.

func CMSampleBufferGetSampleAttachmentsArray(CMSampleBuffer, Bool)

Returns a reference to a CMSampleBuffer's immutable array of mutable sample attachments dictionaries (one dictionary per sample in the CMSampleBuffer).

func CMSampleBufferGetSampleSize(CMSampleBuffer, CMItemIndex)

Returns the size in bytes of a specified sample in a CMSampleBuffer.

func CMSampleBufferGetSampleTimingInfo(CMSampleBuffer, CMItemIndex, UnsafeMutablePointer<CMSampleTimingInfo>)

Retrieves a CMSampleTimingInfo struct describing a specified sample in a CMSampleBuffer.

func CMSampleBufferGetTotalSampleSize(CMSampleBuffer)

Returns the total size in bytes of sample data in a CMSampleBuffer.

func CMSampleBufferHasDataFailed(CMSampleBuffer, UnsafeMutablePointer<OSStatus>?)

Indicates if the sample buffer's data loading request has failed.

func CMSampleBufferDataIsReady(CMSampleBuffer)

Determines if the sample buffer's data is ready.

func CMSampleBufferIsValid(CMSampleBuffer)

Queries whether a sample buffer is still valid.

func CMSampleBufferGetTypeID()

Returns the CFTypeID of CMSampleBuffer objects.

Working with Audio Samples

func CMSampleBufferGetAudioStreamPacketDescriptions(CMSampleBuffer, Int, UnsafeMutablePointer<AudioStreamPacketDescription>?, UnsafeMutablePointer<Int>?)

Creates an array of AudioStreamPacketDescriptions for the variable bytes per packet or variable frames per packet audio data in the provided CMSampleBuffer.

func CMSampleBufferGetAudioStreamPacketDescriptionsPtr(CMSampleBuffer, UnsafeMutablePointer<UnsafePointer<AudioStreamPacketDescription>?>?, UnsafeMutablePointer<Int>?)

Retrieves a pointer to a constant array of AudioStreamPacketDescriptions for the variable bytes per packet or variable frames per packet audio data in the provided CMSampleBuffer.

func CMSampleBufferCopyPCMDataIntoAudioBufferList(CMSampleBuffer, Int32, Int32, UnsafeMutablePointer<AudioBufferList>)

Copies PCM audio data from the given sample buffer into a pre-populated AudioBufferList.

Data Types

typealias CMBuffer

A reference to a CMBuffer object.

class CMSampleBuffer

A reference to an immutable CMSampleBufferRef object.

typealias CMItemIndex

Data type for the item index.

typealias CMItemCount

Data type for the item count.

typealias CMPersistentTrackID

Data type for the persistent track ID.

struct CMSampleTimingInfo

A collection of timing information for a sample in a CMSampleBuffer.

typealias CMBufferGetSizeCallback

Client callback that returns a size_t from a CMBuffer.

typealias CMMuxedStreamType

Muxed media format/subtype.

Constants

Sample Buffer Error Codes

These result codes are used for CMSampleBuffer APIs.

Sample Buffer Notifications

Notifications posted by sample buffer objects.

Sample Buffer Notification Parameters

Keys used in the userInfo dictionaries of notifications posted by sample buffer objects.

Sample Attachment Keys

Attachments associated with individual samples in a buffer.

Sample Buffer Attachment Keys

Buffer-level attachments associated with a sample buffer.

Sample Buffer Dropped Frame Reasons

Values providing the reason for a dropped frame.

Sample Buffer Dropped Frame Reason Information

Values providing additional information about the reason for a dropped frame.

Sample Buffer Lens Stabilization Reasons

Values providing the reason for lens stabilization.

Audio Buffer Alignment Constants

Flags passed to various audio-related functions.

Enumeration

See Also

Sample Processing

CMBlockBuffer

An object used to move blocks of memory through a processing system.

CMFormatDescription

A media format descriptor that describes the samples in a sample buffer.

CMAttachment

An API for attaching additional metadata to a sample buffer.