CMSampleBuffer

An object containing zero or more media samples of a uniform media type.

Overview

CMSampleBuffer is a Core Foundation object containing zero or more compressed (or uncompressed) samples of a particular media type (audio, video, muxed, etc), that are used to move media sample data through the media pipeline. A CMSampleBuffer can contain:

  • A CMBlockBuffer of one or more media samples, or

  • A CVImageBuffer, a reference to the format description for the stream of CMSampleBuffers, size and timing information for each of the contained media samples, and both buffer-level and sample-level attachments.

A sample buffer can contain both sample-level and buffer-level attachments. Sample-level attachments are associated with each individual sample (frame) in a buffer and include information such as timestamps and video frame dependencies. You can read and write sample-level attachments using the CMSampleBufferGetSampleAttachmentsArray(_:createIfNecessary:) function. Buffer-level attachments provide information about the buffer as a whole, such as playback speed and actions to be performed upon consuming the buffer. You can read and write buffer-level attachments using the APIs described in CMAttachment and the keys listed under Sample Buffer Attachment Keys.

It is possible for a CMSampleBuffer to describe samples it does not yet contain. For example, some media services may have access to sample size, timing and format information before the data is read. Such services may create CMSampleBuffers with that information and insert them into queues early, and attach (or fill) the CMBlockBuffers of media data later, when the data becomes ready. To this end, CMSampleBuffers have the concept of data-readiness, which can be tested, set, forced to become ready “now" and so on. It is also possible for a CMSampleBuffer to contain nothing but a special buffer-level attachment that describes a media stream event (for example, "discontinuity: drain and reset decoder before processing the next CMSampleBuffer”). Such a special attachment can also be attached to regular CMSampleBuffers (i.e. that contain media sample data), and if so, the event it describes is defined to occur after the samples in that CMSampleBuffer.

Topics

Creating Sample Buffers

Modifying Sample Buffers

func CMSampleBufferSetDataFailed(CMSampleBuffer, status: OSStatus) -> OSStatus

Marks a sample buffer's data as "failed", to indicate that the data will not become ready.

func CMSampleBufferInvalidate(CMSampleBuffer) -> OSStatus

Makes the sample buffer invalid, calling any installed invalidation callback.

func CMSampleBufferMakeDataReady(CMSampleBuffer) -> OSStatus

Makes a CMSampleBuffer data ready, by calling the client's CMSampleBufferMakeDataReadyCallback.

func CMSampleBufferSetDataBuffer(CMSampleBuffer, newValue: CMBlockBuffer) -> OSStatus

Associates a CMSampleBuffer with its CMBlockBuffer of media data.

func CMSampleBufferSetDataReady(CMSampleBuffer) -> OSStatus

Marks a CMSampleBuffer's data as "ready".

func CMSampleBufferSetInvalidateCallback(CMSampleBuffer, callback: CMSampleBufferInvalidateCallback, refcon: UInt64) -> OSStatus

Sets the CMSampleBuffer’s invalidation callback, which is called during CMSampleBufferInvalidate.

func CMSampleBufferSetOutputPresentationTimeStamp(CMSampleBuffer, newValue: CMTime) -> OSStatus

Sets an output presentation timestamp to be used in place of a calculated value.

func CMSampleBufferTrackDataReadiness(CMSampleBuffer, sampleBufferToTrack: CMSampleBuffer) -> OSStatus

Associates a CMSampleBuffer's data readiness with another CMSampleBuffer's data readiness.

Inspecting Sample Buffers

func CMSampleBufferGetDataBuffer(CMSampleBuffer) -> CMBlockBuffer?

Returns a CMSampleBuffer's CMBlockBuffer of media data.

func CMSampleBufferGetDecodeTimeStamp(CMSampleBuffer) -> CMTime

Returns the numerically earliest decode timestamp of all the samples in a CMSampleBuffer.

func CMSampleBufferGetDuration(CMSampleBuffer) -> CMTime

Returns the total duration of a CMSampleBuffer.

func CMSampleBufferGetFormatDescription(CMSampleBuffer) -> CMFormatDescription?

Returns the format description of the samples in a CMSampleBuffer.

func CMSampleBufferGetImageBuffer(CMSampleBuffer) -> CVImageBuffer?

Returns a sample buffer's CVImageBuffer of media data.

func CMSampleBufferGetNumSamples(CMSampleBuffer) -> CMItemCount

Returns the number of media samples in a CMSampleBuffer.

func CMSampleBufferGetOutputDecodeTimeStamp(CMSampleBuffer) -> CMTime

Returns the output decode timestamp of the CMSampleBuffer.

func CMSampleBufferGetOutputDuration(CMSampleBuffer) -> CMTime

Returns the output duration of a CMSampleBuffer.

func CMSampleBufferGetOutputPresentationTimeStamp(CMSampleBuffer) -> CMTime

Returns the output presentation timestamp of the CMSampleBuffer.

func CMSampleBufferGetPresentationTimeStamp(CMSampleBuffer) -> CMTime

Returns the numerically earliest presentation timestamp of all the samples in a CMSampleBuffer.

func CMSampleBufferGetSampleAttachmentsArray(CMSampleBuffer, createIfNecessary: Bool) -> CFArray?

Returns a reference to a CMSampleBuffer's immutable array of mutable sample attachments dictionaries (one dictionary per sample in the CMSampleBuffer).

func CMSampleBufferGetSampleSize(CMSampleBuffer, at: CMItemIndex) -> Int

Returns the size in bytes of a specified sample in a CMSampleBuffer.

func CMSampleBufferGetTotalSampleSize(CMSampleBuffer) -> Int

Returns the total size in bytes of sample data in a CMSampleBuffer.

func CMSampleBufferDataIsReady(CMSampleBuffer) -> Bool

Determines if the sample buffer's data is ready.

func CMSampleBufferIsValid(CMSampleBuffer) -> Bool

Queries whether a sample buffer is still valid.

func CMSampleBufferGetTypeID() -> CFTypeID

Returns the CFTypeID of CMSampleBuffer objects.

Working with Audio Samples

func CMSampleBufferGetAudioStreamPacketDescriptionsPtr(CMSampleBuffer, packetDescriptionsPointerOut: UnsafeMutablePointer<UnsafePointer<AudioStreamPacketDescription>?>?, sizeOut: UnsafeMutablePointer<Int>?) -> OSStatus

Retrieves a pointer to a constant array of AudioStreamPacketDescriptions for the variable bytes per packet or variable frames per packet audio data in the provided CMSampleBuffer.

Data Types

typealias CMBuffer

A reference to a CMBuffer object.

class CMSampleBuffer

A reference to an immutable CMSampleBufferRef object.

typealias CMItemIndex

Data type for the item index.

typealias CMItemCount

Data type for the item count.

typealias CMPersistentTrackID

Data type for the persistent track ID.

struct CMSampleTimingInfo

A collection of timing information for a sample in a CMSampleBuffer.

typealias CMBufferGetSizeCallback

Client callback that returns a size_t from a CMBuffer.

typealias CMMuxedStreamType

Muxed media format/subtype.

Constants

Sample Buffer Error Codes

These result codes are used for CMSampleBuffer APIs.

Sample Buffer Notifications

Notifications posted by sample buffer objects.

Sample Buffer Notification Parameters

Keys used in the userInfo dictionaries of notifications posted by sample buffer objects.

Sample Attachment Keys

Attachments associated with individual samples in a buffer.

Sample Buffer Attachment Keys

Buffer-level attachments associated with a sample buffer.

Sample Buffer Dropped Frame Reasons

Values providing the reason for a dropped frame.

Sample Buffer Dropped Frame Reason Information

Values providing additional information about the reason for a dropped frame.

Sample Buffer Lens Stabilization Reasons

Values providing the reason for lens stabilization.

Audio Buffer Alignment Constants

Flags passed to various audio-related functions.

Enumeration

See Also

Sample Processing

CMBlockBuffer

An object used to move blocks of memory through a processing system.

CMFormatDescription

A media format descriptor that describes the samples in a sample buffer.

CMAttachment

An API for attaching additional metadata to a sample buffer.