Media Layer

Assets Library Framework

Introduced in iOS 4.0, the Assets Library framework (AssetsLibrary.framework) provides a query-based interface for retrieving photos and videos from the user’s device. Using this framework, you can access the same assets that are normally managed by the Photos application, including items in the user’s saved photos album and any photos and videos that were imported onto the device. You can also save new photos and videos back to the user’s saved photos album.

For more information about the classes and methods of this framework, see Assets Library Framework Reference.

AV Foundation Framework

Introduced in iOS 2.2, the AV Foundation framework (AVFoundation.framework) contains Objective-C classes for playing audio content. You can use these classes to play file- or memory-based sounds of any duration. You can play multiple sounds simultaneously and control various playback aspects of each sound. In iOS 3.0 and later, this framework also includes support for recording audio and managing audio session information.

In iOS 4.0 and later, the services offered by this framework were expanded to include:

• Media asset management

• Media editing

• Movie capture

• Movie playback

• Track management

• Metadata management for media items

• Stereophonic panning

• Precise synchronization between sounds

• An Objective-C interface for determining details about sound files, such as the data format, sample rate, and number of channels

In iOS 5, the AV Foundation framework includes support for streaming audio and video content over AirPlay using the AVPlayer class. AirPlay support is enabled by default, but applications can opt out as needed.

The AV Foundation framework is a single source for recording and playing back audio and video in iOS. This framework also provides much more sophisticated support for handling and managing media items than higher-level frameworks.

For more information about the classes of the AV Foundation framework, see AV Foundation Framework Reference.

Core Audio

Native support for audio is provided by the Core Audio family of frameworks, which are listed in Table 2-1. Core Audio is a C-based interface that supports the manipulation of stereo-based audio. You can use Core Audio in iOS to generate, record, mix, and play audio in your applications. You can also use Core Audio to trigger the vibrate capability on devices that support it.

For more information about Core Audio, see Core Audio Overview. For information about how to use the Audio Toolbox framework to play sounds, see Audio Queue Services Programming Guide.

Core Graphics Framework

The Core Graphics framework (CoreGraphics.framework) contains the interfaces for the Quartz 2D drawing API. Quartz is the same advanced, vector-based drawing engine that is used in OS X. It provides support for path-based drawing, anti-aliased rendering, gradients, images, colors, coordinate-space transformations, and PDF document creation, display, and parsing. Although the API is C based, it uses object-based abstractions to represent fundamental drawing objects, making it easy to store and reuse your graphics content.

For more information on how to use Quartz to draw content, see Quartz 2D Programming Guide and Core Graphics Framework Reference.

Core Image Framework

Introduced in iOS 5, the Core Image framework (CoreImage.framework) provides a powerful set of built-in filters for manipulating video and still images. You can use the built-in filters for everything from simple operations (like touching up and correcting photos) to more advanced operations (like face and feature detection). The advantage of using these filters is that they operate in a nondestructive manner so that your original images are never changed directly. In addition, Core Image takes advantage of the available CPU and GPU processing power to ensure that operations are fast and efficient.

The CIImage class provides access to a standard set of filters that you can use to improve the quality of a photograph. To create other types of filters, you can create and configure a CIFilter object for the appropriate filter type.

For information about the classes and filters of the Core Image framework, see Core Image Reference Collection.

Core MIDI Framework

Introduced in iOS 4.2, the Core MIDI framework (CoreMIDI.framework) provides a standard way to communicate with MIDI devices, including hardware keyboards and synthesizers. You use this framework to send and receive MIDI messages and to interact with MIDI peripherals connected to an iOS-based device using the dock connector or network.

For more information about using this framework, see Core MIDI Framework Reference.

Core Text Framework

Introduced in iOS 3.2, the Core Text framework (CoreText.framework) contains a set of simple, high-performance C-based interfaces for laying out text and handling fonts. The Core Text framework provides a complete text layout engine that you can use to manage the placement of text on the screen. The text you manage can also be styled with different fonts and rendering attributes.

This framework is intended for use by applications that require sophisticated text handling capabilities, such as word-processing applications. If your application requires only simple text input and display, you should continue to use the existing text classes of the UIKit framework.

For more information about using the Core Text interfaces, see Core Text Programming Guide and Core Text Reference Collection.

Core Video Framework

Introduced in iOS 4.0, the Core Video framework (CoreVideo.framework) provides buffer and buffer pool support for the Core Media framework (described in “Core Media Framework”). Most applications never need to use this framework directly.

Image I/O Framework

Introduced in iOS 4.0, the Image I/O framework (ImageIO.framework) provides interfaces for importing and exporting image data and image metadata. This framework makes use of the Core Graphics data types and functions and supports all of the standard image types available in iOS.

In iOS 6 and later, you can use this framework to access EXIF and IPTC metadata properties for images.

For more information about the functions and data types of this framework, see Image I/O Reference Collection.

GLKit Framework

Introduced in iOS 5, the GLKit framework (GLKit.framework) contains a set of Objective-C based utility classes that simplify the effort required to create an OpenGL ES 2.0 application. GLKit provides support for four key areas of application development:

• The GLKView and GLKViewController classes provide a standard implementation of an OpenGL ES–enabled view and associated rendering loop. The view manages the underlying framebuffer object on behalf of the application; your application just draws to it.

• The GLKTextureLoader class provides image conversion and loading routines to your application, allowing it to automatically load texture images into your context. It can load textures synchronously or asynchronously. When loading textures asynchronously, your application provides a completion handler block to be called when the texture is loaded into your context.

• The GLKit framework provides implementations of vector, matrix, and quaternions as well as a matrix stack operation that provides the same functionality found in OpenGL ES 1.1.

• The GLKBaseEffect, GLKSkyboxEffect, and GLKReflectionMapEffect classes provide existing, configurable graphics shaders that implement commonly used graphics operations. In particular, the GLKBaseEffect class implements the lighting and material model found in the OpenGL ES 1.1 specification, simplifying the effort required to migrate an application from OpenGL ES 1.1 to OpenGL ES 2.0.

For information about the classes of the GLKit framework, see GLKit Framework Reference.

Media Player Framework

The Media Player framework (MediaPlayer.framework) provides high-level support for playing audio and video content from your application. You can use this framework to play video using a standard system interface.

In iOS 3.0, support was added for accessing the user’s iTunes music library. With this support, you can play music tracks and playlists, search for songs, and present a media picker interface to the user.

In iOS 3.2, changes were made to support the playback of video from a resizable view. (Previously, only full-screen support was available.) In addition, numerous interfaces were added to support the configuration and management of movie playback.

In iOS 5, support was added for displaying “Now Playing” information in the lock screen and multitasking controls. This information can also be displayed on an Apple TV and with content delivered via AirPlay. There are also interfaces for detecting whether video is being streamed over AirPlay.

For information about the classes of the Media Player framework, see Media Player Framework Reference. For information on how to use these classes to access the user’s iTunes library, see iPod Library Access Programming Guide.

OpenAL Framework

The Open Audio Library (OpenAL) interface is a cross-platform standard for delivering positional audio in applications. You can use it to implement high-performance, high-quality audio in games and other programs that require positional audio output. Because OpenAL is a cross-platform standard, the code modules you write using OpenAL on iOS can be ported to many other platforms easily.

For information about OpenAL, including how to use it, see http://www.openal.org.

OpenGL ES Framework

The OpenGL ES framework (OpenGLES.framework) provides tools for drawing 2D and 3D content. It is a C-based framework that works closely with the device hardware to provide high frame rates for full-screen game-style applications.

You always use the OpenGL framework in conjunction with the EAGL interfaces. These interfaces are part of the OpenGL ES framework and provide the interface between your OpenGL ES drawing code and the native window objects defined by UIKit.

In iOS 3.0 and later, the OpenGL ES framework includes support for both the OpenGL ES 2.0 and the OpenGL ES 1.1 interface specifications. The 2.0 specification provides support for fragment and vertex shaders and is available only on specific iOS-based devices running iOS 3.0 and later. Support for OpenGL ES 1.1 is available on all iOS-based devices and in all versions of iOS.

For information on how to use OpenGL ES in your applications, see OpenGL ES Programming Guide for iOS. For reference information, see OpenGL ES Framework Reference.

Quartz Core Framework

The Quartz Core framework (QuartzCore.framework) contains the Core Animation interfaces. Core Animation is an advanced animation and compositing technology that uses an optimized rendering path to implement complex animations and visual effects. It provides a high-level Objective-C interface for configuring animations and effects that are then rendered in hardware for performance. Core Animation is integrated into many parts of iOS, including UIKit classes such as UIView, providing animations for many standard system behaviors. You can also use the Objective-C interface in this framework to create custom animations.

For more information on how to use Core Animation in your applications, see Core Animation Programming Guide and Core Animation Reference Collection.