Supported Features

QuickTime 7.2.1 provides the following capabilities:

• Export for Web, which exports to iPhone, iPhone (cellular), and Desktop versions of given source movies.

• The creation of reference movies that point to all exported versions of the movie. The ref movie uses relative paths to point to each version. This requires that users keep the reference movie and the versions in the same folder on the server, but allows users to create the movie without specifying URLs and allows them to move the ref movie and its sources as a group without breaking the reference movie.

• Export that generates a poster frame image from the movie’s poster frame that can be used by the embed tag (in the case of the iPhone, for example).

• The generation of an embed tag that includes the poster frame specification and reference movie URL in an HTML snippet. The HTML snippet displays an HTML textfield with the embed tag, as well as help text for inserting the HTML and posting the movies. The HTML file also includes instructions on how to use the exported movie files, poster image, and the HTML snippet.

• The ability to select the folder where the results of the iPhone reference movie creation operation is written. A single subfolder is created in the specified folder to contain all the files.

How It Works

To take advantage of this new capability, follow these steps:

1. Launch QuickTime Player Pro.

2. Select File > Export for Web...

   A dialog appears.

3. Enter the data you want to specify in the fields and checkboxes of the dialog box, as shown in Figure 1-2. In this step, you can specify the name, location of your export, and for which export versions you want to save, that is, iPhone, iPhone (cellular), and Desktop.

   

4. Click Export to export these versions of your QuickTime movie for optimized web delivery to the iPhone and the desktop.

When the user starts the export, the following files are placed in the specified folder (where NAME is the user-specified name):

• NAME-iPhone.m4v

• NAME-iPhone-cell.3gp

• NAME-desktop.m4v

• NAME-poster.jpg

• NAME.mov

• ReadMe.html

For more technical information about exporting movies for iPod, AppleTV, and iPhone, refer to Technical Note TN2188. If you’re a web developer working with the iPhone, see Optimizing Web Applications and Content for iPhone, which provides guidelines that will help you prepare web content and design a website or web-based application for iPhone.

Frame-Accurate Capture and Audio/Video Synchronization

The new capture classes and methods available in the QuickTime Kit provide frame-accurate audio/video synchronization, and frame-accurate capture, meaning you can specify precisely––with timecodes––when you want capturing to occur. You also have access to transport controls of your camcorder, so you can fast forward and rewind the tape.

Note that this is implemented as a brand new capture engine––not simply a wrapper around the Sequence Grabber API.

Using these classes and methods, you can capture media from one or more external sources, including cameras, microphones, and other external media devices, such as capture cards and tapedecks.

After you’ve captured this media, you can record it to one or more output destinations, including but not necessarily limited to the following:

• A QuickTime movie (.mov) file

• A Cocoa view that previews video media captured from the input sources

The devices supported in Mac OS X v10.5 include:

• UDC over USB, including Apple’s built-in iSight camera

• IIDC over FireWire, includes external iSight

• DV cameras

• HDV and Pro DV devices (with the appropriate codecs included with Final Cut Pro). Also, the AVC DAL plug-in, which handles both HDV and DV cameras

• Core Audio HAL devices

• Other devices that have VDIGs through a VDIG shim

How QTKit Capture Works

All QTKit capture applications make use of three basic types of objects: capture inputs, capture outputs, and a capture session. Capture inputs, which are subclasses of QTCaptureInput, provide the necessary interfaces to different sources of captured media.

A capture device input, which is a QTCaptureDeviceInput object––a subclass of QTCaptureInput––provides an interface to capturing from various audio/video hardware, such as cameras and microphones. Capture outputs, which are subclasses of QTCaptureOutput, provide the necessary interfaces to various destinations for media, such as QuickTime movie files, or video and audio previews.

A capture session, which is a QTCaptureSession object, manages how media that is captured from connected input sources is distributed to connected output destinations. Each input and output has one or more connection, which represents a media stream of a certain QuickTime media type, such as video or audio media. A capture session will attempt to connect all input connections to each of its outputs.

As shown in Figure 1-3, a capture session works by connecting inputs to outputs in order to record and preview video from a camera.

A capture session works by distributing the video from its single video input connection to a connection owned by each output. In addition to distributing separate media streams to each output, the capture session is also responsible for mixing the audio from multiple inputs down to a single stream.

Figure 1-4 shows how the capture session handles multiple audio inputs.

As illustrated in Figure 1-4, a capture session sends all of its input video to each output that accepts video and all of its input audio to each output that accepts audio. However, before sending the separate audio stream to its outputs, it mixes them down to one stream that can be sent to a single capture connection.

A capture session is also responsible for ensuring that all media are synchronized to a single time base in order to guarantee that all output video and audio are synchronized.

The connections belonging to each input and output are QTCaptureConnection objects. These describe the media type and format of each stream taken from an input or sent to an output. By referencing a specific connection, your application can have finer-grained control over which media enters and leaves a session. Thus, you can enable and disable specific connections, and control specific attributes of the media entering (for example, the volumes of specific audio channels).

For more information, refer to the QTKit Capture Programming Guide.

Using the QTKit Capture API

The QTKit capture API is comprised of fifteen new classes and more than several hundred new methods, notifications, and attributes. To better understand how you can take advantage of this API in your Cocoa or QuickTime application, you may want to read this section describing the various groupings of the API. The complete description of all these classes and their associated methods is available in the QuickTime Kit Framework Reference.

There are four base classes, six classes devoted to input and output, another three utility classes, one class that deals with device input, and another with the user interface.

Base QTKit Capture Classes

There are four classes in this group that can best be described as base classes, as described in Table 1-1. Understanding how these work is essential to using the QTKit capture API.

The QTCaptureSession class provides an interface for connecting input sources. The method used most commonly in this class is startRunning, which tells the receiver to start capturing data from its inputs and then to send that data to its outputs. Notably, if you’re using this method, when data does not need to be sent to file outputs, previews, or other outputs, your capture session should not be running, so that the overhead from capturing does not affect the performance of your application.

The QTCaptureInput and QTCaptureOutput classes, which are both abstract classes, provide interfaces for connecting inputs and outputs. An input source can have multiple connections, which is common for many cameras which have both audio and video output streams. Using QTCaptureOutput objects, you don’t need to have a fixed number of connections, but you do need a destination for your capture session and all of its input data.

Input and Output QTKit Capture Classes

There are five output classes and only one input class belonging to this group, shown in Table 1-2.

You can use the methods available in the QTCaptureDeviceInput class to handle input sources for various media devices, such as cameras and microphones. The five output classes provide output destinations for QTCaptureSession objects that can be used to write captured media to QuickTime movies, for example, or to preview video or audio that is being captured. QTCaptureFileOutput, an abstract superclass, provides an output destination for a capture sesson to write captured media simply to files.

Utility QTKit Capture Classes

There are three classes belonging to this group: QTCompressionOptions, QTFormatDescription, and QTSampleBuffer, shown in Table 1-3. These are best characterized as utility classes, in that they perform tasks related to representing, for example, the compressions for particular media, or describing the formats of various media samples.

You can use QTCompressionOptions to describe compression options for all kinds of different media, using the compressionOptionsIdentifiersForMediaType: and mediaType methods. Compression options are created from presets keyed by a named identifier. These preset identifiers are listed in the QuickTime Kit Framework Reference in the chapter describing this class.

Using QTSampleBuffer objects, you can get information about sample buffer data that you may need to output or process the media samples in the buffer.

Device Access and User Interface QTKit Capture Classes

There are two classes in this particular group: QTCaptureDevice and QTCaptureView, as shown in Table 1-4.

Each instance of QTCaptureDevice corresponds to a capture device that is connected or has been previously connected to the user’s computer during the lifetime of the application. Note that instances of this class cannot be created directly. Whenever a device is connected to the computer, a single unique instance is created automatically and can be accessed using the deviceWithUniqueID: class method. An array of all currently connected devices can also be obtained using the inputDevices: class method.

Devices can provide one or more stream of a given media type. Your application can search for devices that provide media of a specific type using the inputDevicesWithMediaType: and defaultInputDeviceWithMediaType class methods.

You can use the methods available in the QTCaptureView class, which is a subclass of NSView, to preview video that is being processed by an instance of QTCaptureSession. The class creates and maintains its own QTCaptureVideoPreviewOutput to gather the preview video you need from the capture session.

Developing Applications with QTKit Capture

To take advantage of the new QuickTime Kit capture classes, you can do the following:

• Read the QTKit Capture Programming Guide. The guide discusses how you can build a functioning capture player application that controls the capture of QuickTime movies, adding simple start and stop buttons, and allowing you to output and display your captured files in QuickTime Player as QuickTime movies. For this project, you need an iSight camera, either built-in or plugged into your Macintosh. You also need to have Mac OS X v10.5 installed on your computer. To implement this capture player, you won’t have to write more than 20 or 30 lines of Objective-C code.

  • Using Xcode 3 as your integrated development environment (IDE), along with the new implementation of Interface Builder 3, you’ll see how easy it is to work with the QuickTime Kit framework. In this example, you use the new QTKit capture control provided in the library of controls available in Interface Builder 3. The QTKit capture control performs much of the work for you in implementing the design of the user interface for this application.

• Check out the QTKit capture sample code projects, StillMotion and MyRecorder.

  • The StillMotion sample, available for download at StillMotion, demonstrates how to use the QTKit capture decompressed video output API to build a simple application that creates stop motion animations. The application lets you capture individual frames taken from a video camera connected to the computer and assemble them into a QuickTime movie that can be saved to disk.

  • The MyRecorder code sample, available for download at MyRecorder, is designed to support professional-level video and audio capture, and pro-grade recording of media.

  The QTKit Capture Programming Guide walks you through both code samples, taking a tutorial approach, so you can learn as you go, step-by-step through your project, using the tools available in Xcode 3 and Interface Builder 3. You get to construct and then extend the functionality of the capture player application, adding a minimum of Objective-C code. This is the ideal way to get up to speed with the new QTKit capture methods and classes.

Dealing with Thread-Safety Issues

In the version of QuickTime 7.2.1 that ships with Mac OS X v10.5, new methods are provided that deal with thread-safety. These methods allow applications to manage QTMovie objects on current (non-main) threads. Specifically, five new methods belonging to the QTMovie class have been added. These include the following class and instance methods that deal with handling and managing thread-safety operations of movie objects: enterQTKitOnThread, enterQTKitOnThreadDisablingThreadSafetyProtection, exitQTKitOnThread, attachToCurrentThread, and detachFromCurrentThread. For more information about these new methods, refer to their descriptions in the QTMovie Class Reference.

For more information on thread-safety in QuickTime, refer to Technote TN2125, which discusses in detail best programming practices to ensure thread-safety in your application.

New Classes to Support Core Animation

Two new classes that provide support for Core Animation have been added in QuickTime 7.2.1. Both are subclasses of CALayer.

• QTCaptureLayer. Provides a layer that displays video frames currently being captured from a device attached to the computer, and is intended to provide support for drawing the contents of a capture session into a layer. Note that this class requires rendering using visual contexts.

• QTMovieLayer. Provides a layer into which the frames of a QTMovie can be drawn, and is intended to provide support for drawing the contents of a movie into a layer. Note that this class requires rendering using visual contexts.

For detailed descriptions of these classes and their associated methods, refer to QTMovieLayer Reference and QTCaptureLayer Reference.

Methods Not Supported Under 64-Bit QuickTime Kit

The following is a complete list of the methods that will not be supported under 64-bit QuickTime Kit:

In the QTMovie class:

+ (id)movieWithQuickTimeMovie:(Movie)movie disposeWhenDone:(BOOL)dispose error:(NSError **)errorPtr;

- (id)initWithQuickTimeMovie:(Movie)movie disposeWhenDone:(BOOL)dispose error:(NSError **)errorPtr;

- (Movie)quickTimeMovie;

- (MovieController)quickTimeMovieController;

In the QTTrack class:

+ (id)trackWithQuickTimeTrack:(Track)track error:(NSError **)errorPtr;

- (id)initWithQuickTimeTrack:(Track)track error:(NSError **)errorPtr;

- (Track)quickTimeTrack;

In the QTMedia class:

+ (id)mediaWithQuickTimeMedia:(Media)media error:(NSError **)errorPtr;

- (id)initWithQuickTimeMedia:(Media)media error:(NSError **)errorPtr;

- (Media)quickTimeMedia;

Note that there may be limitations to the current implementation. Some of these methods are subject to change and cannot be explicitly documented at this time. For developers, the essential takeaway is: The QuickTime Kit code in your application, minus the methods listed above, should just work.

How to Register and Unregister Movie and Track Inserts

To register for a movie audio context insert during playback, follow these steps:

1. Get the movie’s current audio context, using GetMovieAudioContext.

2. Register the application insert, using QTAudioContextRegisterInsert.

To unregister a movie audio context Insert, you

1. Call QTAudioContextRegisterInsert with a NULL QTAudioContextInsertRegistryInfoRef

2. If the registry ptr is non-null but the processDataCallback is null, this has the same effect.

To register for a track audio context Insert during playback, you

1. Set the kQTAudioPropertyID_RegisterAudioContextInsert property on the track, providing the same registry info structure that is used for the QTAudioContextRegisterInsert call.

2. To unregister a track audio context Insert, you set the kQTAudioPropertyID_RegisterAudioContextInsert property on the track, with a null processDataCallback call.

The code you need to set the property to register your insert is as follows:

//Begin an extraction session

err = MovieAudioExtractionBegin(...);

//Configure the session

//Set property to register your insert

err = MovieAudioExtractionSetProperty(

        extractionSessionRef,

        kQTPropertyClass_MovieAudioExtraction_Audio,

        kQTMovieAudioExtractionMoviePropertyID_RegisterMovieInsert,

        sizeof(QTAudioContextInsertRegistryInfo),

        regInfo);

//Pull for data

while (...) {

   err= MovieAudioExtractionFillBuffer(..., &flags);

}

//End the session

err = MovieAudioExtractionEnd(...);

New Audio Context Insert Callback Routines

For more detailed explanations of the audio context insert functions, refer to the discussion of each call in System/Library/Frameworks/QuickTime.framework/Headers/Movies.h.

The following callback routines are available in QuickTime 7.2.1:

• AudioContextInsertProcessDataCallback. A client-supplied function to be called during playback to get data from the audio insert.

  This routine is called by the QuickTime audio context for each buffer of audio data it renders. The client receives a source buffer list and a destination buffer list, and is responsible for supplying output buffers to the destination buffer list. This routine is generally called on the IOProc at high thread priority, and so should not perform memory allocation or release, acquire mutex resources, nor take very long to process.

• AudioContextInsertResetCallback. A client-supplied function to be called to initialize and reset for processing data.

  This routine is called by the QuickTime audio context to initialize for rendering. The client is told the sample rate and the maximum number of frames it will be asked to process on any single ProcessData callback (for example, inNumberFrames will always be equal to or less than inMaxFrames). On return, the client reports its processing latency and tail times. This callback is invoked whenever the rendering chain is interrupted (for example, when playback jumps to a new point or changes direction). The client should call AudioUnitReset on any audio units in use, and should be prepared to respond to changes of sample rate or maxframes.

• AudioContextInsertFinalizeCallback. A client-supplied function to be called to release any resources in use by the insert.

  This routine is called by the QuickTime audio context when the audio context is being disposed (that is, when the MovieAudioContext has been reset or the movie was disposed). Once this callback returns, no more calls for this registered insert will be made.

• QTAudioContextInsertRegistryInfo. Parameters for registering an audio context insert. Use this routine with QTAudioContextRegisterInsert and the movie audio extraction kQTMovieAudioExtractionAudioPropertyID_RegisterMovieInsert property.

Audio Context Insert Sample Code

The QTAudioContextInsert sample code project is available for download at QTAudioContextInsert.

The sample demonstrates how a client application of QuickTime can use the audio context Insert APIs to hook in a custom audio processing unit into the QuickTime audio processing chain.

The sample code seeks to establish good practices in using the new APIs. It illustrates steps involved in configuring and registering an insert with the movie or track whose audio data is to be tapped and/or processed. The code gives example implementations of the three callbacks––reset, process data, finalize––that need to be implemented by an insert’s processing logic. Finally, the sample code shows the steps involved in applying an insert to audio being extracted through the QuickTime movie audio extraction APIs.

The files in this sample code that are directly relevant to inserts are:

• ACInsertManager.mm. The attachDetachMovie{Track}LevelInsert routines demonstrate registering and unregistering movie and track level inserts during playback.

• ACInsertProcessor.mm. Shows examples of the ProcessData, Reset and Finalize callbacks that need to be implemented by the client app.

• QuickTimeAudioUtils.c. The prepareMovieForExtraction routine illustrates how movie and track-level inserts can be registered during extraction.