---

The RT Display Path

During normal operation, your effect is called by Final Cut’s RT (RealTime) engine. The RT engine is responsible for updating the video displayed in the canvas window and on any external monitor. The RT engine is engaged during playback and during “scrubbing.”

In this section:

Rendering characteristics
Example 1: DV Footage
Low, Medium, and Dynamic Quality in RT
Example 2: DV Footage at medium quality

Rendering characteristics

In general, the RT engine renders images at their native resolution. This is because the same rendered frame is sent to both the canvas window and the external monitor. Even if the canvas is displaying only one field, or even if the canvas is not displaying the entire image due to zooming or panning, both fields of the image are rendered at 100% resolution.

The RT engine renders each field of an interlaced image individually. This is required since keyframed parameters may have different values in different fields.

The RT engine renders fields in temporal order: the temporally first field is rendered first, the temporally second field is rendered second.

There is an exception to these rules: if the RT engine drops to a lower quality, it may render at a lower resolution, or only render a single field, or both. (This is discussed in-depth in a later section.)

Example 1: DV Footage

How does Final Cut call effects in a simple DV sequence? Here is an example.

DV NTSC footage has been added to a sequence created with the DV NTSC preset. Recall that the settings for DV NTSC are the following:

Now add an FxPlug filter effect to this item, and move the playhead to frame 5 of the sequence. The FxPlug filter is called to render twice: once for the first field of frame 5, and once for the second field of frame 5.

During the rendering process, the following information is available to the effect:

Note these general points:

• The input and output images are full-frame width, but half-frame height.

• The images’ aspect ratios have been divided by two.

• The temporally first field is rendered first.

When Final Cut renders the second field, the information is identical except that the frame number is been incremented by 0.5 because this is the temporally second field. The input and output images are now marked as the upper field of the frame.

Effects that pay attention to frame numbers should not be surprised to receive fractional frame numbers on interlaced sequences.

Low, Medium, and Dynamic Quality in RT

In some circumstances, sequences cannot play back in realtime on all machines. When this happens, the RT engine can adjust quality in several ways in order to maintain reasonable playback performance.

The RT menu in the timeline provides various options that let the user control RT performance. The “Playback Video Quality” section of the menu is the most interesting for FxPlug developers.

The setting for Playback Video Quality is passed as a hint to the codec when video is decompressed during playback. Depending on this setting, the codec may produce a smaller image at lower quality, or only decompress one field instead of an interlaced frame. The actual behavior is up to the codec.

In general, medium quality is half the resolution of high quality, and low quality (if the codec supports it) is one fourth the resolution of high quality.

By default, Playback Video Quality is set to Dynamic. This means that the RT engine, in order to maintain the frame rate, adjusts quality during playback. If the RT engine cannot maintain playback at high quality, it shifts to medium quality—and then may shift back to high quality subsequently if it detects that it can.

The key point is this: your effect should be ready for quality changes in the middle of playback. You should not assume that image dimensions and renderInfo.scale will be constant from one frame to another.

Also note that by default, the RT engine is set to “Scrub High Quality.” This means that rendering may drop to a lower quality during playback.

In “Safe RT” mode, the RT engine only renders effects during playback if it has profiled these effects and knows how fast they can render. Since the RT engine cannot profile third-party effects, including FxPlug effects, these effects are not rendered during playback in Safe RT mode. (Unprofiled effects are rendered during scrub.)

To display rendered FxPlug effects during playback, the Final Cut user must switch to the Unlimited RT setting. In this mode, the RT engine attempts to render everything as quickly as possible. (If the playback quality is set to Dynamic, the RT engine may degrade playback quality in order to maintain playback speed.)

Tip: To test your filter at lower quality settings, change the video playback quality to “Medium” or “Low” and turn “Scrub High Quality” off. In your render method, your plug-in should respond appropriately to changes in renderInfo.scale.

Example 2: DV Footage at medium quality

In the same DV sequence as Example 1, let’s switch the Playback Video Quality to Medium, turn off “Scrub High Quality,” and render frame 5 again. With Medium quality, the DV NTSC codec produces 360 x 240 progressive frames. By throwing away the temporally second field, and half of the pixels in the temporally first field, the codec greatly reduces the time needed to decompress a frame. This helps speed up playback.

The filter is only called to render once per frame. The right row of the table (MEDIUM QUALITY) displays the information passed to the filter. Note how this differs from rendering in high quality:

• The renderInfo’s scaleX and scaleY are set to 0.5.

• The input and output dimensions are half of the full resolution. (This matches the renderInfo.scale values.)

• The input and output images are set to kFxField_NONE.

• The output image is set to kFxFieldOrder_PROGRESSIVE.

The DV codec returns the same result for low quality as for medium quality. For an example a situation where low quality is different than medium quality, try one of the 1080i settings.

---