Getting Your Video Ready for Compression for Playback on iPhone, iPod, and the Web

The secret of good compression is in the preparation. You want your content to look good on wireless devices like iPhone and iPod, as well as on the Web. Since the bandwidth available on these platforms is limited, you must use all of your bit rate efficiently. The high-performance H.264 video codec available in QuickTime is designed for these situations. Still, the more you can do to make your footage compress effectively before postproduction, the better. If you are a content creator, film maker or producer who works with compressed video, read on for tips, techniques and solutions that will make your content stand out.

Keep Changes to a Minimum

Before we talk about making your video compression friendly, it's important to understand why video needs to be compressed in the first place. As you know, each frame of digital video consists of a grid of pixels which represent the light and color in the image. For example, a standard definition NTSC signal is 720 pixels wide and 480 pixels tall. By multiplying the two dimensions, we see that there are 345,600 pixels in the image. 1080p HD video is 1920 pixels wide by 1080 pixels tall, which means each frame takes over two million pixels to represent. Each one of these pixels is a separate data point describing the picture we see.

Figure 1: Even in a small 320x240 image like the one above, there are over 76,000 pixels.

The problem is change. Each frame of video is made up of hundreds of thousands of pixels, which can change at a rate of 30 times per second or more. That's a lot of information. If we try to describe it all, our video files get so big that they’re practically unusable. But if we throw away the wrong information, the picture deteriorates.

Over the years, compression engineers have invented two very useful tactics for efficiently representing all these pixels without sacrificing quality: motion estimation and frame differencing. In a nutshell, motion estimation reduces the amount of change that must be represented for moving objects, while frame differencing enables us to ignore pixels that don’t change from frame to frame.

As you read the suggestions below, keep in mind one basic goal: reducing the amount of frame-to-frame change that has to be represented in your video.

Shooting and Editing for Compression

Before you begin a video project, think about your target platform. The less bandwidth you’ll have, the more resilient to compression your video will need to be. If you aren’t shooting the video yourself, work together with the production staff from the earliest stages of planning and storyboarding. Shooting with compression in mind will save effort later.

Keep it Simple

When possible, simplify the background. Choosing a stationary background instead of a moving one allows the H.264 compressor to discard more repetitive data from the image. Simple backgrounds work especially well for “talking head” shots, such as in a podcast or interview. Pay attention to your camera work, too. Keep pans, tilts, and other camera movements to a minimum to keep backgrounds stationary. Use a tripod to minimize movement from camera shake.

Even if you can’t simplify the background very much, try recomposing the shot a little. Crop out those rustling leaves and busy streets and your bit rate will thank you. Also, be careful about motion graphics. Moving graphics require more bits to compress than stationary ones. Even wardrobe matters: intricate patterns are harder to compress than solids. Don’t sacrifice your artistic vision to simplification, but try to strike a balance.

For example, the two movies below have the same resolution (480x270) and bit rate (150 kbit/sec). In both, we see a beach scene. The difference is the background: the first shows us rolling waves, while the second has stationary dunes. Watch the sky and surf for blocky artifacts and color banding.

Sample Movie 1: A high-motion background causes compression artifacts.

Sample Movie 2: A stationary background gives us cleaner results.

Notice how much cleaner the second movie looks. The first is full of blocky compression artifacts, which are caused by the compressor running out of bit rate to work with. Less motion in the background means less change from frame to frame, and better compression.

Make Transitions Forgiving

Transitions between scenes, such as fades, dissolves, and other digital effects, can be a major trouble spot for compression. Fades are especially hard to compress because they gradually change each pixel frame-by-frame.

When it fits your artistic goals, try a less complex transition. A wipe will be more efficient than a fade or dissolve because it will leave most of the image alone from frame to frame. In extremely low-bandwidth situations, you can further optimize compression by using a horizontal wipe. Horizontal wipes are slightly more efficient than diagonal wipes because they involve a smaller number of macroblocks in the frame.

Figure 2: A diagonal wipe involves more macroblocks simultaneously than a horizontal wipe.

Get the Sizing Right

As you compose your shots, keep in mind the resolution and aspect ratio of the format you’ll be using for delivery. For instance, video shot in 1080i HD (1920x1080, with a widescreen 16:9 aspect ratio) will need to be scaled down and likely cropped for platforms with different requirements, like iPhone.

The image below shows an image shot with a 4:3 aspect ratio (standard definition TV) with a 16:9 screen inset. If you want to crop what’s outside the red box later, keep important elements inside it when you shoot.

Figure 3: 4:3 image cropped to 16:9. Important elements are left intact.

Digitizing Video

Read Digitizing Video to learn how to clean up digitized video for compression.

Deinterlace, Deinterlace, Deinterlace

Today, progressive scan video is the standard for HD video. Back in the days of vacuum tubes, though, it wasn’t possible for a television’s cathode-ray tube to display an entire video frame at once without flickering. To get around this problem, engineers invented interlaced video.

Figure 4: The Interlaced Raster

Where progressive video shows the entire picture at once, interlaced video frames are divided into two “fields”, odd and even, each of which contains only half of the image. Interlaced fields are then displayed at twice the rate of progressive frames.

Progressive scan video is the standard for computer monitors, as well as devices like iPod and iPhone, and most HDTVs (those with 720p or 1080p resolutions). Still, many other devices record and display video in an interlaced format. These include standard definition television sets and many consumer camcorders. It is very important to know what format your source recording is in.

The problem with interlaced video is that it just doesn’t compress very well. Although progress has been made since the early days, H.264 can still work much more efficiently in predicting motion when it occurs in a progressive frame than when it must predict changes between two alternating fields. Since there is no advantage to using interlaced video on the web, you get nothing from the increased bit rate except for a higher bandwidth bill.

If you can, record and capture video in a progressive format. If you must work with interlaced video, deinterlace your content before compressing it. The deinterlacing process creates a progressive frame from interlaced fields. The resulting image will be somewhat softer than the original.

How to Deinterlace

You have a few options for deinterlacing. In Final Cut Studio, Compressor offers the highest quality deinterlacing available: motion compensation. Use this setting for high-resolution projects where render time is not an issue. Try the Motion Adaptive setting for quicker renders without sacrificing much quality. Final Cut Pro also has a deinterlace filter, which is best for fast renders at low resolutions.

For instructions on deinterlacing in Final Cut Pro, see http://docs.info.apple.com/article.html?artnum=58634. For instructions on deinterlacing in Compressor, see http://manuals.info.apple.com/en/Compressor_2.0_LBN_z.pdf (p.9).

Benefits of Deinterlacing

Deinterlacing doesn’t just reduce bit rate. It also helps to reduce annoying motion artifacts and twittering graphics. Let's take a look at these effects and how to recognize them in your video.

Reduce Motion Artifacts

Watch out for rapid motion in an interlaced video. You may see jagged edges or blurry areas on moving objects. This distracting effect happens when one field is updated before the other, and is especially noticeable in freeze-framing. Deinterlacing minimizes this effect.

Figure 5: Video frame before and after deinterlacing.

Note the butterfly's blurry wings in the left still, and the clearer deinterlaced still on the right. Although some blur still exists, this arises from the camera's frame rate and not from interlacing. If the motion had been horizontal rather than vertical, we would see a jagged edge on the sides of the wing instead.

Twittering Graphics and Text

Have you ever noticed your text or graphics bouncing or flickering when you hit “play”, even though they look fine on pause? This “twittering” occurs when interlaced video contains thin horizontal lines. If these lines are only one pixel high, the field containing them will alternate off and on, creating a flicker. If the lines are two pixels high, they will seem to bounce up and down. You’ll see this problem with patterned shirts, too.

Sample Movie 3: Narrow line flickering from interlacing

It’s not hard to fix twitter, but the solution depends on whether you're creating all layers of the content on the spot or using ingested video. If there are no interlaced video tracks in your sequence, set Field Dominance setting to "None." This will prevent the problem from occurring in the first place. In the movies below, notice the huge improvement in quality given by changing the Field Dominance:

Sample Movie 4: Interlaced rolling text.

Sample Movie 5: Deinterlaced rolling text.

Don't miss the improvement made to bit rate here, either - a 2 MB movie has shrunk to 1.2 MB, just by deinterlacing! Even if your video doesn't look as bad as these, it's worth checking the field dominance to save bandwidth. In Final Cut, just select your sequence, then choose Settings from the Sequence menu. Change Field Dominance to "None", as shown in Figure 6 below:

Figure 6: Field Dominance Setting for Final Cut Sequence

Setting the field dominance to "None" will cause problems if you are mixing interlaced and progressive content. In that case, antialias text as you create it. Either way, avoid narrow fonts, fonts smaller than 14 points, and those with small serifs. If necessary, use the anti-alias and flicker filters in Final Cut. Deinterlace the entire composite sequence when you output the video. (And start wearing solids instead of stripes!)

Converting Film to Video

Read Converting Film to Video for tips on improving the compression of film that has been converted to video.

Reducing Visual Complexity After Ingest

After you ingest your video or film, edit your footage with compression in mind. Try to reach a balance between aesthetics and compressibility by optimizing visual complexity.

Complexity is a combination of fine detail and rapid changes from frame to frame. The more complex the image, the harder it is to compress. Good editing choices can help to reduce complexity and improve encoding. Two things to look out for are:

Objects Moving in a Line

Try to limit the number of shots with fast-moving objects. It takes more bit rate to compress shots of objects moving rapidly from one end of the frame to the other. Also, keep in mind that objects moving simultaneously from background to foreground and across the screen create a zooming effect and require an even higher bit rate to encode. Cut them out when you can.

Cyclical Motion

Do you have a shot with motion that follows a repeating pattern? Hold it! Unlike objects moving in a line, repetitive motion such as a waterfall or pendulum can compress well. The H.264 codec will work best if the shot is held for at least a few seconds because longer shots give the encoder more redundancy to compress.

Cropping, Scaling, and Aspect Ratio

Although it seems simple, sizing can be very tricky. There are several mistakes you can make that will really hurt the quality of your video. First of all, make sure it has the proper aspect ratio. If you don’t check this, the video may be stretched or squeezed to fit the device playing it.

The aspect ratio of a video is its width divided by its height. For example, a video with 640x480 video has an aspect ratio of 1.33:1, commonly called 4:3 (that is, four units of width for every three units of height.) Widescreen HDTV uses a 1.78:1 aspect ratio, usually known as 16:9. Most theatrical releases use even wider formats. For recommended settings for Apple devices, see Tech Note 2218: Compressing QuickTime Videos for the Web.

Use a reference movie to target multiple platforms with multiple aspect ratios at once.

Scale or Crop?

Once you've determined the proper aspect ratio for your content, start making adjustments to its size. Before deciding whether to crop or scale the video (or both), you need to know the relationship between its current aspect ratio and the destination aspect ratio.

If the aspect ratios are the same, scale down. For example, if you plan to reduce a 640x480 video to 320x240 for delivery on the web, scale the video down to 50% resolution. Scaling at a 2:1 or 4:1 ratio will lead to cleaner results than scaling at an odd ratio, like 3:1. Avoid scaling video up to larger resolutions than the original.

If your destination size does not divide evenly into the source’s size, consider cropping the video after scaling. Cropping avoids removes the edges of an image to avoid stretching or squeezing it. If you have framed your shots so that they can be cropped without losing significant elements, it may be a good option.

Figure 7: Crop 16:9 footage on the sides to convert to 4:3, and 4:3 on the top and bottom for 16:9.

The easiest way to crop in Final Cut is to use the Crop Tool in the Canvas. First, make sure that you have turned on Image+Wireframe for the Canvas. Then, select the Crop Tool and drag the sides of the Canvas to crop.

 

Figure 8: Turn Image+Wireframe on in the Canvas.

Figure 9: Use the Crop Tool to drag the borders of the Canvas.

Cleaning Up the Image

Once you’ve gotten your video to its target resolution, make adjustments to reduce compression artifacts and bring out detail. If you are targeting multiple platforms with multiple resolutions, adjust each individually after cropping and scaling.

Sharpening

Avoid adjusting sharpness until after your video is at its target size - you might accidentally undo sharpening when scaling the image!

Keep a few key points in mind:

• Sharpening in the luminance channel alone will avoid blotchy spots of color.
• Footage from a telecine unit may already have been over-sharpened by the technician. Tread carefully.
• Applying too much sharpening will reintroduce noise you cleaned earlier, and cause edge problems in high-contrast areas.

Figure 10: Avoid oversharpening images.

The noise caused by over-sharpening will lead to a higher bit rate. So, when in doubt, err on the side of not sharpening the image.

Text

Text is hard to compress. Due to a mathematical peculiarity known as the Gibbs phenomenon, artifacts tend to appear around sharp objects in compressed video, particularly at lower bit rates. This effect is nearly impossible to fix after the fact. You can try giving text-heavy movies higher bit rates to minimize this problem if you have bit rate to spare.

Figure 11: Text with severe Gibbs artifacts.

If increasing the bit rate does not reduce artifacting, consider cropping and rekeying the text and converting it to a metadata text track. This process is as simple as creating a text file with a little bit of markup. You can then render it in the player as a caption overlay, saving considerable bit rate in the process. Text tracks have the additional advantage of being vector based rather than bitmap, which means that they will scale perfectly on the fly to any resolution the viewer may choose.

Temporal Noise

You’ll see this problem more frequently at lower bit rates. Differences appear between areas that should be identical from frame to frame:

Figure 12: Temporal noise creates differences where none should exist.

If you notice temporal noise in your video, increase the bit rate or use Compressor to apply a noise filter during export, as seen below. The Noise Removal filter will blur areas of the image that have low contrast, while leaving high-contrast areas sharp. This technique, known as adaptive noise reduction, will help to reduce temporal noise (and your bit rate.) The effect is designed to be imperceptible to the viewer. Generally, apply the effect to All Channels. Try adjusting the Iterations and Algorithm settings until you get an acceptable result, or see p. 248 of the Compressor 3 User Manual for more information.

Figure 13: Noise Removal filter in Compressor.

Buffer Overflow

If you set your bit rate too low for the complexity or resolution of a video, it can lead to bursts of noise in the image. These typically last only a few frames. If you experience bursts of artifacts, take note of where in the video they are occurring. If they happen during a transition, try a different transition to reduce the spike in bit rate. Increasing the bit rate may also help. If you have reached the maximum bit rate permissible for your delivery method, you must reduce visual complexity before encoding or deliver at a smaller resolution.

Audio Sweetening

Don’t ignore the sound! High quality audio is every bit as important as video. Equally important, noisy audio takes more bits to compress. A few adjustments will save you bit rate and keep your audience form being distracted by sound distortion and noise.

To begin sweetening your audio in Final Cut, follow these steps in order:

1. Use the Hum Remover filter to cancel any hums. These frequently arise from ungrounded cables or other electrical equipment. Try filtering at 60 Hz (or 50 Hz outside North American and Japan) and adjust the Q until you get the desired result. Try the 1st - 5th harmonic checkboxes if you also hear hums on higher frequencies.
2. Use a high pass filter to cut out low frequency rumbles.
3. Eliminate hiss with a low pass filter.
4. Use a noise gate to remove background noise during silent passages.
5. Equalize, but in moderation. Don’t undo noise reduction by boosting frequencies you’ve just filtered.
6. Find the peak audio level. Set the overall output 3dB lower to avoid distortion.

Final Thoughts

Following this set of guidelines will help you to plan and edit your video so that it compresses better in H.264 for iPod, iPhone, Apple TV and the Web. By addressing problems as early as possible, you avoid magnifying them later in post production. Remember that these are only guidelines – experiment with different settings and techniques to find the best preprocessing workflow for your content.

For More Information

• Timebase Correction (Wikipedia)
• Frame Synchronizer (Wikipedia)
• Gibbs Phenomenon (Wikipedia)
• Cliff Wootton, A Practical Guide to Video and Audio Compression. Elsevier, 2005.

Posted: 2008-09-16