Module 2- Playing a Movie

Overview

In the first module, we learned how to create a window with a QTCanvas and draw an image in the canvas. This module expands upon that example and adds a movie that plays in the upper left hand corner of the window (see image, right).

Also in this module, we will learn about movie-related classes such as MoviePlayers and Compositors. We will also learn how to control the playback rate of the movie, start and stop the movie, and present controls to the user. At the end of this module, you will understand how QuickTime handles time-based media and understand how to present a movie in a window.

Introducing Time-based Media in QuickTime for Java

When most people think of QuickTime, they think of movie playback. Although QuickTime contains a lot more than just facilities for playing movies, QuickTime for Java would certainly be incomplete if it did not provide rich support for time-based multimedia.

QuickTime allows you to manipulate time-based data such as video and audio sequences. The term "movie" is used to describe any data that is represented in a time-based fashion. The data for this media is stored in objects called "movies".

Just like its analog sister, the motion picture, a single QuickTime movie can contain multiple tracks (or streams) of data. The track does not contain the data directly. It instead refers to a media structure that may contain a reference to the actual media data. The data could reside in a number of places; locally on the hard disk, on a CD-Rom drive, on a network volume, or remote on an HTTP or ftp server. It may even be streamed from an RTSP server. The movie can have a mixture of references to media in different locations. The soundtrack could be stored on a CD, while one video track could be local, and a second video track streamed from a server live.

This flexibility allows your program to take advantage of a large number of media formats and internet protocols without having to worry about the details of keeping the data synchronized. QuickTime can handle the communication, presentation, and playback of the media for you.

We will discuss these concepts in more detail as we study the code sections below.

Back to top

Setting up the project

This project is based on the previous module and assumes that you have completed it. If you would like to download a completed copy of the module 1, you may do so. Before proceeding with these steps, we have renamed the main class to "Zoo2" and changed all instances of this class to prevent confusion.

In our previous module, the entire project consisted of a single source file. In this module, we will split the window functionality off into an additional class file (AnimalPane). This will allow us to modify that single class without having to change the file that contains main. Since we will be working with two separate source files, I have added a title (in blue) to each source snippet so that you can be aware of the source file that the excerpt comes from.

Let's start by creating a new file and saving it as "AnimalPane.java" in your project folder.

public class AnimalPane 
{ 
    public AnimalPane() 
    { 
    }
}

This new file will contain all of our code for media loading, creation of the QTCanvas, and display. Once we have declared our class and constructor, we will need to go to our main file to preform some source modifications.

Back to top

Modifying Zoo2.java

Now that we have a new source file that we will use for all of our media loading and display, we need to move the code (highlighted below) from main into AnimalPane.java. Open Zoo2.java and go to the constructor:

    public Zoo2( String s ) 
    {
        super(s);

         setResizable( false );              
        setBounds( 0, 0, WIDTH, HEIGHT ); 

        QTCanvas myQTCanvas = new QTCanvas( 
                 QTCanvas.kInitialSize, 0.5F, 0.5F );
        add( myQTCanvas );

        try
        {
            QTFile imageFile = new QTFile( 
                QTFactory.findAbsolutePath( 
                "data/zebra/ZebraBackground.jpg" ));
            GraphicsImporterDrawer mapDrawer = 
                new GraphicsImporterDrawer( imageFile );
            myQTCanvas.setClient( mapDrawer, true );

        }
        catch ( IOException e )
        {
             e.printStackTrace();
        }
        catch ( QTException e )
        {
             e.printStackTrace();
        }  

        addWindowListener( new WindowAdapter() 		
        {
            public void windowClosing( WindowEvent we )
            {
                QTSession.close();		
                dispose();					
            }
            public void windowClosed( WindowEvent we )
            {
                System.exit( 0 );		
            }
        });
    }

Select the highlighted code (above) and copy it to the clipboard. We will move this code into AnimalPane.java.

Replace the copied code with the following:

    public Zoo2( String s ) 
    {
        super(s);

         setResizable( false );              
        setBounds( 0, 0, WIDTH, HEIGHT ); 

        QTCanvas myQTCanvas = new QTCanvas( 
                 QTCanvas.kInitialSize, 0.5F, 0.5F );
        add( myQTCanvas );

        AnimalPane zebraPane = new AnimalPane();

        addWindowListener( new WindowAdapter() 		
        {
            public void windowClosing( WindowEvent we )
            {
                QTSession.close();		
                dispose();					
            }
            public void windowClosed( WindowEvent we )
            {
                System.exit( 0 );		
            }
        });
    }

Back to top

Adding Media Loading to AnimalPane

Now go back to AnimalPane.java and paste the copied source:

public class AnimalPane 
{ 
    public AnimalPane() 
    { 
 

        try
        {
            QTFile imageFile = new QTFile( 
                QTFactory.findAbsolutePath( 
                "data/zebra/ZebraBackground.jpg" ));
            GraphicsImporterDrawer mapDrawer = 
                new GraphicsImporterDrawer( imageFile );
            myQTCanvas.setClient( mapDrawer, true );

        }
        catch ( IOException e )
        {
             e.printStackTrace();
        }
        catch ( QTException e )
        {
             e.printStackTrace();
        }  

    } 
}

Your source should now look like the above. Keep in mind that this code is not yet in a working state. For example, myQTCanvas does not exist in this source file- it is a local data member of main. We will take care of these issues in the next couple of steps.

Using Compositors in QTJ

There are many different ways of displaying time-based media (movies) in QTJ. There are a series of classes such as the MoviePlayer class that are capable of becoming clients of the QTCanvas and displaying the movie in the canvas. In our situation, we have a movie and an image. Both cannot simultaneously be clients of the canvas. We need an object that can manage both media objects, the movie and the image, at the same time.

We use a Compositor to perform this service. The Compositor is a class that provides the capability to composite complex images from disparate image sources and then treat the result as a single image which is presented to the user. It uses the QuickTime SpriteWorld internally to perform the compositing tasks. The member objects can be time-based or single frame objects.

Since QuickTime manages the presentation of all of the objects added to the compositor, we do not need to worry about the bookkeeping involved in updating and drawing the objects in the Canvas. As we will shortly see, the Compositor provides an easy mechanism for managing multiple types of media in a single visible space.

Back to top

Adding the Compositor

The first step in the process of creating a compositor is to make a new QDGraphics object:

public class AnimalPane 
{ 
    public AnimalPane() 
    { 

        QDRect size = new QDRect(Zoo2.WIDTH, Zoo2.HEIGHT);

        try
        {

            QDGraphics gw = new QDGraphics( size );

            QTFile imageFile = new QTFile( 
                QTFactory.findAbsolutePath( 
                "data/zebra/ZebraBackground.jpg" ));
        ...

A QDGraphics object is a complete drawing environment that specifies how color drawing operations occur. It includes information such as background and foreground drawing color, pen width, and the characteristics of the drawing port. We will need a QDGraphics object to pass to the constructor of our Compositor.

First we define a rectangle that specifies the size of the QDGraphics port we will be using. This rectangle is based on the height and width variables that we defined in Zoo to use when creating the QTCanvas. These variables come in handy here as well.

Once we have defined the size of our graphics port, we create a new QDGraphics object and pass it the size rectangle.

Now we can actually create our compositor:

public class AnimalPane 
{ 
    public AnimalPane() 
    { 
        QDRect size = new QDRect(Zoo2.WIDTH, Zoo2.HEIGHT);
        try
        {
            QDGraphics gw = new QDGraphics( size );

            compositor = new Compositor( gw, QDColor.white,
                                         30, 1 );

            QTFile imageFile = new QTFile( 
                QTFactory.findAbsolutePath( 
                "data/zebra/ZebraBackground.jpg" ));     

            ...
    }

    protected Compositor compositor;

}

We create a local Compositor variable at the bottom of the file, protected so that other classes cannot access the variable directly. Then we create our compositor directly underneath the line where we created the QDGraphics object.

The compositor constructor takes several parameters. The first is our QDGraphics object which defines the port that the compositor will use for drawing operations. The second variable is a color that specifies the background color to be used by the compositor. Since we want our background to be white, we use the constant QDColor.white as the parameter.

The final two parameters control the time base. The first number is the scale while the second number is the period. By specifying a scale of 30, we are specifying that our compositor runs at a rate of 30 times per second. The period is the amount of time that can elapse between invocations of associated actions. When we specify a scale, we are specifying the maximum rate that objects within the compositor can run at. Even if a media object displayed within the compositor is running at 60 frames per second, our compositor updates at 30 frames per second.

Back to top

Adding media to the Compositor

Once the compositor is created, media must be added to the compositor before it will display (similar to the way we added our image media as the client of the QTCanvas).

public class AnimalPane 
{ 
    public AnimalPane() 
    { 
        QDRect size = new QDRect(Zoo2.WIDTH, Zoo2.HEIGHT);
        try
        {
            QDGraphics gw = new QDGraphics( size );
            compositor = new Compositor( gw, QDColor.white,
                                         30, 1 );

            QTFile imageFile = new QTFile( 
                QTFactory.findAbsolutePath( 
                "data/zebra/ZebraBackground.jpg" ));
            GraphicsImporterDrawer mapDrawer = 
                new GraphicsImporterDrawer( imageFile );

            ImagePresenter presenter = 
                ImagePresenter.fromGraphicsImporterDrawer(drawer);
            presenter.setLocation( 110, 110 );	
            compositor.addMember( presenter, 2 );

        }
        catch ( IOException e )
        {
             e.printStackTrace();
        }
        
        ...

Compositors require member objects that know how to render themselves. Having a GraphicsImporterDrawer is not sufficient. We need a drawable object which we extract from the GraphicsImporterDrawer by declaring an image presenter object and assigning it to the result of the fromGraphicsImporterDrawer( ) constructor of the ImagePresenter class.

Next, we position the ImagePresenter within the Compositor object.

When dealing with a compositor, it is important to remember that any items displayed within it are drawn in local coordinate space.

To help illustrate the discrete layering order of objects within our window, please reference the diagram to the right.

In our frame, we have created a QTCanvas. This canvas is created to be the same size as the Frame it is contained in. We have also specified that items within the QT Canvas are to be centered. This affects the Compositor object, since it is contained directly in the QTCanvas. Although our Compositor is the same size as the canvas, we could choose to have a smaller Compositor which would appear centered in the QTCanvas.

To position the image within the compositor, we call the setLocation( ) method of the presenter. In this case, we inset the image 110 pixels from the top left corner of the Compositor.

Finally, we add the image to the compositor by calling the addMember( ) method from the compositor. We pass two parameters. The first is the object that we are adding ( the ImagePresenter ), and the second is the layer number that the object will be placed on within the compositor.

One of the key features of the compositor class is its ability to organize and display contained media objects in any number of discrete layers. Objects on layer 1 draw "on top" of objects in successively numbered layers. Objects in top most layers obscure objects that occupy the same coordinate space of objects in lower layers.

For example, the diagram at the right illustrates the layering of items in the compositor with the addition of a Movie object.

The ImagePresenter object is added to the Compositor in layer 2. If we add a Movie object in layer 1, this object "floats" above the media in lower layers.

If the movie and the image occupy the same space in the compositor, the movie will draw on top of and thus obscure part of the image.

Back to top

Loading and playing a movie

Now that we have loaded our image and added it to the compositor, it is time to load and play our movie.

public class AnimalPane 
{ 
    public AnimalPane() 
    { 
        QDRect size = new QDRect(Zoo2.WIDTH, Zoo2.HEIGHT);
        try
        {
            ...

           QTFile imageFile = new QTFile( 
                QTFactory.findAbsolutePath( 
                "data/zebra/ZebraBackground.jpg" ));
           GraphicsImporterDrawer mapDrawer = 
                new GraphicsImporterDrawer( imageFile );            
           ImagePresenter presenter = 
                ImagePresenter.fromGraphicsImporterDrawer(drawer);
            presenter.setLocation( 110, 110 );	
            compositor.addMember( presenter, 2 );

            
            Movie m = makeMovie( new QTFile( 
                QTFactory.findAbsolutePath( 
                "data/zebra/Zebra.mov" )));
            md = new MoviePresenter( m );			
            compositor.addMember( md, 1 );			
            compositor.getTimer().setRate(1);	
            md.setRate(1);

        }
        catch ( IOException e )
        {
             e.printStackTrace();
        }
        
        ...

    }

    protected Compositor compositor;

    protected MoviePresenter md;

In order to display movies in our compositor, we need an object that knows how to display a QuickTime movie. The MoviePresenter class is very similar to the ImagePresenter class we used earlier (and is in fact derived from ImagePresenter). The primary difference is that the MoviePresenter is capable of displaying temporal media formats.

So our first step is to go down to the bottom our AnimalPane file and declare a protected MoviePresenter object. This is the object we will be adding to our compositor so that we can display our movie on the screen.

Now we can go back up to our constructor and load our movie right under the code we previously wrote for adding our ImagePresenter to the Compositor. Looking at our image code, we can see that we first created a QTFile and then a GraphicsImporterDrawer with the QTFile. Lastly, we created a ImagePresenter from the GraphicsImporterDrawer.

We can use a very similar approach for loading a movie. We need to create a QTFile, so we need to do something like this:

QTFile theMOV = new QTFile( QTFactory.findAbsolutePath( "data/zebra/Zebra.mov" ));

But, there is no class corresponding to the GraphicsImporterDrawer for dealing with movies, so we will write our own. We want a method that we can pass a file to that will return an object that we can associate with a MoviePresenter. Since MoviePresenters can be constructed with a Movie object, we will use that as the return type for our method.

Our function prototype will look something like this:

public Movie makeMovie( QTFile theFile );

Don't worry about this code yet. We will write that in our next step. For now, we only need to concentrate on using this method correctly:

Movie m = makeMovie( theMOV );

or more succinctly:

Movie m = makeMovie( new QTFile( QTFactory.findAbsolutePath( "data/zebra/Zebra.mov" )));

Once we have a movie, we create a MoviePresenter object:

md = new MoviePresenter( m );

And add it to the compositor:

compositor.addMember( md, 1 );

We choose to put it in the first layer so that it draws on top of our background image. Lastly, we set the rate of the compositor and the rate of the MoviePresenter to 1:

compositor.getTimer().setRate(1);
md.setRate(1);

The rate is a floating point value that specifies the playback timing scale. If the rate is 0, the movie (or time base of the compositor) is stopped. The value 1.0, specifies 100% speed in a forward direction while 2.0 would specify double speed. The image below illustrates how this scale works:

In this diagram, rate values appear at the top of the scale. Percentage values appears underneath the scale. This percentage indicates the percentage of the native movie speed that the movie will play back at the given rate. Negative numbers indicate that the movie is played backwards and positive numbers indicate that the movie is playing forwards. Thus a rate value of -1.2 indicates that the movie is playing backwards at 120% of its frame rate. If a movie is 15 frames per second, and has a rate of 2, the compositor will play 30 frames per second (if it is capable) and will thus play the movie twice as fast as it would play back at a rate of 1.

Note that both the movie player and the compositor have separate rates. The compositor's rate will affect the rates of any time-based media objects that it contains.

To understand how this works, let us consider the following scenario:

Let's say that we have a compositor that has been constructed with a frame rate of 30. We add three movies to it. The first has a rate of 1 and is 30 frames per second (see chart, left). The second has a rate of 2 and is 20 frames per second. The third movie has a rate of -1.5 and is also 20 frames per second.

If the compositor has a rate of one, all of the movies will play back at their own individual rates. Ie, the first movie would play back normally at 30 frames per second while the second would play back double speed. It would play back at 30 frames per second (because that is the maximum rate that the compositor could play at) and every fourth frame would be skipped (in order to maintain the double-speed rate). The third movie would play back one an half times speed backwards at thirty frames per second. No frames would be skipped because the compositor is capable of playing all frames at its maximum frame rate.

Now if we were to change the compositor rate to -.6, this will affect the playback of all movies it manages. Thus, the first movie would play back in the compositor at a rate of -.6 (60% speed backwards) and would have a final frame rate of 20 frames per second. The second movie would play back in the compositor at a rate of -1.2 (120% speed backwards) and would have a final frame rate of 24. The third movie will play in the compositor at a rate of .9 (90% speed forwards) at a rate of 18 frames per second. The compositor's negative rate reversed the backwards playback direction so that the movie is now playing forwards. Note that even though the rate of the compositor is -0.6, it still maintains a maximum frame rate of 30 frames per second although the media it controls have reduced frame rates because of the compositor rate change.

Back to top

Making the Movie

Now it is time to implement the makeMovie routine that we used in the previous step:

public class AnimalPane 
{ 
    public AnimalPane() 
    { 
        ...
    }

    protected Movie makeMovie( QTFile f ) 
              throws IOException, QTException
    {
        OpenMovieFile movieFile = OpenMovieFile.asRead(f);
        Movie m = Movie.fromFile( movieFile );
        m.getTimeBase().setFlags( StdQTConstants.loopTimeBase );	
        return m;	
    }

   
    protected Compositor compositor;
    protected MoviePresenter md;
}

We create our method immediately after our animal pane constructor. We declare that our method may throw two common exceptions that QuickTime for Java routines typically throw. This alerts callers of our method that they need to properly handle any exception thrown by our routine.

The first step, is to declare an OpenMovieFile object. This object represents a QuickTime movie that can be opened for reading and writing. In this case, we need to use it to open a movie on disk and read it into a Movie object in memory.

We declare the OpenMovieFile and call the asRead( ) method with our QTFile as a parameter. Then we create a Movie object by calling the static fromFile( ) method with our OpenMovieFile object as a parameter. This creates a movie in memory from the resource that we read into a file.

Lastly, we need to make the movie loop so that when it gets to the end of the movie, it starts playing again from the beginning. This is accomplished by getting the time base of the movie (by calling getTimeBase ) and setting the movie flag to loopTimeBase from StdQTConstants. If we did not want the movie to loop, we would not need this step.

When we are done with the movie, we return it from the function. As you can see this method is very short. Although we could have choosen to inline this code right in the constructor, we decided to make a separate method because it makes the code a bit cleaner and will allow us greater flexibility later. The great thing about this code, is that it works for any time-based media format supported by quicktime. It doesn't even have to have video. It could be an aiff audio stream, an AVI movie, a MOV movie, an MPEG video, MPEG audio, etc. It QuickTime knows how to handle the format, our code will handle it!

Finishing the project

Now there's just a few remaining loose ends to tie up before we can try out our program. If we were to run it right now, our window would come up with nothing in it at all. That is because our compositor is not yet associated with the QTCanvas, so it has nowhere to draw. As you may recall from the previous module, the QTCanvas has to have a client class that is responsible for drawing. In our case, we want to make our compositor the client of the QTCanvas. But there is one small problem. The QTCanvas is in our Zoo2 class and the Compositor is a member of AnimalPane. We could access the compositor from the AnimalPane, but it is protected.

Why not just make it public? Well, it is protected for a reason. Good object-oriented design teaches us that we don't want to have classes directly messing with our internals. Therefore, we will provide an API that allows other classes to gain access to our compositor:

public class AnimalPane 
{ 
    public AnimalPane() 
    { 
        ...
    }

    public Compositor getCompositor( ) { return compositor; }

    protected Movie makeMovie( QTFile f ) 
              throws IOException, QTException
    {
        ...
	
    }
   
    protected Compositor compositor;
    protected MoviePresenter md;
}

We added a method getCompositor( ) that simply returns our protected compositor data member. This may seem like a brain-dead thing to do, but consider a case where our AnimalPane was managing several compositor objects. Then we could modify getCompositor to return the active compositor. This makes much more sense than making all of your member variables public and having external classes trying to figure out which compositor is the correct one to use.

Now we need to modify Zoo2.java to set our compositor as the client of the QTCanvas:

    public Zoo2( String s ) 
    {
        super(s);

        setResizable( false );              
        setBounds( 0, 0, WIDTH, HEIGHT ); 

        QTCanvas myQTCanvas = new QTCanvas( 
                 QTCanvas.kInitialSize, 0.5F, 0.5F );
        add( myQTCanvas );

        AnimalPane zebraPane = new AnimalPane();

         try
        {
            myQTCanvas.setClient( zebraPane.getCompositor(), true );
        }
        catch ( QTException e )
        {
            e.printStackTrace();
        }

        addWindowListener( new WindowAdapter()
        {	
        
            ...

And that's all there is to it! We are finished. If you would like to see the entire source file of either Zoo2.java or AnimalPane.java, feel free to follow the links.

Back to top

Summary

This module introduced some topics such as how to deal with time-based media. We learned some fundamental concepts including how to open and read a QuickTime movie into memory. We even wrote code that works regardless of the media format.

We learned how to create compositors and assign media to the compositor in layers, how to control the movie playback rate, and how the rate relates to the compositor. Finally, we learned how to associate a compositor with our QTCanvas so that the compositor can draw its contents to the screen.

We covered a lot of material, but this is really just the tip of the iceberg. QuickTime for Java is a very full-featured and powerful API, that integrates with Java in an elegant fashion. In our next module, we will learn how to use QuickTime to render text in our compositor.

Back to top

Further Exploration

The clever reader will realize quickly that the information that was presented here only scratches the surface of what could be done using QuickTime for Java. Although we will expand this example in future modules, there are few things that you could do on your own as exploration that will help you better understand how QuickTime for Java works.

Each one of these examples has a source code solution, so if you have trouble, feel free to consult it.

Experiment with one or more of the following:

1. Modify the movie playback rate and the compositor rate to see how these values affect playback
2. Substitute your own media for the zebra movie. (What happens if you specify a media file from a streaming server?)
3. Play around with the layers in the compositor to see how this affects display order.
4. Apply a graphics mode to the movie so that it draws semi-transparently over the zebra background.
5. See if you can figure out how to display a controller and associate it with the movie. (Hint: you will need to use the MovieController class.