Java 1.2 Unleashed
One of the most interesting features that you can add to an applet or application is the capability to play multimedia files, such as audio or video. Recognizing the need for a multimedia API, JavaSoft developed the Java Media Framework (JMF), which consists of an API for playing and receiving multimedia files in a variety of audio and video formats. The JMF includes API packages, media stream/file players, codecs, and the Java Sound Engine.
In this chapter, you'll learn how to use the capabilities provided by the JMF to incorporate audio and video support into your applications and applets. You'll learn the basics of audio and video file formats, learn how to use the JMF player, and investigate the JMF API. You'll also learn about JMF's support for the Real-Time Transport Protocol (RTP). When you finish this chapter, you'll be able to add multimedia features to your applications and applets.
There are a number audio and video file formats that are used to store sounds and moving images. Each of these formats represents a compromise between fidelity, range, file size, and performance. For example, there is a limited range of frequencies that can be detected by the human ear. Within this range, the fidelity of an audio file is determined by the rate at which the sound is sampled (samples/second), the amount of information per sample (8, 16, or 32 bits), and the number of sound channels (mono or stereo). The greater the sample rate and sample size, the greater the file size. File size can be offset by the use of compression, but at higher compression rates there may be detectable impacts on system performance. Specialized codecs (compressors/decompressors) may be used to improve performance.
Video format tradeoffs are similar to audio tradeoffs. However, the size of most video files dwarfs that of audio files. Video fidelity is a function of the size of a video frame (in pixels), the number of colors per pixel, and the number of frames per second. Compression is extremely important in video because video files are so large. Specialized codecs are required for use in nearly all video formats.
Examples of common audio formats are as follows:
The use of these audio formats is organized along political lines. The most common audio format used with Microsoft Windows-based systems is Microsoft's Wave format (.WAV). The most common format used with Solaris is the Sun Audio Format (.AU). Historically, new audio file formats were introduced for use with different computer hardware and software platforms. The RMF format is intended to be a platform- independent format. The MIDI format is not based on sound sampling but is a digital format for identifying the instruments, rhythms, and notes used in a musical composition. In this respect, it is more like digital sheet music or an audio animation.
Examples of common video formats are as follows:
Of these video formats, MPEG produces the highest-quality video using the smallest file size. This is a result of the superior compression techniques it uses. The QuickTime and AVI formats were developed for use with the Macintosh and PC. The QuickTime format is more popular, and QuickTime players are available for a number of operating system platforms. The Vivo and ActiveMovie formats support streaming video, or video that is downloaded a little at a time from an Internet stream instead of as an entire file. The Vivo format is an excellent format for use with Web applications. ActiveMovie is Microsoft's venture into this area.
NOTE: Apple Computer provides an implementation of QuickTime that is independent of the JMF. It is referred to as QuickTime for Java and is available from Apple's Web site at http://www.apple.com/quicktime/.
The Java Media Framework is an API for using audio and video within Java applications and applets. This API supports the playing of a wide variety of media types. It also provides examples of media-playing applications and applets. The JMF is available from JavaSoft's Web site at http://www.javasoft.com/products/java-media/ jmf/index.html. It is packaged as a self-extracting, self-installing file. To install the JMF on Windows 95, 98, or NT, your system should include the following:
Both ActiveMovie and Direct X 2.0 are available from Microsoft at http://www.microsoft.com/directx/resources/devdl.htm.
The media types supported by JMF 1.0 include the following:
The best way to get a feel for the capabilities provided by JMF is to use the JMF Player. If you installed the JMF under Windows, a JMF program group should have been created for you. Double-click the JMF Player icon to launch the JMF Player. Figure 21.1 shows its initial display.
FIGURE 21.1.The JMF Player.
Select Open File from the File menu to launch an Open file dialog box. Navigate to the samples\media subdirectory of the directory in which you installed the JMF. You should find some example media files in this directory. Open Sample1.mov to play a QuickTime movie. Figure 21.2 shows a snapshot of the movie that is displayed.
FIGURE 21.2. Playing a QuickTime movie with the JMF Player.
When you are finished playing the movie, click the pause icon in the lower-left corner of the application window. Then open the Sample2.mpg movie from the same directory. Figure 21.3 provides a snapshot of how this movie is displayed.
Figure 21.3. Playing an MPEG movie with the JMF Player.
You can also try some of the audio samples that are included with the JMF. I was impressed by the clarity of the sound that was played on my notebook computer.
Having had a taste of the capabilities provided by JMF, I'll bet you can't wait to start using it in your applets and applications. We'll cover the JMF API first and then illustrate the use of this API with an application and an applet.
The JMF API consists of the javax.media and the javax.media.protocol packages. These packages are standard extension APIs. The javax.media package is fairly large, consisting of 13 interfaces and 27 classes. Twenty-six of these classes and interfaces define events and event listeners. Event handling is a big part of media playing. However, we'll cover the other classes and interfaces before we introduce the JMF events and event handlers.
The Player interface is the most important element of the javax.media package. Classes that implement this interface are used to play actual multimedia objects. The Player interface extends the MediaHandler, Controller, and Duration interfaces. The realize() method (inherited from Controller) is used to construct the media-specific portions of a Player object. The start() method signals that the media should be played as soon as possible. Other methods are provided for accessing the visual component and controllers associated with the Player object.
The Controller interface extends the Clock interface to manage the state of a media object. Classes that implement this interface manage the following media states:
A Controller defines methods and generates events that support the management of these states. The Player interface is a subinterface of Controller that supports the playing of media.
The Control interface defines methods for exerting control over a media object. The getControlComponent() method returns a GUI component used to control a media object. The CachingControl interface extends the Control interface to define methods used to control the loading of media files/data. The GainControl interface defines methods for getting and setting audio signal gain.
The MediaHandler interface is implemented by classes that manage media that is retrieved from a DataSource object (defined in javax.media.protocol). This interface is extended by the Player and MediaProxy interfaces. The MediaProxy interface is implemented by classes that transform data from one DataSource object to another DataSource object.
The Manager class provides access to system-dependent media resources via static methods. The createPlayer() method is used to create a Player object that is capable of playing a media object referenced by a MediaLocator, URL, or DataSource object. Other methods allow DataSource and TimeBase objects to be created.
The MediaLocator class describes the location of media to be played. MediaLocator objects are constructed by passing a URL object, or a string that represents a URL, to the MediaLocator constructor. The MediaLocator class provides methods for accessing the media's location. The PackageManager class maintains a store of package prefix names used to locate protocol-handling and other media-related classes.
The Time class encapsulates time as used by media players. It provides time constants and methods for accessing time down to the nanosecond level. The TimeBase interface defines methods for a constant, uninterruptable source of time. The Clock interface defines methods for managing time with respect to the playing of media. The Duration interface defines the getDuration() method for obtaining information about the duration of a media object.
As previously mentioned, event handling is a big part of media playing. The javax.media package defines a variety of events to provide feedback to media handling software. The MediaEvent interface is implemented by all JMF events. These event handling classes form the class hierarchy.
ControllerEvent--Base interface for all Controller events.
TransitionEvent--Generated when a Controller changes to a new state.
StartEvent--Generated when a Controller enters the Start state.
RealizeCompleteEvent--Generated when a Controller changes from the Realizing to the Realized state.
PrefetchCompleteEvent--Generated when a Controller moves from the Prefetching to the Prefetched state.
StopEvent--Generated when a Controller enters the Stop state.
DataStarvedEvent--Generated when a Controller has lost data or stopped receiving data.
DeallocateEvent--Generated when the resources of a Controller have been deallocated and need to be reallocated to continue media operations.
EndOfMediaEvent--Generated when a Controller has reached the end of its media and is stopping.
RestartingEvent--Generated when a Controller changes from the Started state to the Prefetching state.
StopByRequestEvent--Generated when a Controller is stopped as a result of its stop() method being invoked.
StopAtTimeEvent--Generated when a Controller reaches its stop time.
StopTimeChangeEvent--Generated by a Controller when its stop time is updated.
ControllerClosedEvent--Generated when a Controller is no longer operational.
ControllerErrorEvent --Generated when an error occurs that causes a Controller to stop functioning.
ConnectionErrorEvent--Generated when an error occurs within a DataSource object.
ResourceUnavailableEvent--Generated when a Controller is unable to access a required resource.
InternalErrorEvent--Generated as the result of an internal error in a Controller.
CachingControlEvent--Generated by a CachingControl object when the caching state changes.
MediaTimeSetEvent--Generated when the media of a Controller has had its time updated.
RateChangeEvent--Generated when the rate of a Controller changes.
DurationUpdateEvent--Generated when the duration of a Controller changes.
GainChangeEvent--Generated by a GainControl object when its state changes.
The ControllerListener interface is used to handle events generated by Controller objects. These events consist of ControllerEvent and all of its subclasses. The GainChangeListener interface is used to handle the GainChangeEvent, which is generated by GainControl objects.
The javax.media.protocol package has nine interfaces and six classes that are used to transfer data from its source to a media player. These classes and interfaces are as follows:
In many cases, you won't have to effect the actual transfer of data from a source to a Player. As you'll see in the MediaApplication application in Listing 21.1, Player implementations transfer data as part of their prefetch processing.
NOTE: Future versions of JMF will support media capture and media conferencing.
Now that you've been introduced to the classes and interfaces of the JMF API, we'll put together an example of an application and applet that uses these classes and interfaces to play audio and video media. The MediaApplication program in the following section presents a simplified version of the JMF Player. This application can play all of the media that the JMF Player can play, but its GUI features have been minimized so that the application source code can fit in and be described within a single chapter. After presenting the MediaApplication program, the next section shows how to include media playing capabilities in an applet.
The MediaApplication program in Listing 21.1 covers all of the basics of media playing using the JMF. Make sure that you have the JMF installed before running the program. Its initial display is shown in Figure 21.4. The program isn't as pretty as the JMF Player, but at 20% of its size, it is much easier to understand. The MediaApplication program is capable of playing all of the media types that the JMF Player plays.
Select Open from the File menu and open the Sample1.mov file that is contained in the samples\media subdirectory of the JMF directory. Note that the program informs you that it is loading the media player and the media (see Figure 21.5).
When the appropriate media player and the selected media have been loaded, the program plays the media in the center of the application window. Note that a media control slider is displayed at the bottom of the application window, as shown in Figure 21.6.
Select Open from the File menu to play other media files, such as the MPEG movie or audio files that are provided with the JMF. Note that the old GUI components are removed and new ones are added when switching between media files.
FIGURE 21.4. The MediaApplication opening display.
FIGURE 21.5. The user is notified that the media is loading.
FIGURE 21.6. The MediaApplication displays a QuickTime movie.
){kind=link}
){kind=link}
){kind=link}
){kind=link}
){kind=link}
){kind=link}
import java.awt.*;
import java.awt.event.*;
import ju.ch09.*;
import javax.media.*;
public class MediaApplication extends Frame
implements ControllerListener {
// Declare media-related variables
Player player = null;
Player newPlayer = null;
Component visualComponent = null;
Component controllerComponent = null;
// Other variables
Object menuItems[][] = {{"File","Open","-","Exit"}};
MenuItemHandler mih = new MenuItemHandler();
MyMenuBar menuBar = new MyMenuBar(menuItems,mih,mih);
int screenWidth = 400;
int screenHeight = 400;
TextField text = new TextField();
String directory = ".";
public static void main(String args[]){
MediaApplication app = new MediaApplication();
}
public MediaApplication() {
super("MediaApplication");
setMenuBar(menuBar);
add("North",text);
setSize(screenWidth,screenHeight);
addWindowListener(new WindowEventHandler());
show();
}
String getFileName() {
// Display file dialog
FileDialog dialog = new FileDialog(MediaApplication.this,
"Open Media File", FileDialog.LOAD);
dialog.setDirectory(directory);
dialog.show();
if(dialog.getFile()==null) return null;
directory = dialog.getDirectory();
String file = directory + dialog.getFile();
return file;
}
Player createPlayer(String fileName) {
Player newPlayer;
try {
MediaLocator locator = new MediaLocator("file:"+fileName);
if(locator == null) return null;
newPlayer = Manager.createPlayer(locator);
}catch(Exception ex) {
text.setText(ex.toString());
return null;
}
return newPlayer;
}
void realizeComplete() {
visualComponent = player.getVisualComponent();
controllerComponent = player.getControlPanelComponent();
if(visualComponent != null)
add("Center",visualComponent);
if(controllerComponent != null)
add("South",controllerComponent);
validate();
player.prefetch();
}
void prefetchComplete() {
text.setText("");
if(player.getTargetState() != Controller.Started)
player.start();
}
void controllerError() {
player.close();
if(visualComponent != null) remove(visualComponent);
if(controllerComponent != null) remove(controllerComponent);
validate();
visualComponent = null;
controllerComponent = null;
player.removeControllerListener(this);
player = null;
}
void controllerClosed() {
if(visualComponent != null) remove(visualComponent);
if(controllerComponent != null) remove(controllerComponent);
player = null;
System.gc();
System.runFinalization();
if(newPlayer!=null) {
player = newPlayer;
newPlayer = null;
player.addControllerListener(this);
text.setText("Loading ...");
player.realize();
}
validate();
}
public synchronized void controllerUpdate(ControllerEvent e) {
// Determine event type
if(e instanceof RealizeCompleteEvent) realizeComplete();
else if(e instanceof PrefetchCompleteEvent) prefetchComplete();
else if(e instanceof ControllerErrorEvent) controllerError();
else if(e instanceof ControllerClosedEvent) controllerClosed();
}
class MenuItemHandler implements ActionListener, ItemListener {
public void actionPerformed(ActionEvent ev){
String s=ev.getActionCommand();
if(s.equals("Exit")){
System.exit(0);
}else if(s=="Open"){
// Get the name of the media file
String fileName = getFileName();
if(fileName == null) return;
// Create player for file
newPlayer = createPlayer(fileName);
// Stop old player
boolean closingPlayer = false;
if(player!=null){
closingPlayer = true;
player.close();
}
if(newPlayer == null) return;
if(!closingPlayer) {
player = newPlayer;
player.addControllerListener(MediaApplication.this);
text.setText("Loading ...");
player.realize();
}
}
}
public void itemStateChanged(ItemEvent e){
}
}
class WindowEventHandler extends WindowAdapter {
public void windowClosing(WindowEvent e){
// Stop the player
if(player!=null) {
player.close();
while(player!=null) {
// wait a half second
try{
Thread.currentThread().sleep(500);
}catch(Exception ex) {
}
}
}
System.exit(0);
}
}
}
The MediaApplication class implements the ControllerListener interface so that it can handle Controller-related events involved in the loading and playing of media. The player and newPlayer variables are used to manage the current Player object and any new Player object that is created. The visualComponent and controllerComponent variables reference the visual component displayed by a Player object and the component that is used to control the Player.
The user plays a new media file by opening it using the Open menu item in the File menu. To understand how this works, we'll start with the actionPerformed() method of the MenuItemHandler class and trace the thread of execution.
When the Open menu item is selected, the actionPerformed() method invokes the getFileName() method to retrieve the name of the file to be opened. The getFileName() method displays a File Open dialog box to the user. If the user selects a file, actionPerformed() passes the file name to the createPlayer() method. The createPlayer() method creates a Player object by first creating a MediaLocator object that identifies the selected media file's location, and then invoking the createPlayer() method of the Manager class to create a Player object that is suitable for the media. The new Player object is then returned to actionPerformed().
The actionPerformed() method checks to see if there is currently a media file being played. If so, it invokes the close() method of the current Player object. The starting of the new player is then put off until the current player generates the ControllerClosedEvent.
If there is no current player, actionPerformed() adds the MediaApplication instance as a listener for Controller-related events and then invokes the realize() method of the new Player object. The TextField is also updated with a loading message.
At this point, a new media file has been opened. If there was no previous file being played, the new Player is in the Realizing state. Otherwise, the old Player is about to be changed into the Stop state. The rest of the media playing involves handling of ControllerEvent events.
The controllerUpdate() method implements the ControllerListener interface and provides a central point for handling all Controller-related events. Remember that all Player objects are also Controller objects. The controllerUpdate() method checks the event to see which subclass of ControllerEvent it is an instance of. It then invokes one of the following four methods, depending on the type of event:
As you can see from the MediaApplication's description, very little code is required to set up a Player. Once a Player has been set up, the bulk of the processing involves handling the events that occur as the Player moves from state to state.
The MediaApplet, shown in Listing 21.2, shows how media playing capabilities can be incorporated into an applet. The MediaApplet is a simplification of MediaApplication that plays a single QuickTime file. The HTML file shown in Listing 21.3 is used to run MediaApplet. Before running the applet, copy the Sample1.mov file from the samples\media directory of JMF to your ju\ch21 directory. When you open media.htm with appletviewer, it displays the applet shown in Figure 21.7. MediaApplet can be easily tailored to play other media files.
FIGURE 21.7. The MediaApplet displays a QuickTime movie.
Having covered MediaApplication, the source code of MediaApplet will be easy to understand. The MediaApplet class implements the ControllerListener interface and declares the player, visualComponent, and controllerComponent variables in the same manner as MediaApplication.
The init() method lays out the applet and creates the Player object. It creates a URL object for the Sample1.mov file and then passes this URL to the createPlayer() method of the Manager class. It then adds the MediaApplet instance as an event handler for ControllerEvent events and invokes the Player's realize() method.
The stop() method responds to the stopping of the applet by closing the current Player and waiting until it has been completely closed. It does this to keep the media from being played in the absence of a visible applet.
The realizeComplete(), prefetchComplete(), controllerError(), controllerClosed(), and controllerUpdate() methods handle Controller-related events in the same manner as MediaApplication.
import java.applet.*; import java.awt.*; import java.awt.event.*; import ju.ch09.*; import javax.media.*; import java.net.*; public class MediaApplet extends Applet
){kind=link}
implements ControllerListener {
// Declare media-related variables
Player player = null;
Component visualComponent = null;
Component controllerComponent = null;
// Other variables
TextField text = new TextField();
public void init() {
setLayout(new BorderLayout());
add("North",text);
// Create player
try {
URL url = new URL(getDocumentBase(),"Sample1.mov");
player = Manager.createPlayer(url);
} catch(Exception ex) {
text.setText(ex.toString());
}
if(player != null) {
player.addControllerListener(this);
text.setText("Loading ...");
player.realize();
}else text.setText("Unable to play media.");
}
public void stop() {
if(player!=null) player.close();
while(player!=null) {
// wait a half second
try{
Thread.currentThread().sleep(500);
}catch(Exception ex) {
}
}
}
void realizeComplete() {
visualComponent = player.getVisualComponent();
controllerComponent = player.getControlPanelComponent();
if(visualComponent != null)
add("Center",visualComponent);
if(controllerComponent != null)
add("South",controllerComponent);
validate();
player.prefetch();
}
void prefetchComplete() {
text.setText("");
if(player.getTargetState() != Controller.Started)
player.start();
}
void controllerError() {
player.close();
if(visualComponent != null) remove(visualComponent);
if(controllerComponent != null) remove(controllerComponent);
validate();
visualComponent = null;
controllerComponent = null;
player.removeControllerListener(this);
player = null;
}
void controllerClosed() {
if(visualComponent != null) remove(visualComponent);
if(controllerComponent != null) remove(controllerComponent);
player = null;
validate();
}
public synchronized void controllerUpdate(ControllerEvent e) {
// Determine event type
if(e instanceof RealizeCompleteEvent) realizeComplete();
else if(e instanceof PrefetchCompleteEvent) prefetchComplete();
else if(e instanceof ControllerErrorEvent) controllerError();
else if(e instanceof ControllerClosedEvent) controllerClosed();
}
}
<HTML> <HEAD> <TITLE>A Media-Playing Applet</TITLE> </HEAD> <BODY> <APPLET CODE="MediaApplet.class" HEIGHT=400 WIDTH=400> </APPLET> </BODY>
</HTML>
The JMF also provides support for the Real-Time Transport Protocol (RTP). RTP is a protocol for transferring audio and video data in real-time from a media server to media players. It is a streaming protocol and does not guarantee error-free delivery of data. Typical media players drop late or error packets in order to keep up with new packets that are being received. RTP is designed to use the UDP transport protocol. It is supported by both Microsoft and Netscape and promises to be a popular protocol for streaming audio and video over the Internet. RTP is described in RFC 1889, which is available at http://www.cis.ohio-state.edu/rfc/rfc1889.
JMF supports RTP playback (client-side RTP) through the RTP Session Manager API. This API is contained in the javax.medi.rtp package. The RTP Session Manager API was not fully implemented at the time of this writing.
In this chapter, you learned how to use the capabilities provided by the JMF to incorporate audio and video support into your applications and applets. You covered the basics of audio and video file formats, learned how to use the JMF player, and investigated the JMF API. You also learned about JMF's support for RTP. In the next chapter, you'll learn how to create Java-based animations for use in applets and applications.
© Copyright 1998, Macmillan Publishing. All rights reserved.