by Michael Morrison
Java programs have been criticized from early on because of their relatively slow execution speeds. Admittedly, compared to natively compiled programs written in languages like C/C++ or Pascal, Java programs are pretty sluggish. However, this complaint has to be weighed heavily against the inherently cross-platform nature of Java, which simply isn't possible with native programs such as those generated by C/C++ and Pascal compilers. In an attempt to alleviate the inherent performance problems associated with processor-independent Java executables, various companies are offering just-in-time (JIT) Java compilers, which compile Java bytecode executables into native programs just before execution.
This chapter explores JIT compilers and how they impact the overall landscape of Java. You learn all about the Java virtual machine and how JIT compilers fit into its organization. Furthermore, you learn about specific types of Java programs that benefit the most from JIT compilation. By the end of this chapter, you'll have a better understanding of this exciting new technology and how it can improve the performance of your own Java programs.
To fully understand what a just-in-time (JIT) compiler is and how it fits into the Java runtime system, you must have a solid understanding of the Java virtual machine (VM). The Java VM is a software abstraction for a generic hardware platform and is the primary component of the Java system responsible for portability. The purpose of the VM is to allow Java programs to compile to a uniform executable format, as defined by the VM, which can be run on any platform. Java programs execute within the VM itself, and the VM is responsible for managing all the details of actually carrying out platform-specific functions.
When you compile a Java program, it is compiled to be executed under the VM. Contrast this to C/C++ programs, which are compiled to be run on a real (nonvirtual) hardware platform, such as a Pentium processor running Windows 95. The VM itself has characteristics very much like a physical microprocessor, but it is entirely a software construct. You can think of the VM as an intermediary between Java programs and the underlying hardware platform under which all programs must eventually execute.
Even with the VM, at some point, all Java programs must be resolved to a particular under-lying hardware platform. In Java, this resolution occurs within each particular VM implementation. The way this works is that Java programs make calls to the VM, which in turn routes them to appropriate native calls on the underlying platform. Knowing this, it's fairly obvious that the VM itself is highly platform dependent. In other words, each different hardware platform or operating system must have a unique VM implementation that routes the generic VM calls to appropriate underlying native services.
Because a VM must be developed for each different platform, it is imperative that it be as lean as possible. Another benefit of having a compact VM is the ability to execute Java programs on systems with fewer resources than desktop computer systems. For example, JavaSoft has plans to use Java in consumer electronics devices such as televisions and cellular phones. A compact, efficient VM is an essential requirement in making Java programs run in highly constrained environments such as these.
Just as all microprocessors have instruction sets that define the operations they can perform, so does the Java VM. VM instructions compile into a format known as bytecodes, which is the executable format for Java programs that can be run under the VM. You can think of bytecodes as the machine language for the VM. It makes sense, then, that the JDK compiler generates bytecode executables from Java source files. These bytecode executables are always stored as .class files. Figure 44.1 shows the role of the VM in the context of the Java environment.
Figure 44.1: The role of the VM in the Java environment.
In Figure 44.1, notice how the VM is nestled within the Java runtime system. It is through the VM that executable bytecode Java classes are executed and ultimately routed to appropriate native system calls. A Java program executing within the VM is executed a bytecode at a time. With each bytecode instruction, one or more underlying native system calls may be made by the VM to achieve the desired result. In this way, the VM is completely responsible for handling the routing of generic Java bytecodes to platform-specific code that actually carries out a particular function. The VM has an enormous responsibility and is really the backbone of the entire Java runtime environment. For more gory details about the inner workings of the VM, refer to Chapter 34, "Java Under the Hood: Inside the Virtual Machine."
JIT compilers alter the role of the VM a little by directly compiling Java bytecode into native platform code, thereby relieving the VM of its need to manually call underlying native system services. The purpose of JIT compilers, however, isn't to allow the VM to relax. By compiling bytecodes into native code, execution speed can be greatly improved because the native code can be executed directly on the underlying platform. This stands in sharp contrast to the VM's approach of interpreting bytecodes and manually making calls to the underlying platform. Figure 44.2 shows how a JIT compiler alters the role of the VM in the Java environment.
Figure 44.2: The role of the VM and JIT compiler in the Java environment.
Notice that instead of the VM calling the underlying native operating system, it calls the JIT compiler. The JIT compiler in turn generates native code that can be passed on to the native operating system for execution. The primary benefit of this arrangement is that the JIT compiler is completely transparent to everything except the VM. The really neat thing is that a JIT compiler can be integrated into a system without any other part of the Java runtime system being affected. Furthermore, users don't have to fool with any configuration options; their only clue that a JIT compiler is even installed may simply be the improved execution speed of Java programs.
The integration of JIT compilers at the VM level makes JIT compilers a legitimate example of component software; you can simply plug in a JIT compiler and reap the benefits with no other work or side effects.
Even though JIT compiler integration with the Java runtime system may be transparent to everything outside the VM, you're probably thinking that there are some tricky things going on inside the VM. In fact, the approach used to connect JIT compilers to the VM internally is surprisingly straightforward. In this section, I describe the inner workings of Borland's AppAccelerator JIT compiler, which is the JIT compiler used in Netscape Navigator 3.0. Although other JIT compilers, such as Microsoft's JIT compiler in Internet Explorer, may differ in some ways, they ultimately must tackle the same problems. By understanding Borland's approach with AppAccelerator, you gain insight into the implementation of JIT compilers in general.
The best place to start describing the inner workings of the AppAccelerator JIT compiler is to quickly look at how Java programs are executed without a JIT compiler. A Java class that has been loaded into memory by the VM contains a V-table (virtual table), which is a list of the addresses for all the methods in the class. The VM uses the V-table whenever it has to make a call to a particular method. Each address in the V-table points to the executable bytecode for the particular method. Figure 44.3 shows what the physical V-table layout for a Java class looks like.
Figure 44.3: The physical V-table layout for a Java class.
| Note |
The term V-table is borrowed from C++, where it stands for virtual table. In C++, V-tables are attached to classes that have virtual methods, which are methods that can be over-ridden in derived classes. In Java, all methods are virtual, so all classes have V-tables. |
When a JIT compiler is first loaded, the VM pulls a little trick
with the V-table to make sure that methods are compiled into native
code rather than executed. What happens is that
each bytecode address in the V-table is replaced with the address
of the JIT compiler itself. Figure 44.4 shows how the bytecode
addresses are replaced with the JIT compiler address in the V-table.
Figure 44.4: The physical V-table layout for a Java class with a JIT compiler present.
When the VM calls a method through the address in the V-table, the JIT compiler is executed instead. The JIT compiler steps in and compiles the Java bytecode into native code and then patches the native code address back to the V-table. From now on, each call to the method results in a call to the native version. Figure 44.5 shows the V-table with the last method JIT compiled.
Figure 44.5: The physical V-table layout for a Java class with one JIT-compiled method.
One interesting aspect of this approach to JIT compilation is that it is performed on a method-by-method basis. In other words, the compilation is performed on individual methods, as opposed to entire classes. This is very different from what most of us think of in terms of traditional compilation. Just remember that JIT compilation is anything but traditional!
Another added benefit of the method-by-method approach to compilation
is that methods are compiled only when they are called. The first
time a method is called, it is compiled; sub-
sequent calls result in the native code being executed. This approach
results in only the methods that are actually used being compiled,
which can yield huge performance benefits. Consider the case of
a class in which only four out of ten methods are being called.
The JIT compiler compiles only the four methods called, resulting
in a 60-percent savings in compile time (assuming that the compile
time for each of the methods is roughly the same).
Just in case you're worried about the original bytecode once a
method has been JIT compiled, don't worry, it's not lost. To be
honest, I didn't completely tell the truth about how the
V-table stores method address information. What really happens
is that each method has two V-table entries, one for the bytecode
and one for the native code. The native code address is the one
that is actually set to the JIT compiler's address. This V-table
arrangement is shown in Figure 44.6.
The purpose of having both bytecode and native code entries in the V-table is to allow you to switch between which one is executed. In this way, you can simultaneously execute some methods as bytecode and some using JIT-compiled native code. It isn't immediately apparent what benefits this arrangement will have, but the option of conditionally using the JIT compiler at the method level is something that may come in handy.
Okay, so the JIT compiler is integrated with the VM primarily through the V-table for each class loaded into memory. That's fine, but how is the JIT compiler installed and recognized by the VM in the first place? When the VM is first loaded, it looks for the JIT compiler and loads it if it is found. After loading, the JIT compiler installs itself by hooking into the VM and modifying the class-loading mechanism to reference the compiler. From this point on, the VM doesn't know or care about the compiler. When a class is loaded, the compiler is notified through its hook to the VM and the V-table trickery is carried out.
You may have some concerns about how JIT compilers impact the security of Java programs-because they seem to have a lot of say over what gets executed and how. You'll be glad to know that JIT compilers alter the security landscape of Java very little. The reason is that JIT compilation is performed as the last stage of execution, after the bytecode has been fully checked by the runtime system. This is very important because native code can't be checked for security breaches like Java bytecode can be. So it is imperative that JIT compilation occur on bytecode that has already been security checked by the runtime system.
It is equally important that native code (code that has been JIT compiled) is executed directly from memory and isn't cached on a local file system to be executed later. Again, doing so would violate the whole idea of checking every Java program immediately before execution. Actually, this approach wouldn't qualify as JIT compilation anyway, because the code wouldn't really be compiled just in time.
In terms of security, the cleanest and safest approach to JIT compilation is to compile bytecode directly to memory and throw it away when it is no longer needed. Because native code is disposable in this scenario, it is important that it can be quickly recompiled from the original bytecode. This is where the approach of compiling only methods as they are called really shines.
None of the details surrounding JIT compilers would really matter if they didn't perform their job and speed up the execution speed of Java programs. The whole point of JIT compilation is to realize a performance gain by compiling VM bytecode to native code at runtime. Knowing this, let's take a look at just how much of a performance improvement JIT compilers provide.
In assessing JIT compiler performance, it's important to understand exactly where performance gains are made. One common misconception surrounding JIT compilation is the amount of code affected. For example, if a particular JIT compiler improves the execution speed of bytecode by an order of ten (on average), then it seems only logical that a Java program executing under this JIT compiler would run ten times faster. However, this isn't the case. The reason is that many programs, especially applets, rely heavily on the Java AWT, which on the Windows platform is written entirely in native C. Because the AWT is already written in native code, programs that rely heavily on the AWT don't reap the same performance gains as programs that depend on pure Java bytecode. A heavily graphical program that makes great use of the AWT may see performance gains by an order of only two or three.
On the other hand, a heavily processing-intensive Java program that uses lots of floating-point math may see performance gains closer to an order of fifteen. This happens because native Pentium code on a Windows machine is very efficient with floating-point math. Of course, other platforms may differ in this regard. Nevertheless, this will probably remain a common theme across all platforms: nongraphical programs are less affected by JIT compilation than computationally intensive programs.
Now that I've tempered your enthusiasm a little for how greatly JIT compilation impacts performance, let's look at some hard numbers that show the differences between interpreted and JIT-compiled code across different JIT compiler implementations. Figure 44.7 shows a graph of Netscape Navigator 3.0's performance benchmarks for various JIT-compiled Java operations as measured using Pendragon Software's CaffeineMark 2.01 benchmark suite.
In looking at Figure 44.7, you may be wondering exactly what the numbers mean. The numbers show the relative performance of Netscape Nagivator as compared to the Symantec Café applet viewer running in debug mode. The Café applet viewer produces scores of exactly 100 on all benchmark tests, so scores above 100 represent a higher browser execution speed than the Café applet viewer. Likewise, scores lower than 100 represent slower browser execution. You can see that, in some areas, Navigator blows away the Café applet viewer with scores in the thousands. In other areas, however, the JIT-compiled Navigator code slips a little and is actually slower than the interpreted code. Most of these areas are related to graphics operations, which highlights the fact that Café has more efficient graphics support than Navigator.
Figure 44.8 shows the results of running the same benchmark tests on the JIT compiler in Microsoft Internet Explorer 3.0.
It's interesting to note that Internet Explorer outperformed Navigator on all tests except one. This is expected because Microsoft claims to have the fastest Java implementation around. Even so, this is still the first round of support for JIT compilers, so expect to see plenty of competition in the future between the JIT compiler implementations in different browsers. Marketing hype aside, you can see from these figures that JIT compilation improves performance significantly in many areas regardless of your browser of choice. Navigator and Internet Explorer both show an overall performance improvement that is over eleven times faster than Café's interpreted approach. Just remember that this improvement depends largely on the type of applet you are running and whether it is processing or graphics intensive.
In this chapter, you learned about just-in-time (JIT) compilers and how they impact the Java runtime system. You began the chapter by peering into the runtime system to see exactly where JIT compilers fit in. In doing so, you learned a great deal about the Java virtual machine (VM), which is largely responsible for the integration of JIT compilers into the runtime system. Once you gained an understanding of how JIT compilers relate to the VM, you moved on to learning about the details of a particular JIT compiler implementation. This look into the inner workings of a real JIT compiler helped give you insight into what exactly a JIT compiler does.
Even though the technical details of JIT compilers are important to understand, little of it would be meaningful if JIT compilers didn't deliver on their promise to improve Java execution speed. For this reason, you spent the last part of the chapter learning about the specific areas where JIT compilers improve Java performance. Furthermore, you saw benchmark tests comparing JIT compiler performance in the two most popular Web browsers available. Through these benchmark tests, you were able to get an idea of how dramatically JIT compilation can improve performance.