Java 1.1 Unleashed
- 53 -
Just-in-Time Compilers
by Michael Morrison
IN THIS CHAPTER
- Understanding the Java VM
- JIT Compilers and the VM
- Inside a JIT Compiler
- Security and JIT Compilers
- JIT Compiler Performance
- Alternatives to JIT Compilers
Java programs have been criticized from early on because of their relatively slow execution speeds. Admittedly, compared to natively compiled programs written in languages like C/C++ or Pascal, Java programs are pretty sluggish. However, this complaint has to be weighed heavily against the inherently cross-platform nature of Java, which simply isn't possible with native programs such as those generated by C/C++ and Pascal compilers. In an attempt to alleviate the inherent performance problems associated with processor-independent Java executables, various companies are offering just-in-time (JIT) Java compilers, which compile Java bytecode executables into native programs just before execution.
This chapter explores JIT compilers and how they impact the overall landscape of Java. You learn all about the Java virtual machine and how JIT compilers fit into its organization. Furthermore, you learn about specific types of Java programs that benefit the most from JIT compilation. By the end of this chapter, you'll have a lot better understanding of this exciting technology and how it can improve the performance of your own Java programs.
Understanding the Java VM
To fully understand what a just-in-time (JIT) compiler is and how it fits into the Java runtime system, you must have a solid understanding of the Java virtual machine (VM). The Java VM is a software abstraction for a generic hardware platform and is the primary component of the Java system responsible for portability. The purpose of the VM is to allow Java programs to compile to a uniform executable format, as defined by the VM, that can be run on any platform. Java programs execute within the VM itself, and the VM is responsible for managing all the details of actually carrying out platform-specific functions.
When you compile a Java program, it is compiled to be executed under the VM. Contrast this to C/C++ programs, which are compiled to be run on a real (non-virtual) hardware platform, such as a Pentium processor running Windows 95. The VM itself has characteristics very much like a physical microprocessor, but it is entirely a software construct. You can think of the VM as an intermediary between Java programs and the underlying hardware platform on which all programs must eventually execute.
Even with the VM, at some point, all Java programs must be resolved to a particular under-lying hardware platform. In Java, this resolution occurs within each particular VM implementation. The way this works is that Java programs make calls to the VM, which in turn routes them to appropriate native calls on the underlying platform. Knowing this, it's fairly obvious that the VM itself is highly platform dependent. In other words, each different hardware platform or operating system must have a unique VM implementation that routes the generic VM calls to appropriate underlying native services.
Because a VM must be developed for each different platform, it is imperative that it be as lean as possible. Another benefit of having a compact VM is the ability to execute Java programs on systems with fewer resources than desktop computer systems. For example, JavaSoft has plans to use Java in consumer electronics devices such as televisions and cellular phones. A compact, efficient VM is an essential requirement in making Java programs run in highly constrained environments such as these.
Just as all microprocessors have instruction sets that define the operations they can perform, so does the Java VM. VM instructions compile into a format known as bytecode, which is the executable format for Java programs that can be run under the VM. You can think of bytecodes as the machine language for the VM. It makes sense, then, that the JDK compiler generates bytecode executables from Java source files. These bytecode executables are always stored as .class files. Figure 53.1 shows the role of the VM in the context of the Java environment.
The role of the VM in the Java environment.
In Figure 53.1, notice how the VM is nestled within the Java runtime system. It is through the VM that executable bytecode Java classes are executed and ultimately routed to appropriate native system calls. A Java program executing within the VM is executed a bytecode at a time. With each bytecode instruction, one or more underlying native system calls may be made by the VM to achieve the desired result. In this way, the VM is completely responsible for handling the routing of generic Java bytecodes to platform-specific code that actually carries out a particular function. The VM has an enormous responsibility and is really the backbone of the entire Java runtime environment. For more gory details about the inner workings of the VM, refer to Chapter 33, "Java Under the Hood: Inside the Virtual Machine."
JIT Compilers and the VM
JIT compilers alter the role of the VM a little by directly compiling Java bytecode into native platform code, thereby relieving the VM of its need to manually call underlying native system services. The purpose of JIT compilers, however, isn't to allow the VM to relax. By compiling bytecodes into native code, execution speed can be greatly improved because the native code can be executed directly on the underlying platform. This stands in sharp contrast to the VM's approach of interpreting bytecodes and manually making calls to the underlying platform. Fig-ure 53.2 shows how a JIT compiler alters the role of the VM in the Java environment.
The role of the VM and JIT compiler in the Java environment.
Notice that instead of the VM calling the underlying native operating system, it calls the JIT compiler. The JIT compiler in turn generates native code that can be passed on to the native operating system for execution. The primary benefit of this arrangement is that the JIT compiler is completely transparent to everything except the VM. The really neat thing is that a JIT compiler can be integrated into a system without any other part of the Java runtime system being affected. Furthermore, users don't have to fool with any configuration options; their only clue that a JIT compiler is even installed may simply be the improved execution speed of Java programs.
The integration of JIT compilers at the VM level makes JIT compilers a legitimate example of component software; you can simply plug in a JIT compiler and reap the benefits with no other work or side effects.
Inside a JIT Compiler
Even though JIT compiler integration with the Java runtime system may be transparent to everything outside the VM, you're probably thinking that there are some tricky things going on inside the VM. In fact, the approach used to connect JIT compilers to the VM internally is surprisingly straightforward. In this section, I describe the inner workings of Borland's AppAccelerator JIT compiler, which is the JIT compiler used in Netscape Navigator 3.0. Although other JIT compilers, such as Microsoft's JIT compiler in Internet Explorer, may differ in some ways, they ultimately must tackle the same problems. By understanding Borland's approach with AppAccelerator, you gain insight into the implementation of JIT compilers in general.
The best place to start describing the inner workings of the AppAccelerator JIT compiler is to quickly look at how Java programs are executed without a JIT compiler. A Java class that has been loaded into memory by the VM contains a V-table (virtual table), which is a list of the addresses for all the methods in the class. The VM uses the V-table whenever it has to make a call to a particular method. Each address in the V-table points to the executable bytecode for the particular method. Figure 53.3 shows the physical V-table layout for a Java class.
The physical V-table layout for a Java class.
NOTE: The term V-table is borrowed from C++, where it stands for virtual table. In C++, V-tables are attached to classes that have virtual methods, which are methods that can be over-ridden in derived classes. In Java, all methods are virtual, so all classes have V-tables.
When a JIT compiler is first loaded, the VM pulls a little trick with the V-table to make sure that methods are compiled into native code rather than executed. What happens is that each bytecode address in the V-table is replaced with the address of the JIT compiler itself. Fig- ure 53.4 shows how the bytecode addresses are replaced with the JIT compiler address in the V-table.
The physical V-table layout for a Java class with a JIT compiler present.
When the VM calls a method through the address in the V-table, the JIT compiler is executed instead. The JIT compiler steps in and compiles the Java bytecode into native code and then patches the native code address back to the V-table. From now on, each call to the method results in a call to the native version. Figure 53.5 shows the V-table with the last method JIT compiled.
The physical V-table layout for a Java class with one JIT-compiled method.
One interesting aspect of this approach to JIT compilation is that it is performed on a method-by-method basis. In other words, the compilation is performed on individual methods, as opposed to entire classes. This is very different from what most of us think of in terms of traditional compilation. Just remember that JIT compilation is anything but traditional!
Another added benefit of the method-by-method approach to compilation is that methods are compiled only when they are called. The first time a method is called, it is compiled; subsequent calls result in the native code being executed. This approach results in only methods being compiled that are actually used, which can yield huge performance benefits. Consider the case of a class in which only four out of ten methods are called. The JIT compiler compiles only those four methods, resulting in a 60-percent savings in compile time (assuming that the compile time for each of the methods is roughly the same).
Just in case you're worried about the original bytecode once a method has been JIT compiled, don't worry, it's not lost. To be honest, I didn't completely tell the truth about how the V-table stores method address information. What really happens is that each method has two V-table entries: one for the bytecode and one for the native code. The native code address is the one that is actually set to the JIT compiler's address. This V-table arrangement is shown in Figure 53.6.
The physical V-table layout for a Java class with both bytecode and native code entries.
The purpose of having both bytecode and native code entries in the V-table is to allow you to switch between which one is executed. In this way, you can simultaneously execute some methods as bytecode and some using JIT-compiled native code. It isn't immediately apparent what benefits this arrangement will have, but the option of conditionally using the JIT compiler at the method level is something that may come in handy.
Okay, so the JIT compiler is integrated with the VM primarily through the V-table for each class loaded into memory. That's fine, but how is the JIT compiler installed and recognized by the VM in the first place? When the VM is first loaded, it looks for the JIT compiler and loads it if it is found. After loading, the JIT compiler installs itself by hooking into the VM and modifying the class-loading mechanism to reference the compiler. From this point on, the VM doesn't know or care about the compiler. When a class is loaded, the compiler is notified through its hook to the VM and the V-table trickery is carried out.
Security and JIT Compilers
You may have some concerns about how JIT compilers impact the security of Java programs--because they seem to have a lot of say over what gets executed and how. You'll be glad to know that JIT compilers alter the security landscape of Java very little. The reason is that JIT compilation is performed as the last stage of execution, after the bytecode has been fully checked by the runtime system. This is very important because native code can't be checked for security breaches like Java bytecode can be. So it is imperative that JIT compilation occur on bytecode that has already been security checked by the runtime system.
It is equally important that native code (code that has been JIT compiled) is executed directly from memory and isn't cached on a local file system to be executed later. Again, doing so would violate the whole idea of checking every Java program immediately before execution. Actually, this approach wouldn't qualify as JIT compilation anyway, because the code wouldn't really be compiled just in time.
In terms of security, the cleanest and safest approach to JIT compilation is to compile bytecode directly to memory and throw it away when it is no longer needed. Because native code is disposable in this scenario, it is important that it can be quickly recompiled from the original bytecode. This is where the approach of compiling only methods as they are called really shines.
JIT Compiler Performance
None of the details surrounding JIT compilers would really matter if they didn't perform their job and speed up the execution speed of Java programs. The whole point of JIT compilation is to realize a performance gain by compiling VM bytecode to native code at run time. Knowing this, let's take a look at just how much of a performance improvement JIT compilers provide.
In assessing JIT compiler performance, it's important to understand exactly where performance gains are made. One common misconception surrounding JIT compilation is the amount of code affected. For example, if a particular JIT compiler improves the execution speed of bytecode by an order of ten (on average), it seems only logical that a Java program executing under this JIT compiler would run ten times faster. However, this isn't the case. The reason is that many programs, especially applets, rely heavily on the Java AWT, which on the Windows platform is written entirely in native C. Because the AWT is already written in native code, programs that rely heavily on the AWT don't reap the same performance gains as programs that depend on pure Java bytecode. A heavily graphical program that makes great use of the AWT may see performance gains by an order of only two or three.
On the other hand, a heavily processing-intensive Java program that uses lots of floating-point math may see performance gains closer to an order of fifteen. This happens because native Pentium code on a Windows machine is very efficient with floating-point math. Of course, other platforms may differ in this regard. Nevertheless, this will probably remain a common theme across all platforms: Nongraphical programs are less affected by JIT compilation than computationally intensive programs.
Now that I've tempered your enthusiasm a little for how greatly JIT compilation impacts performance, let's look at some hard numbers that show the differences between interpreted and JIT-compiled code across different JIT compiler implementations. Figure 53.7 shows a graph of Netscape Navigator 3.0's performance benchmarks for various JIT-compiled Java operations as measured using Pendragon Software's CaffeineMark 2.01 benchmark suite.
Performance benchmarks for various JIT-compiled Java operations in Netscape Navigator 3.0.
In looking at Figure 53.7, you may be wondering exactly what the numbers mean. The numbers show the relative performance of Netscape Navigator as compared to the Symantec Café applet viewer running in debug mode. The Café applet viewer produces scores of exactly 100 on all benchmark tests, so scores above 100 represent a higher browser execution speed than the Café applet viewer. Likewise, scores lower than 100 represent slower browser execution. You can see that, in some areas, Navigator blows away the Café applet viewer with scores in the thousands. In other areas, however, the JIT-compiled Navigator code slips a little and is actually slower than the interpreted code. Most of these areas are related to graphics operations, which highlights the fact that Café has more efficient graphics support than Navigator.
Figure 53.8 shows the results of running the same benchmark tests on the JIT compiler in Microsoft Internet Explorer 3.0.
Performance benchmarks for various JIT-compiled Java operations in Microsoft Internet Explorer 3.0.
It's interesting to note that Internet Explorer outperformed Navigator on all tests except one. This is expected because Microsoft claims to have the fastest Java implementation around. Even so, this is still the first round of support for JIT compilers, so expect to see plenty of competition in the future between the JIT compiler implementations in different browsers. Marketing hype aside, you can see from these figures that JIT compilation improves performance significantly in many areas, regardless of your browser of choice. Navigator and Internet Explorer both show an overall performance improvement that is over eleven times faster than Café's interpreted approach. Just remember that this improvement depends largely on the type of applet you are running and whether it is processing or graphics intensive.
Alternatives to JIT Compilers
At this point, I hope I have made a pretty decent case in support of JIT compilers as a way of improving the overall performance of Java. Before we settle down with JIT compilers as the only answer to Java's performance woes, it's important to investigate whether there may be other alternatives. I could make some guesses about how Java performance can be improved with other means, but I don't really have to because a couple of other avenues are already being investigated. The rest of this chapter focuses on these other approaches and what they have to offer.
The first alternative to JIT compilers involves rewriting the Java interpreter to be more efficient. Faster, more efficient Java interpreters have already been developed by companies such as Microsoft and have improved the performance of Java code by a factor of two or three over the speed of the standard JDK interpreter. Even so, there are practical limitations surrounding interpreted code as a whole. In other words, faster interpreters are great, but they can never single-handedly boost the performance of Java to any large extent. A more aggressive approach is necessary to reap significant gains.
Another approach involves the use of flash compilers, which are definitely more aggressive than optimized Java interpreters. Unlike JIT compilers, in which the interpreter selectively compiles functions, flash compilers compile all class files as they are downloaded from a server. This approach completely bypasses the Java interpreter, alleviating the associated overhead. Flash compilers fully compile applets and applications to machine code, resulting in execution speeds on the order of compiled languages such as C++. Furthermore, the compiled machine code can be cached for later execution. The ability to cache code is particularly beneficial on corporate intranets, where users regularly execute the same applets and applications.
Asymetrix has developed a fully functioning flash compiler as part of its SuperCede integrated development environment. The SuperCede flash compiler is also available as a separate Web browser plug-in. The SuperCede flash compiler can generate code with minimal optimizations (for users who want to optimize for download time) or with maximum optimizations (for users who want to optimize for execution speed). The SuperCede flash compiler is implemented in the SuperCede virtual machine, which is the only Java virtual machine to support interoperability between Java and ActiveX.
It is difficult to predict if or how JIT compilers, flash compilers, and improved interpreters will coexist as Java matures. For the time being, it looks as if JIT compilers are the strongest candidates for widespread performance gains, as is evidenced by their inclusion in the two most popular Web browsers (Netscape Navigator and Microsoft Internet Explorer). However, the future is still wide open and Java has plenty of room to improve in terms of performance.
Summary
In this chapter, you learned about just-in-time (JIT) compilers and how they impact the Java runtime system. You began the chapter by peering into the runtime system to see exactly where JIT compilers fit in. In doing so, you learned a great deal about the Java virtual machine (VM), which is largely responsible for the integration of JIT compilers into the runtime system. Once you gained an understanding of how JIT compilers relate to the VM, you moved on to learning about the details of a particular JIT compiler implementation. This look into the inner workings of a real JIT compiler helped give you insight into what exactly a JIT compiler does.
Even though the technical details of JIT compilers are important to understand, little of it would be meaningful if JIT compilers didn't deliver on their promise to improve Java execution speed. For this reason, you spent some time learning about the specific areas where JIT compilers improve Java performance. You also saw benchmark tests comparing JIT compiler performance in the two most popular Web browsers available. Through these benchmark tests, you were able to get an idea of how dramatically JIT compilation can improve performance. You finished up the chapter by looking at a couple of other approaches to improving the performance of Java code and how these other options relate to JIT compilers.
©Copyright, Macmillan Computer Publishing. All rights reserved.