Chapter 19

Extending Your Programs with Native Methods

CONTENTS

• An Overview of Native Methods
• Who Am I? A Java Class to Identify the User
  • The Class Definition
• The Nuts and Bolts of Native Methods
  • Configuring Your Environment for Solaris
  • Building Native Methods for Solaris
  • Configuring Your Environment for Windows 95/NT
  • Building Native Methods for Windows 95/NT
  • Troubleshooting Native Methods
• The Method and the Function
• Arguments and Return Values
• Using Java Objects from Native Methods
  • Accessing Instance Variables from Native Methods
  • Accessing Class Variables from Native Methods
  • Calling a Java Constructor
  • Calling a Java Method
  • Calling a Static Java Method
• Multithreading and Native Methods
• Native Methods and Exceptions
• Using Java Strings from Native Methods
• Summary

The Java Virtual Machine provides sophisticated capabilities for creating user interfaces, performing Internet and WWW accesses, and for general-purpose programming. Unfortunately, the Java Virtual Machine has some (deliberate) limitations. First and foremost, the JVM only supports functionality that applies to many platforms. That means that unaided Java applications cannot take full advantage of their host platforms. The Java development team also introduced some artificial limitations in the interest of security. For the most part, these restrictions are not onerous. In fact, users can have a relatively high degree of confidence that Java applications and applets are not malicious. Without such assurances, the World Wide Web community would surely have revolted at the notion of self-downloading, self-executing code in a Web page. Just imagine the prospect of a cross-platform virus that attaches itself to Web pages! In some cases, however, these benevolent restrictions prevent applications from making the most of Java.

Consider these situations:

• You have a large, specialized library of C or C++ functions that you would like to use from Java applications.
• Your application must use a non-standard (that is, non-JDBC) database access library.
• An existing application that communicates through an operating system-specific channel (such as Windows DDE or UNIX shared memory) must be integrated with a new Java application.
• Your application needs to automatically determine operating system-specific information, such as the user's Netware ID.

Native methods can solve all of these problems. A Java class can declare a native method to indicate that the actual code for the method is provided in another language. (At present, native methods must be implemented in C. Support for other languages is in the works.) That code is compiled to the native machine code of your particular platform, hence the name "native method." Before it can be used by the Java Virtual Machine, the compiled code must be linked into a dynamically loadable library suitable for use on the target platform. Finally, the DLL or .so file must be installed on the target computer.

There are two parts to every native method, the Java declaration and the "native" implementation. As you do with all other methods, you declare a native method inside a Java class. Native methods can be final, static, or synchronized. They can throw exceptions and be inherited (unless they are final, of course). Because the implementation is compiled for the target platform, it can take full advantage of the capabilities of that platform. Native methods provide a powerful means of extending the Java Virtual Machine.

You have probably guessed by now that native methods are not cross-platform by nature. Your native methods will be as portable as the C code in which you write them and the APIs they call. Because the most common reason for creating native methods is to use a platform-specific API, native methods tend not to be very portable.

Because the dynamically loadable library must be installed on the target machine prior to execution of the Java classes that need it, you will have to face all the issues of software distribution and configuration management. On the other hand, native methods can only be called from Java applications, which have to be distributed and installed themselves. So, the added difficulty of distributing multiple native-method libraries should not pose too great a hurdle.

An Overview of Native Methods

All functions in Java are object methods. Even the declaration part of your native methods must be contained within a class. Therefore, the first step to creating native methods is to define one or more class interfaces. If you are dealing with a large number of functions, it will be useful for you to partition them into logical, coherent groups. These should form the nucleus of each class. There are basically two approaches for defining a class interface that will wrap the native methods. Suppose you are dealing with native methods to provide file access. Assume that your target platform's standard runtime library supports multithreading. If it does not, you will have to serialize access to all RTL functions.

One approach we could take is to create a File class that contains a mixture of native and Java methods that implement an abstraction of a disk file. (This is the approach used most often by the Java development team.) As an alternative design, we could create two classes: a File class that presents an abstract view of files, and a FileImp class that contains all of the native methods relevant to files. The File class would then contain a reference to a single global instance of FileImp. See Figure 19.1.

Figure 19.1: Encapsulating a native interface.

Ultimately, both techniques result in a File class that presents a clean abstraction of a file. The first approach (the single-class approach) has a significant drawback in terms of maintainability, however. Whenever the File class changes, the dynamically loadable library must be rebuilt. In fact, you would need to regenerate the C header and C stub files and recompile the library for every interface change in File.

With the second approach (the split interface), FileImp is insulated from changes to File. Likewise, changes to FileImp do not affect File. The methods of FileImp do not need to be a direct mapping from the target API. You can give them abstract names and easily define an interface, which will rarely (if ever) need to change. For example, the POSIX API defines two sets of file-manipulation functions. Roughly, the sets correspond to those which use handles returned by open, and those which use a structure pointer returned by fopen. The Win32 API defines a group of file-manipulation functions that use a Handle returned by CreateFile. Your FileImp class would define a single method called Open as illustrated by Listing 19.1. Now any client of File can call Open without regard to the underlying system-level API.

Listing 19.1. Definition of the native Open method.

class FileImp {
      .
    public native boolean Open(String Pathname);
      .
 }

After you have designed your class interfaces, you will need to create the Java classes. These will be identical to any other Java class, except for the use of the keyword native. Native signals to javac that this method body will be provided by a library loaded at runtime. Therefore, you do not provide a method body for a native method. The method declaration looks the same as if you were declaring an abstract method. This is no coincidence. In both cases, you are telling javac that you want to defer the definition of this method until later. With an abstract method, you are saying that "later" means a subclass. With a native method, "later" means runtime loadable C code.

After creating and compiling your Java classes, you will use javah to create two C files: a header file and a stub file. (See Chapter 14, "Using javah" for usage information for javah.) The header file contains a structure definition and function declarations (prototypes) for the native methods. The stub file contains some "glue" code, which you should never need to change. Because both files created by javah are generated automatically, you should never change either of them. javah maps the instance variables and methods of the class directly into the C files. So, if you ever change the Java class, you must regenerate the C files. If you do not, strange and unpredictable behavior will result. For example, javah created the header in Listing 19.3 from the class in Listing 19.2.

Listing 19.2. This is the original PortfolioEntry class.

class PortfolioEntry {
    String  TickerSymbol;
    float   LastQuote;
    float   BoughtAtPrice;
    float   LastDividends;
    float   LastEPS;
 }

Listing 19.3. The structure definition javah created from PortfolioEntry.

typedef struct ClassPortfolioEntry {
     struct Hjava_lang_String *TickerSymbol;
     float LastQuote;
     float BoughtAtPrice;
     float LastDividends;
     float LastEPS;
 } ClassPortfolioEntry;

Now suppose that you add a new integer member, NumberOfShares, to the Java class, right after the TickerSymbol. If you do not recreate the header file, your C code will be using the old structure layout. Therefore, your C code would access every member from LastQuote. Your C code would use the offset for NumberOfShares (an integer) as if it were LastQuote (a float)! This mismatch can cause some of the most irreproducible and hard-to-find bugs. A similar problem can arise with the methods. In that case, you are even more likely to have problems, because any change in a method's signature means that you have to regenerate the stub and header files.

One good way to avoid this problem is to make sure that you rarely need to change your classes after you write them. Good design will help there, but changes are inevitable. Creating a makefile is the best way to ensure that you always use the most up-to-date header and stub files. Some version of "make" is available for every platform supported by the JDK. Later in this chapter, you will find sample makefiles for Java classes with native methods.

Javah creates two files for you, but it leaves the most important file up to you. Along with the header and stub files, you will need to write an implementation file. The implementation file must contain the definitions of the functions that the header file declares. All of the really interesting things happen in the implementation file. Figure 19.2 depicts the relationships among the four files-the Java source, the C header, the C stubs, and the C implementation files.

Figure 19.2: The relationship between the Java class file and the three C files..

At this point, a simple example will illustrate the interaction between the Java class and the three C files.

Who Am I? A Java Class to Identify the User

To demonstrate the entire process, from the Java class all the way to the implementation file, we will develop a class that can return the current user's name as a string. This example will use Windows NT native methods.

The Class Definition

What do we really want to know? For now, we just want to know the user's login name. Various platforms have different API calls to get this information. Good design dictates that we should have a method name that is descriptive of what we want to know, not how we learn it. So, let's just call this method Username. It will return a Java string.

Remember that there are no global functions in Java. Even though we really only need one function, we have to put it in a class. At first, this might seem like a lot of trouble. After all, who wants to define a complete class just to get one method? But think about the future. Will this class always have just one method? Odds are that, like everything else, the requirements for this class will evolve over time. Perhaps there are other details about users that would be of interest to your application (or even a future application). The extra few minutes spent here in creating a class give you a framework in which to build additional functionality later.

Listing 19.4 shows the Java source for the UserInformation class.

Listing 19.4. The Java class definition of UserInformation.

 1:  class UserInformation {
 2:     static {
 3:        System.loadLibrary("UserInformation");
 4:     }
 5:     public native String Username();
 6:  }

Here is a line-by-line description of the code:

Line 1: This line defines the class UserInformation. There is nothing fancy about this class; it inherits directly from java.lang.Object. Although this class does not show it, it is usually a good idea to make native methods-sometimes entire classes-"final." Otherwise, anyone can inherit from your native methods and possibly subvert their behavior.

Line 2: This line defines a static block of code. Static blocks are executed when the class itself is loaded. You can think of a static block as a constructor for the class as a whole.

Line 3: When this class is loaded, direct the runtime system to load the dynamically loadable library UserInformation. Exactly how the library is located and loaded varies by target platform. See the appropriate "Configuring Your Environment" section for your platform later in this chapter.

Line 5: This line defines the method Username as a native method that returns a Java String object.

Now, compile this class using javac. (For details on javac usage, see Chapter 9, "javac: The Java Compiler.")

> javac UserInformation.java

This will produce the usual class file UserInformation.class. The next step is to create the header and stub files using javah. (For details on javah usage, see Chapter 14.)

> javah UserInformation

> javah -stubs UserInformation

This will produce two files, UserInformation.h and UserInformation.c. These are the header and stub files. You can disregard UserInformation.c (until it is time to compile, of course.) Listing 19.5 contains the header file UserInformation.h as created by javah. Your output file may vary in some details, depending on your platform and version of the JDK.

Listing 19.5. The header file UserInformation.h created from the UserInformation.class class file.

 0   /* DO NOT EDIT THIS FILE - it is machine generated */
 1   #include <native.h>
 2   /* Header for class UserInformation */
 3
 4   #ifndef _Included_UserInformation
 5   #define _Included_UserInformation
 6
 7   typedef struct ClassUserInformation {
 8       char PAD;   /* ANSI C requires structures to have at least one member */
 9   } ClassUserInformation;
 10  HandleTo(UserInformation);
 11
 12  #ifdef __cplusplus
 13  extern "C" {
 14  #endif
 15  struct Hjava_lang_String;
 16  extern struct Hjava_lang_String
      *UserInformation_Username(struct HUserInformation *);
 17  #ifdef __cplusplus
 18  }
 19  #endif
 20  #endif

Notice the structure definition on lines 7 through 9. Because we did not define any members of the Java class, javah created this dummy structure. This will keep C compilers happy, but you should not attempt to use PAD to store information. The Java specification makes no guarantees about the contents of PAD. For our purposes, the only line of real interest is line 16, where the declaration for Username appears. This is the function that our implementation file must define. Notice that instead of taking a pointer to a ClassUserInformation structure, the C function gets a pointer to an HUserInformation structure. See the sidebar "Where Did HUserInformation Come From?" for details on this structure.

Finally, the time has come to write the implementation file. By convention, the implementation filename ends in "Imp". For UserInformation, our implementation file is named UserInformationImp.c. Listing 19.6 shows the implementation file for UserInformation, followed by a line-by-line description of the code. (We will not explore the target platform's API functions. After all, this is a book about Java, not C.)

Listing 19.6. The implementation file UserInformationImp.c.

 1:  /* UserInformationImp.c                                    */
 2:  /* Implementation file for Java class UserInformation      */
 3:  #include <StubPreamble.h>
 4:  #include "UserInformation.h"
 5:  #include <winnetwk.h>
 6:
 7:  struct Hjava_lang_String *UserInformation_Username(
 8:     struct HUserInformation *hUserInformation
 9:  )
10:  {
11:    char szUserName[128];
12:    DWORD cchBuffer = sizeof(szUserName);
13:    if(NO_ERROR == WNetGetUser(NULL,szUserName,&cchBuffer))
14:       return makeJavaString(szUserName, sizeof(szUserName));
15:    else {
16:       printf("UserInformation_Username: GetLastError = %x\n",
17:          GetLastError());
19:       return makeJavaString("", 0);
20:    }
21: }

Here is a line-by-line description of the implementation file:

Line 3: Every native method implementation file must include StubPreamble.h. Indirectly, it provides all of the macros, definitions, and declarations that enable C code to interoperate with the Java Virtual Machine.

Line 4: Include the header file created by javah. This brings in the structure definitions ClassUserInformation and HUserInformation.

Line 5: Include the Win32 header file, which defines WNetGetUser-the function we will use to find the user's name.

Lines 7-9: This function signature must be identical to the corresponding function declaration in UserInformation.h.

Lines 11-12: Declare two local variables-a character array that will hold the C string of the user's name, and a double word that will indicate the buffers size.

Line 13: Retrieve the user's login name by calling WNetGetUser and checking the return value for failure. If WNetGetUser fails because the buffer is too small, cchBuffer will contain the actual buffer length needed.

Line 14: If the call to WGetNetUser succeeded, construct a Java string object from the C string. The function makeJavaString is declared in javaString.h, along with several other functions which C code can use to manipulate Java string objects. Not all Java classes have such convenient C interfaces. In general, you will use the classes' inherent capabilities by calling the Java code from your C code.

Lines 15-19: If the call to WGetNetUser failed, print an error message on stdout and construct an empty Java string to return. Returning an empty string is more conscientious than returning a NULL. Java has garbage collection, so you do not need to worry about the eventual destruction of these strings.

Now we just need to compile and link the two C files. For complete details on building the code, see the following sections called "Building Native Methods for Solaris" and "Building Native Methods for Windows 95/NT," whichever is appropriate for your target platform. For now, the sample makefile in Listing 19.7 will make things more convenient. (It will also make sure that your header and stub files stay current with respect to your .java file.) Remember to put tabs, not spaces, at the beginnings of the action lines.

Listing 19.7. Sample makefile suitable for use on Windows 95/NT.

 cc         = cl
 LFLAGS     = -MD -LD
 CLASSPATH  = .;$(JAVA_HOME)\lib\classes.zip
 
 CLASS      = UserInformation
 
 all: $(CLASS).dll $(CLASS).class main.class
 
 main.class: main.java
     javac main.java
 
 $(CLASS).class: $(CLASS).java
     javac $(CLASS).java
 
 $(CLASS).dll: $(CLASS)Imp.c $(CLASS).class
     javah -classpath $(CLASSPATH) $(CLASS)
     javah -stubs -classpath $(CLASSPATH) $(CLASS)
     $(cc) $(CLASS)Imp.c $(CLASS).c -Fe$(CLASS).dll $(LFLAGS) mpr.lib javai.lib

Once you compile your native methods into a dynamically loadable library, you can create a simple class to exercise this native method from an application. When you are testing an application that uses native methods, the command-line Java interpreter "java" can be a tremendous help. In this case, the class main simply creates an instance of UserInformation and prints the results of Username(). Figure 19.2 shows the sequence of events that makes this happen.

>java main
Your username is: mtnygard

Now that you have gone through the entire process one step at a time, you can delve into the nuts and bolts. The next sections will refer back to this example from time to time as they explore the details of the interaction between native methods and pure Java code.

The Nuts and Bolts of Native Methods

Because native language methods are so closely tied to a particular platform, the difference in procedures is more pronounced than with the JDK itself. For example, the compilers for different platforms have wildly different command-line arguments. Because of the possible variations and permutations, this section cannot present an exhaustive reference on the native compilers. Therefore, you should always consider the appropriate compiler reference as the ultimate authority. The procedures in these sections worked at the time this was written, using JDK 1.0.2 on the following platforms: SPARC Solaris 2.5, x86 Solaris 2.5, Microsoft Windows 95, Microsoft Windows NT 3.51, and Microsoft Windows NT 4.0 Beta. The Solaris platforms use the usual "cc" compiler. The Windows platforms have been tested using Visual C++ 2.0 and 4.0.

Configuring Your Environment for Solaris

If you have not already done so, you should modify your PATH variable to include the /java/bin directory. In addition, setting your JAVAHOME and CLASSPATH variables properly will help javah run smoothly.

When your class's static block calls System.loadLibrary, the Java runtime will search for the library in the current directory. If you intend to install your dynamic libraries in any other directory, you will need to set your library search path to include that directory. For example, if your libraries are stored in java_lib in your home directory, use the following command in Bourne and Korn shells:

$ LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$HOME/java_lib
$ export LD_LIBRARY_PATH

In C shell, use this:

% setenv LD_LIBRARY_PATH "$LD_LIBRARY_PATH:$HOME/java_lib"

Building Native Methods for Solaris

Use the following command to compile the code and link the dynamic library:

$ cc -G UserInformation.c UserInformationImp.c \
> -o libUserInformation.so

You will probably need to use the -I flag to tell the compiler where to find the Java header files:

$ cc -G -I${JAVA_HOME}/include -I${JAVA_HOME}/include/solaris \
> UserInformation.c UserInformationImp.c \
> -o libUserInformation.so

The linker will create libUserInformation.so in your current directory. For details about the linker options, refer to the man pages for cc and ld.

To execute System.loadLibrary("libname"), the Solaris Java Virtual Machine will search for a library named liblibname.so.

Configuring Your Environment for Windows 95/NT

You should either make these changes to your C:\AUTOEXEC.BAT file or to a batch file you will run every session.

Setting your JAVA_HOME and CLASSPATH variables properly will help javah run smoothly. In addition, if you have not already done so, you should modify your PATH variable to include the %JAVA_HOME%\bin directory.

When your class's static block calls System.loadLibrary, the Java runtime will search for the library in the PATH variable, as well as the directory in which the executable lives. If you intend to install your dynamic libraries in any directory other than these, you will need to set your path to include that directory.

To make the compile process a little easier, you might want to set the IncLUDE and LIB variables to point to the Java files:

set IncLUDE=%JAVA_HOME%\include;%IncLUDE%
set LIB=%JAVA_HOME%\lib;%LIB%

Building Native Methods for Windows 95/NT

Use the following command to compile and link the dynamic library:

C:\> cl UserInformation.c UserInformationImp.c
-FeUserInformation.dll -MD -LD [other_libs] javai.lib

The linker will create UserInformation.dll in the current directory. For details on the linker options, refer to the Visual C++ online manual.

To execute System.loadLibrary("libname"), the Windows 95/NT Java Virtual Machine will search for a library named libname.dll.

Troubleshooting Native Methods

Here are some common exceptions you might see when you run your program.

java.lang.UnsatisfiedLinkError no hello in LD_LIBRARY_PATH
   at java.lang.Throwable.(Throwable.java)
   at java.lang.Error.(Error.java)
   at java.lang.LinkageError.(LinkageError.java)
   at java.lang.UnsatisfiedLinkError.(UnsatisfiedLinkError.java)
   at java.lang.Runtime.loadLibrary(Runtime.java)
   at java.lang.System.loadLibrary(System.java)
   at UserInformation.(UserInformation.java:5)
   at
java.lang.UnsatisfiedLinkError: Username
   at main.main(main.java:6)

This exception appears on Solaris systems. It means that you have a library path set, but the particular library is not in it. You need to modify your library path to include the directory where your library lives.

java.lang.NullPointerException
   at java.lang.Runtime.loadLibrary(Runtime.java)
   at java.lang.System.loadLibrary(System.java)
   at UserInformation.(UserInformation.java:5)
   at
java.lang.UnsatisfiedLinkError: Username
   at main.main(main.java:6)

This exception also appears on Solaris systems. It indicates that you do not have a library path set at all, and that the runtime cannot find your library without it. You should either set your library path or move your library to the current directory.

Unable to load dll 'UserInformation.dll' (errcode = 485)
Exception in thread "main" java.lang.UnsatisfiedLinkError:
   no UserInformation in shared library path
   at java.lang.Runtime.loadLibrary(Runtime.java:268)
   at java.lang.System.loadLibrary(System.java:266)
   at UserInformation.<clinit>(UserInformation.java:3)
   at
java.lang.UnsatisfiedLinkError: Username
   at main.main(main.java:6)

This is essentially just the Windows version of the same problem. Again, the solution is to copy the library into a directory that is in the PATH, or to modify the PATH to include the library's directory.

java.lang.UnsatisfiedLinkError: Username
        at main.main(main.java:6)

If you get this exception by itself, without a larger walkback above it, then your library is missing the function being called by this native method.

As you can see, when you are using native methods, most of the problems show up when the runtime attempts to load the library. Unfortunately, there is no simple solution. This is basically an installation and configuration management problem. Worse yet, there is no way for one class to catch exceptions that occur when the JVM is loading another class.

The Method and the Function

There are two parts to every native method: the Java declaration and the native language definition. The Java Virtual Machine provides enough capabilities that the native language component can do virtually everything that a typical Java method can. This section will examine the two sides of a native method: the Java declaration and the native language definition. The next few sections will examine how the native language component works with the JVM to make all of this work.

Starting with the Java declaration, consider the method signature from the UserInformation example earlier in the chapter.

public native String Username();

It looks more or less the same as any other method. The native keyword means that the method body is not needed. (In fact, it is not permitted.) The native keyword tells the compiler that the definition (the method body) is provided in a different language.

In this example, no arguments were needed. However, you can pass arguments to native methods and get return values from them. These arguments and return values can include objects. See "Arguments and Return Values" for specifics.

We used javah to create the header file for the native side. Take a look at the function signature created by javah.

struct Hjava_lang_String *UserInformation_Username(struct HUserInformation *);

By examining this piece by piece, you can see how the Java to C linkage works. First, the return type is declared as struct Hjava_lang_String *. In the native language, all objects are accessed through a handle-a special structures that allows access to an object's instance variables and the class's methods (the object's vtable.) You can translate Hjava_lang_String directly to "java.lang.String". Handles are always passed as pointers for efficiency.

As you can see, the function name itself is comprised of the class name, an underscore, and the method name itself. If you wanted to add a new native method to find out the user's disk quota, it would be named something like UserInformation_DiskQuota. Because javah can generate all of the function declarations, there is rarely a need to create these names, but because the names are not "mangled" (the way early C++ to C translators did), the resulting code is very readable.

The final item in this signature is the argument struct HUserInformation *. It is an automatic parameter, which will be explained in the next section.

Arguments and Return Values

Pure Java methods can pass arguments to native methods just as they would to any other method call. Native methods can return any value that a Java method would. How does this work when two such vastly different languages are involved? It works because the Java Virtual Machine exposes some of its internals to the native language. At the same time, the header files that support native methods use features of the C preprocessor and compiler to shield you, the developer, from the guts of the Java implementation. Essentially, when you are developing a native method, you are hooking right into the most fundamental aspects of Java's implementation. As a result, you must be careful to follow the rules, or you risk writing fragile code and upsetting the JVM itself.

Simple data types map directly from Java to C. Some difficulties arise because arrays are first class objects in Java, whereas they are aggregate data types in C. Arrays appear in C code as Java objects. Table 19.1 shows the data type mapping from Java to C.

The most important argument for any native method is the first parameter passed to all native methods. It is referred to as an automatic parameter. Look back at the example earlier in this chapter. The function UserInformation_Username was passed a struct HUserInformation * as the first (and only) argument. This pointer serves the same function as the implicit this pointer in C++. That is, the first parameter passed to a native method points to the object instance for which that method is being called. You can use this pointer to access the instance variables of the class. With it, you can even call other methods of the object. This automatic parameter takes the form of a handle. (See the sidebar "What are Handles, Anyway?".)

The confluence of handles, multithreading, and garbage collection leads to some interesting consequences for your C code. First of all, your C code must be fully re-entrant. Because a Java application can have an arbitrary number of threads, any one of those threads can call your native method-even if another thread is already executing in that method, unless you synchronize the method. See "Multithreading and Native Methods." Also, remember that the garbage collection runs in an idle thread, but another thread may request that the garbage collector run immediately.

How does this affect your native methods? The garbage collector may relocate objects in memory! As long as you maintain a reference to your arguments, particularly your this pointer, it will not be relocated. However, if you try to store the obj pointer from the object handle (for example, hUserInformation->obj) in a global variable, two very bad things will happen. First, the garbage collector may relocate or even destroy the object. This is because each object has a reference count-a count of the outstanding handles to that object. When the object's reference count hits zero, it is put into a pool that can be garbage collected at any time. When your method returned, the handle passed to it went out of scope and the object's reference count was decremented. Although you still have a copy of the handle, the object may get destroyed without warning. For instance, another thread may cause the reference count to hit zero, ven though your variable still has a pointer. Then your variable points to deallocated memory. The second problem with this scenario comes from the multithreading itself. Any function that uses global data is suspect in a multithreaded program. It is very difficult to prove that a function using global data will behave correctly when faced with re-entrant calls. Here again, the most likely consequence is an invalid pointer. The JVM is well protected, but it can still be crashed. Never believe anyone who tells you that it is impossible to get a core dump or GPF from Java! Anytime native methods are involved, the JVM is only as robust as those native methods.

In Table 19.1, all of the Java array types appear as handles in the object instances. Java treats arrays as first-class types and implements them as objects. When passing an array to a native method, the JVM translates it into a C-like array. However, the array instance variables are still (Java) arrays when accessed from C. Therefore, the C code must treat it as an object.

Returning primitive and complex data types works exactly the same way as primitive and complex arguments. Sometimes, as in the UserInformation example, this may involve constructing new Java objects in C code. The next section explains how to accomplish this feat.

Using Java Objects from Native Methods

In native methods, all objects appear as handles. Handles are at the heart of the Java object model, but you will be shielded from directly manipulating handles. A group of C macros and functions permit you to deal with handles largely as an abstract type. So what can you do with a handle?

Accessing Instance Variables from Native Methods

For starters, you can access all of an object's instance variables-all those that are visible to your code, at least. In order for your native method to access the data members of your object, you dereference the handle with the macro unhand(). Unhand() is defined in the header file interpreter.h, which is included by StubPreamble.h. Listing 19.8 shows an example of accessing an object's instance variables from a native method.

Listing 19.8. Using unhand() to access instance variables.

typedef struct ClassUserInformation {
   long iUserID;
} ClassUserInformation;
HandleTo(UserInformation);
...
void UserInformation_SetUserID(struct HUserInformation *hUI,
    long newValue)
{
   unhand(hUI)->iUserID = newValue;
}

The unhand() macro returns a pointer to an ordinary C structure. (For handles to objects of your class, it returns a pointer to the structure defined in your header file.) In all cases, the members of the C structure have the same names as the members of the Java class. The types may be different, however. Check Table 19.1 for the type mapping. You can read and write to the members of the structure. In fact, you can even pass pointers to them as arguments to other functions. They behave in all ways as members of ordinary C structures, because that is exactly what they are. When you use unhand() on a handle, you are actually looking at the fundamental Java implementation of an object. The structures you see using the handle are the real structures; there is no behind-the-scenes marshalling before the call.

Accessing Class Variables from Native Methods

Class variables (static variables) do not appear in the instance variable structure generated by javah. This is because they are not stored with each instance of a class, but rather with the class structure itself. The Java runtime provides a function that you can use to get a pointer to the class variable getclassvariable.

long *getclassvariable(struct ClassClass *cb, char *fname);

Notice that this function, like many other Java runtime functions, requires a pointer to the class structure. Fortunately, every object instance carries this pointer around. The obj_classblock macro from interpreter.h takes an object handle as an argument and returns a pointer to the class structure.

Why does getclassvariable return a pointer to a long? It returns a pointer because the native language code should be able to modify the class variables, as well as see them. The return type is long, but in C, all pointers to data are the same size. So, it is safe to cast the returned pointer to a pointer to whatever type the class variable really is.

Be warned that unless your native methods are synchronized, modifying class variables can be problematic. Always remember that, even though this method is in C, it really is being called from a multithreaded environment. There may be other threads executing Java methods or native methods. They may be doing any number of things, including accessing the class variable at exactly the same time as the native method. Any time global data is involved, whether it is class-wide or truly global, synchronization is an issue. See "Multithreading and Native Methods" for more details.

Wouldn't it be nice if your native language code could also call Java code? Yes, it really would. Fortunately, we can do exactly that. It all starts with the object handle. (Where else?) You have to use a slightly more complicated mechanism than you might think at first. Calling another C function from a function pointer in a structure is something familiar to all of us, something like:

hUserInformation->Username();                 // DO NOT DO THIS!

Although this would have the net effect of transferring execution to the other function, it is not sufficient. Here's why: When calling a C function, a stack frame is created with the return address and some arguments. (Implementations vary somewhat; this is necessarily a very general description.) In Java, there is much more context provided for a function call. In addition to the processor stack, the JVM maintains a great deal of stack information on its own, internal stack. (Presumably, combining this stack with the processor stack, both in hardware, is one of the benefits of a "Java chip.")

In order to make the method call, we really have to ask the JVM to do the call for us. Here are the three functions declared in interpreter.h that will make these calls:

HObject *execute_java_constructor(ExecEnv *, char *classname,
   ClassClass *cb, char *signature, ...);
long execute_java_dynamic_method(ExecEnv *, HObject *obj,
   char *method_name, char *signature, ...);
long execute_java_static_method(ExecEnv *, ClassClass *cb,
   char *method_name, char *signature, ...);

Each function serves a specific purpose and is described separately in the following section.

Calling a Java Constructor

Native language code can construct an instance of any Java class that is currently loaded. Here again, the native code is hooking directly into the implementation of Java. Therefore, the new object will behave exactly as it would if you had constructed it in Java code. The general form of a native language constructor is this:

hNewObject = (HClassName *)execute_java_constructor(NULL,
    "ClassName", NULL, "ConstructorSignature", ...);

The first parameter here is the ExecEnv or exception environment that applies to the constructor call. You can safely pass a NULL here, and the constructor will operate with the default exception environment. If you are using sophisticated exception handling in your native code, please see the section "Native Methods and Exceptions."

The interesting pieces of this call are shown in italics. First of all, in order for the native method to use the object that gets constructed, it must know the object's class. Usually, this comes from including a header file for that class. Only a few classes have headers distributed with the JDK: ClassLoader, String, Thread, and ThreadGroup. However, by using javah, you can create header files for whatever classes you need. (See Chapter 14 for details.) If you do not need to access the object's methods or instance variables, you can cast the return value from execute_java_constructor to an HObject pointer. HObject is the handle to the base class Object.

The next item of interest in this call is the class name. This is a text string that specifies what class is to be instantiated. This class must already be loaded, or reside on the class path. When you pass a class name, the Java runtime must perform a lookup on the class itself. However, if you have a pointer to the class itself, you can leave the class name argument NULL and pass the ClassClass pointer in the third argument. For example, if you wanted to clone an existing object of any class, you might use code like this:

struct HObject *Cloner_Clone(struct HCloner *hCloner,
                             struct HObject *hObj)
{
   HObject *hNewInst;
   hNewInst = execute_java_constructor(NULL, NULL,
                 obj_classblock(hObject), "(A)", hObj);
   return hNewInst;
}

This native method can clone any object that has a copy constructor. The macro obj_classblock is another of the handle convenience macros. For any object, it will return a pointer to the class structure for that object. Many of the built-in functions of the Java runtime require a pointer to the class structure "ClassClass".

The constructor signature is used to select which constructor to invoke. It is a character string that specifies the number and type of arguments to follow. Table 19.2 shows the possible characters and their meanings.

By concatenating characters from this set, you specify what arguments are to follow. Two characters have special significance: the parentheses indicate the beginning and end of the argument list. For a constructor, parentheses should enclose the entire argument list. For other methods, the signature will also indicate the return type of the method. In the preceding example, the signature is "(A)", which means the constructor must simply take one argument-an object instance. This implies that our constructor could receive objects of other classes as arguments. If we knew the class name, say Foo, for example, we could write the signature as "(LFoo;)", which means the constructor must take an instance of class Foo or its subclasses, as an argument. Here is a revised version of the Clone method, which takes into account the class name of the argument passed in:

struct HObject *Cloner_Clone(struct HCloner *hCloner,
                             struct HObject *hObj)
{
   HObject *hNewInst;
   char     signature[80];

   sprintf(signature, "(L%s;)", classname(obj_classblock(hObj)));
   hNewInst = execute_java_constructor(NULL, NULL,
                 obj_classblock(hObject), signature, hObj);
   return hNewInst;
}

Notice that, although "name" is a simple member of struct ClassClass, we use the classname macro (from oobj.h) instead of a direct access. In general, you should never access a structure member directly. Instead, use one of the macros provided in oobj.h or interpreter.h (both included by StubPreamble.h). The macros shield you from future changes to the structures. What if there is no macro to access a particular member? Then you probably are not meant to use that member in the first place!

After the constructor signature, you can place a variable length argument list. These are the arguments that will be passed to your constructor. You should be careful to make sure that the type, number, and order of your arguments match the signature. Otherwise, your constructor may be called with garbage for arguments.

If there is an error while calling the constructor, execute_java_constructor will return NULL.

Calling a Java Method

Now for the really good part. You can call any Java method from native code. It all works through the handle to the object instance. If the method to be called is static, see the next section. For dynamic methods-including final and native methods-use the execute_java_dynamic_method function:

long execute_java_dynamic_method(ExecEnv *, HObject *obj,
    char *method_name, char *signature, ...);

The first parameter is the same as the first argument to execute_java_constructor-an exception environment. Again, it is safe to pass a NULL for this parameter. If you require more sophisticated exception handling for the native code, see "Native Methods and Exceptions."

The second parameter is the object instance itself. This can be any valid object handle, whether it was passed from Java code as an argument or constructed in the native method itself.

The next parameter is the name of the method to be invoked. If this method does not exist in the object passed, execute_java_dynamic_method will throw an exception.

Finally, the fourth argument is the signature of the instance method. Again, the signature indicates the type and number of arguments to be passed to the instance method being called. This must be the same number and type of remaining arguments in the call to execute_java_dynamic_method.

The signature for a method differs slightly from the signature for a constructor. A method signature needs to indicate the expected return type. The signature can show this with an extra character after the closing parenthesis. For example, consider a method that takes arguments of two classes and a float and returns a byte:

public byte[] FunkyMethod(Object fc, SecondClass sc,
                          float difference);

This method would have the signature "(ALSecondClass;F)B". The call to execute_java_dynamic_method would then have three arguments after the signature: a generic object, an instance of SecondClass, and a float.

The return value from execute_java_dynamic_method depends on the method being invoked. It is declared as long, however, so you will need to cast it to the appropriate type. (Because long is wide enough to hold any primitive type or a handle, it is safe to cast it to whatever the method really returns.) Be careful, though. Because you are calling the method by a function, not directly, the compiler cannot possibly notify you of any changes in the method definition.

Calling a Static Java Method

Calling a class method from native code is similar to calling a dynamic method. Use the execute_java_static_method function:

long execute_java_static_method(ExecEnv *, ClassClass *cb,
   char *method_name, char *signature, ...);

This is entirely analogous to the execute_java_dynamic_method function, with one crucial difference. Instead of taking an object instance as a parameter, execute_java_static_method requires a class structure. This class structure can come from an object instance, using the obj_classblock macro, or it can be retrieved using the FindClass function:

ClassClass* FindClass(ExecEnv *ee, char *classname, bool_t resolve);

FindClass will return a pointer to the class structure for any class name you pass in. This function may cause several things to happen. First, the named class may be loaded, which will in turn cause the named class's static block to execute. Exceptions may be thrown by FindClass itself, if it cannot find the class, or by the static block of the named class, when it is loaded.

Like the other runtime functions that can throw exceptions, FindClass can take an exception environment as an argument. As usual, it is safe to pass a NULL here, in which case FindClass will use the current exception environment.

Multithreading and Native Methods

Even though the native methods are written in C, they still execute within the context of the Java environment, including multithreading. Native methods will sometimes have sections of code that modify global (class or application) data, which modify important state variables, or which must call non-re-entrant functions. (Some platform-specific APIs fall into this category.) These sections are called critical sections, because it is critical that no more than one thread executes the section of code at a time. Critical sections are not unique to native methods: the same issues exist when dealing with multithreading in pure Java code.

In order to ensure that the native methods maintain the application in a consistent state, these native methods will have to be synchronized with other threads. The simplest way to accomplish this is to declare the native methods as synchronized in the Java code. Sometimes, however, this will be insufficient, either for performance reasons (for example, a method that does a long computation, but only changes the condition variable infrequently), or because a multithreaded application needs to use an existing object that does not have synchronized methods (most do not).

In these cases, your native code can perform the synchronization directly. In Java code, you could put the critical section inside a synchronized block. The native method analogue to a synchronized block directly uses a monitor.

Monitors prevent two threads from simultaneously executing a section of code. Monitors were first introduced in C.A.R. Hoare's seminal paper "Communicating Sequential Processes" (Communications of the ACM, Vol. 21, No. 8, August 1978). Typically, each condition variable has a monitor associated with it. The monitor acts as a lock on that data. Unless a thread holds the monitor, it cannot modify or inspect that data. Obviously, a thread should hold the monitor as briefly as possible.

Monitors provide the Java runtime system support for thread synchronization. Every object instance has a unique monitor attached to it. In this case, the entire object is considered a condition variable. Through a trio of functions, native methods can also use monitors.

void MonitorWait(unsigned int, int);
void MonitorNotify(unsigned int);
void MonitorNotifyAll(unsigned int);

These functions are analogous to the wait(), notify(), and notifyAll() functions in Java:

The Java implementation of monitors has a nice feature that helps to prevent deadlock. Consider this scenario: a synchronized native method calls monitorWait, to wait for another synchronized method (native or not) to notify the monitor. Because the first method is synchronized, it already holds the monitor when monitorWait is called. How is the second method ever supposed to execute? The implementation of monitorWait makes this possible by releasing the monitor on entry. This allows the second method to execute and notify the monitor. Once the monitor is notified, the first thread awakens. On exit from monitorWait, the first thread acquires the monitor again.

All three of the monitor functions are declared as receiving an unsigned integer. To get the monitor for an object, use the obj_monitor macro on the object's handle.

Here is an example of a synchronized string. This is an example of a producer/consumer problem. (Most synchronization problems can be reduced to one of two classic examples, producer/consumer and the colorfully named "dining philosophers problem." The dining philosophers problem is discussed in Chapter 20, "Working with Threads.") In a producer/consumer problem, one thread produces data items while another consumes them. Both operations must be synchronized so that the consumer gets each item exactly once. Listing 19.9 contains the Java definition of the synchronized string class, SyncString. Listing 19.10 shows the native method implementations of the SyncString.Set and SyncString.Get methods. Note once again that this could easily have been accomplished without the use of native methods. You will find that native methods are only useful in a very small set of circumstances.

Listing 19.9. Java class for SyncString.

class SyncString {
    String str;
    int bAvail;

    public native synchronized void Set(String s);
    public native synchronized String Get();

    static {
       try {
           System.loadLibrary("SyncString");
       } catch (UnsatisfiedLinkError e) {
           System.err.println("Cannot find library SyncString.");
           System.exit(-1);
       }
    }
}

Listing 19.10. Native language implementation for SyncString.

#include <StubPreamble.h>
#include "SyncString.h"


void SyncString_Set(struct HSyncString *this,
                    struct Hjava_lang_String *str)
{
    while(unhand(this)->bAvail == TRUE)
        monitorNotify(obj_monitor(this));

    unhand(this)->bAvail = TRUE;
    unhand(this)->str = str;

    monitorNotify(obj_monitor(this));
}

struct Hjava_lang_String *SyncString_Get(struct HSyncString *this)
{
    Hjava_lang_String *hstrTemp;

    while(unhand(this)->bAvail == FALSE)
        monitorWait(obj_monitor(this));

    unhand(this)->bAvail = FALSE;
        hstrTemp = unhand(this)->str;

    monitorNotify(obj_monitor(this));

    return hstrTemp;
}

Native Methods and Exceptions

Native methods can throw exceptions just like a regular Java method. As with any other method that throws an exception, the native method in the Java class must declare the exceptions that it can throw. This looks exactly like the throws clause for any other method.

Your native code throws the actual exception using the SignalError function:

void SignalError(ExecEnv *ee, char *Exception, char *SourceHint);

The first argument is the exception environment to use for the exception. Generally, you will pass a NULL here to indicate the current environment.

The second argument is the fully specified exception name. This name includes the package in which the exception is defined. It should also be specified as a path, replacing the usual period separators with forward slashes (that is, UNIX-style path separators).

The third argument is a source code hint. You can pass a string that describes the error or provides some useful details. This string will appear on the walkback if the exception is not handled. It is safe to pass a NULL here.

Using Java Strings from Native Methods

Because there is an "impedance mismatch" between Java strings and C strings, there are several convenience functions that allow C code to use the Java strings. Java strings offer several advantages over C strings, including reference counting, automatic memory management, and Unicode support. On the other hand, no C runtime library function or API function will expect a Java string. The utility functions provide a bridge between Java and C strings.

These functions are defined in javaString.h, which will be included automatically if you include StubPreamble.h:

• The following function will print a Java string on stdout:
  void javaStringPrint(Hjava_lang_String *)
• The following function returns the length of the Java string, in characters:
  int javaStringLength(Hjava_lang_String *)
• The following function will construct a new Java string from a given C string. The second argument indicates the length of the buffer passed in the first argument:
  Hjava_lang_String *makeJavaString(char *, int)
• The following interesting function allocates some temporary memory for a C string initialized from the given Java string. Once all references to this storage are eliminated, the memory is automatically deallocated. In other words, as long as you keep this pointer in a variable, the memory is valid:
  char *makeCString(Hjava_lang_String *)
• The following function uses "malloc" to allocate some memory for a C string initialized from the given Java string:
  char *mallocCString(Hjava_lang_String *)
• The following function will convert the characters of the given Java string into a unicode string. It assumes that the buffer passed in as the second argument already exists. The third argument specifies the buffer length. The function returns the unicode string's address:
  unicode *javaString2unicode(Hjava_lang_String *, unicode*, int)
• The following function will convert the characters of the given Java string into a C string. It assumes that the buffer passed in as the second argument already exists. The third argument specifies the buffer length. The function returns the C string's address:
  char *javaString2Cstring(Hjava_lang_String *, char *, int)

Summary

Native methods provide a powerful means of extending the capabilities of the Java runtime environment. Although native methods are inherently platform-specific, there are certain cases where they might be used. By tapping directly into the implementation of the Java object model, the native language functions can access instance and class variables, construct objects, and call static and dynamic methods. Native methods can be synchronized like ordinary Java methods. They can also throw exceptions. To bridge the gap between Java strings and C strings, the Java runtime environment provides a number of utility functions.

The steps to constructing native methods are as follows:

1. Define the class interface.
2. Write the Java classes.
3. Compile the Java classes.
4. Use javah to create the header and stub files.
5. Write the implementation file.
6. Compile the C files into a dynamically loadable library.
7. Install the dynamically loadable library on the target platform.