I’ve been analyzing Java remoting protocols at work over the past couple of days, and thought I’d share some of the results here. Specifically, I’m going to share the results of our protocol benchmarking.
Every application will have different requirements in this area, but most criteria will include performance. At the very least, you will want to know how much (if any) performance you are sacrificing for the sake of your other requirements.
The Protocols
The protocols under consideration are Java’s RMI/JRMP, Oracle’s ORMI (with and without HTTP tunneling enabled), Spring’s HttpInvoker, Caucho’s Hessian and Burlap, and three flavors of Apache XML-RPC (Sun-based, HttpClient-based and Lite-based).
RMI/JRMP is Java’s default binary remoting protocol, and its big advantage is that it supports full object serialization of Serializable and Externalizable objects. RMI/JRMP has historically not been as easy to use as some of the other options, but if you’re using Spring (as we are), it’s as easy to use as anything else (except Apache XML-RPC; see below).
ORMI is OC4J’s EJB remoting protocol, which we’re currently using in production. This protocol was originally developed as part of the Orion application server, whose codebase formed the basis for OC4J. The only differentiator between ORMI and RMI/JRMP is likely to be performance. Of course, this protocol probably won’t be an option unless your serverside component is deployed on OC4J. ORMI supports tunneling via HTTP and HTTPS, should you run into firewall or proxy problems.
Spring’s HttpInvoker is another Java-to-Java binary remoting protocol. It’s also very easy to use and supports full object serialization of Serializable and Externalizable objects. The big difference between HttpInvoker and RMI/JRMP is that HttpInvoker transports data via HTTP, which makes it easier to use when you start running into proxy and firewall issues. Unfortunately, HttpInvoker isn’t an option if you’re not using Spring on both the server and the client.
Caucho’s Hessian protocol is a slim, binary cross-platform remoting protocol. Most cross-platform protocols are XML-based, and thus sacrifice a significant amount of performance in order to achieve interoperability. Hessian’s draw is that it achieves cross-platform interoperability with minimal performance degradation. However, Hessian uses a custom reflection-based serialization mechanism which can be troublesome in certain scenarios (think Hibernate proxies).
Currently in a draft state, Hessian 2 is the second incarnation of the Hessian protocol. You probably don’t want to use it in your production applications quite yet, but it looks very promising. Of course, it’s so easy to switch between Hessian and Hessian 2 that you might want to have a peek just for the heck of it [1].
As far as I can tell, Caucho’s Burlap is essentially Hessian-in-XML. As such, Burlap is not a binary protocol and should be expected to suffer accordingly in the performance department. This being the case, I haven’t been able to figure out exactly why anyone would choose to use Burlap instead of Hessian… aside from a requirement from the marketing department that your RPC be buzzword compliant.
Apache XML-RPC is Apache’s implementation of XML-RPC, an XML-based RPC specification which focuses on interoperability. Apache XML-RPC may be a bit more difficult to set up than the other options if you’re using Spring. The infrastructure for the other protocols is built into the Spring framework, but you’ll have to write the Apache XML-RPC / Spring integration yourself.
Apache XML-RPC supports a number of transport factories under the covers. The three transport factories tested were the default transport factory (based on Sun’s HttpURLConnection), the HttpClient transport factory (based on HttpClient) and the Lite transport factory (based on an Apache XML-RPC minimal HTTP client implementation).
The Configuration
The test consisted of remote invocations to a single method which takes an integer size parameter and returns a list of the specified size. The objects in the returned list each contained 10 instance variables (4 strings of about 40 characters each, 2 dates, 1 long, 1 integer, 1 double and 1 float, none of which were null). The lists returned were static, so there was no list construction overhead.
The results of the first 10 invocations were thrown away, in order to allow the VMs to “warm up.” The next 100 invocations were tallied and averaged.
The tests were performed on a Windows XP SP2 machine with 2GB of RAM and a 2Ghz Core 2 Duo CPU, using Sun’s JVM 1.5.0_09. The client and server were run in two separate VMs on the same host, in order to avoid spurious networking artifacts [2].
The following library versions were used: Spring 2.0.6, Jetty 6.1.5, OC4J 10.1.3.2.0, slf4j 1.4.3, log4j 1.2.14, Apache XML-RPC 3.1, Commons HttpClient 3.1, and Caucho Hessian 3.1.3.
Spring was used on both the clientside and on the serverside in order to hide the remoting details from the interface consumer and implementor. All serverside components were run inside a single Jetty servlet container, except for the ORMI serverside testing components which were run in OC4J.
Vendor extensions and GZIP requesting were enabled for all Apache XML-RPC tests. Streaming and GZIP compressing were enabled for all Apache XML-RPC tests except the Lite tests (the Lite transport factory does not support these options). These configurations represent the best performance optimizations available to the various Apache XML-RPC flavors.
The Results
The following graph illustrates the response times of the various protocols for invocations returning smaller lists:
The following graph illustrates the response times of the various protocols for invocations returning larger lists:
Conclusion
It’s important to note that no general conclusions can be derived from the absolute numbers represented above. Rather, the numbers must be examined relative to each other.
That said, the following observations may be made:
The binary protocols (RMI, ORMI, HttpInvoker and Hessian) are always faster than the XML-based protocols (Burlap and the Apache XML-RPC variants) — except for ORMI with HTTP tunneling enabled. Performance is pretty even amongst the binary protocols — except for Hessian, which performs well only when compared to the XML-based protocols, and ORMI with HTTP tunneling enabled, which performs on a par with the XML-RPC variants. Burlap has much better performance than the XML-RPC variants. Native RMI and Hessian 2 have the best performance until the remote method invocations start returning larger lists, at which point vanilla ORMI takes a slight lead. Changing Apache XML-RPC’s transport factory does not seem to have a very large effect on performance. It’s amazing how fast standard ORMI is, compared to ORMI with HTTP tunneling enabled. Hessian 2 bears watching!
Other Interesting Links
JBoss Remoting Framework Benchmarks: Tom Elrod’s benchmark of the JBoss Remoting framework. Includes part of the Spring Remoting framework.
Nominet’s Protocol Benchmarks: Interesting benchmark of many of the protocols here considered. Intriguingly, HttpInvoker is found to have consistently better performance than RMI/JRMP, a finding which contradicts our results.
Footnotes
[1] If you’re using Spring’s Hessian integration and the latest Hessian JARs, moving to Hessian 2 is as easy as adding the following property to your HessianProxyFactoryBean:
<property name="proxyFactory">
<bean class="com.caucho.hessian.client.HessianProxyFactory">
<property name="hessian2Request" value="true" />
<property name="hessian2Reply" value="true" />
</bean>
</property>
[2] The tests were later run on two separate machines over a controlled network, in order to verify that the local tests are representative; they are.
