maandag 4 februari 2013

Comparing Weblogic filestore backed JMS with Oracle AQ backed JMS

In this blog post I will look at two methods of creating JMS queues with durable subscribers in Oracle Weblogic Server 11g. One backed by a persistent file store and the other backed by an Oracle AQ implementation. I will compare several properties of these implementations such as configuration required, performance and maintaining subscribers. I'll describe the issues I encountered during the setup and how I've solved them. I've performed the tests on the Oracle supplied Virtualbox image; http://www.oracle.com/technetwork/middleware/soasuite/learnmore/vmsoa-172279.html

Configuration

First I'll shortly describe the different implementations used. Then I'll show how the queues can be tested using SOAP UI.

AQ implementation

To setup the JMS implementation which is backed by an Oracle AQ, the following steps need to be performed (which are described in more detail on; http://docs.oracle.com/cd/E21764_01/integration.1111/e10231/adptr_jms.htm#BABBBGGB (in 8.4.8));

In the database
- create user
- grant user required rights
- create multi consumer queue table
- create queue using queue table
- start created queue

In the Weblogic server
- create datasource
- create JMSModule
- create foreign server inside JMSModule
- configure foreign server for using datasource and specify queue (destination)
- create connection factory inside foreign server
- create durable subscriptions (using PL/SQL scripts)

Filestore implementation

This is described on; http://middlewaremagic.com/weblogic/?p=4403

The steps to perform are in short;
- create a FileStore
- create JMSServer
- create JMSModule
- create connection factory inside JMSModule
- create subdeployment inside JMSModule
- create topic using FileStore and target to subdeployment
- create durable subscriptions (using Weblogic console)

Using SOAP UI to test the implementations

This is described on; http://learnwithpavan.wordpress.com/2012/02/06/hermesjms-configuration-for-weblogic-11g/. Below are some pointers and screenshots to help get this working.

HermesJMS

First configure HermesJMS; add weblogic.jar (from <MIDDLEWARE_HOME>\wlserver_10.3\server\lib) to the HermesJMS claspath in the hermes.bat file.

Next under the providers tab when creating a session, add the weblogic.jar again.


Then restart HermesJMS and add Weblogic as loader and specify the properties.

To be able to use this configuration in SOAP UI, check the tutorial on; http://www.soapui.org/JMS/working-with-jms-messages.html

To configure in Hermes the two implementations, in the binding you need to specify the connection factory. For the Oracle AQ implementation this is defined as Remote JNDI name under the foreign server in the created JMSModule.


For the Filestore implementation, this is defined directly under the JMSModule  as JNDI name. This is of course if you followed the mentioned descriptions.


I encountered in HermesJMS the following error;
weblogic hermesjms javax.jms.JMSException: Could not create InitialContext: t3://127.0.0.1:7001: Bootstrap to 127.0.0.1/127.0.0.1:7001 failed. It is likely that the remote side declared peer gone on this JVM

This was caused by a network connection which could not be correctly established in my local setup. To avoid similar issues (VM running locally, DHCP server and suspend/resume actions of the VM, misconfigured hosts file, bridged VM network adapter, Windows firewall, etc...), I decided to install SOAP UI inside the VM which worked fine.

I found that the binding setting in which the connectionfactory was specified, did not cause different results when running Discover in HermesJMS (see below).


SOAP UI

In SOAP UI you can add a JMS endpoint to an existing service by right-clicking the binding and selecting 'Add JMS endpoint'. Here you can use the HermesJMS configuration which you have created earlier.

SOAP UI does not need to have weblogic.jar in the classpath. It determines the classpath required based on the HermesJMS config.


After the endpoint is added, it can be tested the same way as a regular webservice. You can check if the service can publish and receive to/from the created topic.


The above part is for the Filestore JMS. When trying the AQ JMS however I got the following exception;

weblogic.jms.common.InvalidDestinationException: [JMSClientExceptions:055090]Foreign destination

This was caused by setting the wrong connection factory in the HermesJMS configuration. I changed the binding value to aqjms/testForeignConnectionFactory (the connection factory defined in the foreign server configuration). Then I encountered various other errors which apparently were already encountered by others (http://jianmingli.com/wp/?p=2950). Also I later found that Oracle has provided a working, well documented, example of this; http://www.oracle.com/technetwork/middleware/weblogic/aq-jms-demo-171733.zip. Checking the documentation in the example is worthwhile!

At first I got the following exception; java.lang.UnsupportedOperationException: Remote JDBC disabled

I fixed it as followed; locate setDomainEnv.sh and set WLS_JDBC_REMOTE_ENABLED to true and restart the server (see for example; https://forums.oracle.com/forums/thread.jspa?threadID=989569).

The next error I encountered was;

Caused by: java.rmi.MarshalException: error marshalling return; nested exception is:
    java.io.NotSerializableException: oracle.jdbc.driver.T4CConnection
    at weblogic.rjvm.ResponseImpl.unmarshalReturn(ResponseImpl.java:237)
    at weblogic.rmi.internal.BasicRemoteRef.invoke(BasicRemoteRef.java:223)
    ... 19 more
Caused by: java.io.NotSerializableException: oracle.jdbc.driver.T4CConnection
    at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1164)
    at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:330)

This is fixed by setting the JDBC properties not using JNDI lookup in the foreign server to obtain the datasource.


Then I was able to send a message in HermesJMS. The next error I encountered is in SOAP UI;

Fri Feb 01 06:36:08 PST 2013:ERROR:oracle.jms.AQjmsException: ORA-24047: invalid agent name durableSubscriptionaqjms/TestX, agent name should be of the form NAME
ORA-06512: at "SYS.DBMS_AQADM_SYS", line 6270
ORA-06512: at line 1
ORA-06512: at "SYS.DBMS_AQJMS", line 129
ORA-06512: at line 1

I decided to add a subscriber in the database;

DECLARE
   subscriber          sys.aq$_agent;
BEGIN
   subscriber := sys.aq$_agent('SUBSCRIBER', null, null);
   DBMS_AQADM.ADD_SUBSCRIBER(
      queue_name         => 'JMSTOPIC',
      subscriber         =>  subscriber);
END;
/
COMMIT;

If you forget to commit, the below part will make SOAP UI hang!

Specify the subscriber while enqueueing;


Finally it worked... Now I could compare and test the different implementations.

Results

There are of course a lot of properties which can be compared. I choose performance, management of durable subscribers and monitoring of messages. I didn't look into it in much detail but it should give some idea of major differences between the implementations.

Performance

AQ JMS

Load test in SOAP UI. First I tested the AQ JMS implementation.

With 50 threads; avg response; 449.82ms, failure rate; 90.17%


With one thread; avg response; 76.05ms, failure rate; 0%


More then one thread lead to slow responses and a high error rate. I don't believe the failure rate is something which can't be fixed with some tweaking of settings but I haven't looked into it.

Filestore JMS

I should be honest here. I've set the Client ID policy to unrestricted and the Subscription Sharing Policy to Sharable. See below.


With 50 threads; avg response 58.79ms, failure rate; 0.018%


With one thread; avg response 20.37ms, failure rate; 0%


Maintaining the subscribers

When using AQ JMS, the durable subscribers can be maintained by using the Oracle Enterprise Manager and by using PL/SQL scripts. When using the Filestore implementation, the durable subscribers can be maintained from the Weblogic console and by using WLST.

AQ JMS
Support from the Oracle Enterprise manager; http://docs.oracle.com/cd/B28359_01/server.111/b28420/manage.htm#i1005928

Adding subscribers can also be achieved by scripting them with PL/SQL;

Adding;
DECLARE
   subscriber          sys.aq$_agent;
BEGIN
   subscriber := sys.aq$_agent('SUBSCRIBER', null, null);
   DBMS_AQADM.ADD_SUBSCRIBER(
      queue_name         => 'JMSTOPIC',
      subscriber         =>  subscriber);
END;

/

Removing;
DECLARE
   subscriber       sys.aq$_agent;
BEGIN
   subscriber := sys.aq$_agent ('SUBSCRIBER', null, null);
   DBMS_AQADM.REMOVE_SUBSCRIBER(
      queue_name => 'JMSTOPIC',
      subscriber => subscriber);
END;
/


Filestore JMS
Weblogic interface

This can also be scripted by using WLST e.g.  http://www.javamonamour.org/2011/09/wlst-deleting-durable-subscribers.html

Checking messages

AQ JMS
AQ JMS messages can be checked by looking at the queue tables and viewes;


Also the queue behavior can be altered using for example SQLDeveloper (http://www.oracle.com/technetwork/developer-tools/sql-developer/overview/index.html).

Filestore JMS
Messages can be viewed from the Weblogic console. The behavior of the queue can also be altered from the console.
Conclusion

Initial configuration required
AQ JMS requires configuration in the Oracle database and in the Weblogic console. A filestore JMS implementation requires only (and less) configuration in the weblogic server. Configuration of testing tools required more work (mainly due to the errors encountered) in SOAP UI /HermesJMS for the AQ JMS implementation then for the Filestore JMS implementation. If you know how exactly you have to configure SOAPUI/HermesJMS, the work required on the client side is similar.

Performance
The performance of the Filestore JMS implementation was significantly better (about 7 times) when using 50 threads. When using one thread, the performance of the Filestore JMS implementation was about 3 times better. The failure rate for the AQ JMS implementation when using 50 threads was very high. This is likely due to a configuration error or wrong test case.

Subscribers and managing messages
Managing subscribers is similar for both implementations in complexity, GUI support and scripting options. Messages can be managed similarly in both implementations. I've not looked into error handling and re-enqueueing options.