Wednesday, February 20, 2013

Server Push using Atmosphere (Glassfish + PrimeFaces)

Server Push is described as a mechanism in which the server initiates a transaction to a client (http://en.wikipedia.org/wiki/Push_technology). This mechanism has various interesting implementations which improve the options for interaction on a website such as an online chatbox (http://www.primefaces.org/showcase/push/chat.jsf) in which users are informed of new logins and messages.

There are various technologies which implement mechanisms to achieve the experience of server push such as;

- Native Comet (The WebServer has API for Comet)
- Native WebSockets (The WebServer has API for WebSocket)
- WebSockets
- Long-Polling
- Http Streaming
- JSONP
- Server-Sent Events

I will not go into detail for each of these options.

Atmosphere

Which technology is a wise choice to follow and implement? Every method has it's own drawbacks. I myself have considered implementing a custom Long-polling mechanism in the past. WebSockets is new and gaining popularity ('hip' in dutch ;), however a lot of browsers do not natively support it yet. See a list on; http://stackoverflow.com/questions/1253683/what-browsers-support-html5-websocket-api. When WebSockets is not supported you want the option to fall back on more primitive options. This would however require implementing different technologies and thus more work. Luckily there are frameworks which provide an abstraction to the technology based on what the server and client support. Atmosphere is one of those frameworks; https://github.com/Atmosphere/atmosphere. Atmosphere provides a wide array of support options for server and clientside of the server push mechanism; https://github.com/Atmosphere/atmosphere/wiki/Supported-WebServers-and-Browsers. Also when using Atmosphere, future technologies will most likely not require more rework then an Atmosphere upgrade.

Implementation

I wanted to create a website which provided chat support and some other options. I didn't want to have to click a refresh button to update my screen but I wanted events triggered by clients to lead to updates; I wanted a server push mechanism. I chose my development environment to be Netbeans, my server to be Glassfish and my JSF framework to be PrimeFaces (see http://javaoraclesoa.blogspot.nl/2013/02/a-simple-j2ee-webapplication-netbeans.html). Glassfish has good support options for server push mechanisms; all the mentioned technologies can be used with the latest version of Glassfish. PrimeFaces natively supports Atmosphere (https://github.com/Atmosphere/atmosphere/wiki/Atmosphere-PlugIns-and-Extensions).

Glassfish Comet support

At first I got the following errors in Glassfish;

SEVERE: AtmosphereFramework exception
java.lang.IllegalStateException: Make sure you have enabled Comet or make sure the Thread invoking that method is the same as the Servlet.service() Thread.
at com.sun.grizzly.comet.CometContext.addCometHandler(CometContext.java:295)
at com.sun.grizzly.comet.CometContext.addCometHandler(CometContext.java:314)
at org.atmosphere.container.GrizzlyCometSupport.suspend(GrizzlyCometSupport.java:139)
at org.atmosphere.container.GrizzlyCometSupport.service(GrizzlyCometSupport.java:121)
at org.atmosphere.container.GlassFishWebSocketSupport.service(GlassFishWebSocketSupport.java:70)

My first guess whas that I needed to enable Comet support. I looked it up and found the following;

When I activated it and restarted the server however, the setting was lost. I found the following bug; http://markmail.org/message/3fo3tyo72pew4jv4. The Comet support setting was not always persisted. I found some other options to try; using asadmin to add the option (http://blog.eisele.net/2012/09/primefaces-push-with-atmosphere-on.html) and manually editing the domain.xml. Both didn't fix my problem. My Glassfish server was a Netbeans embedded version. Then I found the following checkbox and the problem was fixed.


PrimeFaces implementation

All the code required was the following in the JSF file;

<p:socket onMessage="handleMessage" channel="/messages" />
<p:remoteCommand name="updateMessagesTable" actionListener="#{messageBean.loadMessages}" update="MessagesDataTable"/>
            
<script type="text/javascript">
                function handleMessage(data) {
                    updateMessagesTable();
                }
</script>

The following in the web.xml file;

    <servlet>
        <servlet-name>Push Servlet</servlet-name>
        <servlet-class>org.primefaces.push.PushServlet</servlet-class>
    </servlet>
    <servlet-mapping>
        <servlet-name>Push Servlet</servlet-name>
        <url-pattern>/primepush/*</url-pattern>
    </servlet-mapping>

And the following in the backing bean of the JSF page;

    private void doPushUpdate() {
        PushContext pushContext = PushContextFactory.getDefault().getPushContext();
        pushContext.push("/messages", "update");
    }

How does this work?

In the messageBean, when the doPushUpdate function is called, all clients receive asynchronous updates on the specified channel. This triggers them to update the messages table.

<p:socket onMessage="handleMessage" channel="/messages" /> in the JSF page responds to a message on the channel /messages. It calls the Javascript function handleMessage when an asynchronous server push event is received. The line; <p:remoteCommand name="updateMessagesTable" actionListener="#{messageBean.loadMessages}" update="MessagesDataTable"/> makes available a Javascript function called updateMessagesTable() which calls the messageBean function loadMessages after which it updates my MessagesDataTable. The Javascript function handleMessage calls the updateMessagesTable function and in effect the messageBean function. The handleMessage function is not required since I could have called the updateMessagesTable function directly, but I wanted a location at which I could do something in Javascript with the received message.

How does it look?

Keep in mind that this is my first play with server push and PrimeFaces. I was pretty satisfied with the result and the relative ease at which it was achieved. After every message all clients get updated. If you want to know what PrimeFaces is capable of, check out their impressive showcase (http://www.primefaces.org/showcase).


What more could you want?

I could have dynamically updated the table or other portions of the site instead of refreshing it completely. Also I have not implemented a lazy loading mechanism; http://www.primefaces.org/showcase/ui/datatableLazy.jsf

The potential usages of server push mechanisms are numerous. Server push allows communication with a server as mediator. For example it can be implemented to allow direct communication with website visitors or allows visitors to immediately interact with each other. It also allows external events (happening on the server) to influence what visitors get to see on your site on the fly.

The code can be downloaded here

Tuesday, February 12, 2013

A simple J2EE webapplication; Netbeans + PrimeFaces + JPA

Webapplications are not my area of expertise, but once in a while I like to try some things to see how easy it would be to get something done. This time I wanted a webinterface which would be able to get/display data from a database and put stuff back in. I wanted a solution which would be quick to develop and stable in an enterprise environment. I chose to use open source software for this. I am however impatient and want quick results.

Development environment

I was looking for a development environment which would allow me to be productive; had good support for webdevelopment (wizards for generating code for me) and Maven (for easy dependency management). I started out with Eclipse. I followed the following tutorial to get it installed in Ubuntu; https://help.ubuntu.com/community/EclipseIDE. Ubuntu comes packaged with an old Eclipse version so to get the newest version working, some things needed to be done. After that I installed the Maven support and the WebToolKit (WTK) plugin. After much fiddling with the plugins, I decided to quit trying to get this to work; I couldn't get it working quickly enough and I would probably encounter other issues along the way when continuing on this path.

My second choice for IDE was Netbeans. I installed the latest version and Maven, JSF integration, Glassfish integration worked out of the box without any work. I was pleased with Netbeans.

Front-end framework

For the front-end I decided on a JSF implementation which is seen quite often nowadays. I took a look at the following comparison of JSF frameworks; http://www.mastertheboss.com/richfaces/primefaces-vs-richfaces-vs-icefaces. Based on the comparison and my bad experiences with Richfaces and server push, I decided to go with Primefaces (try something new). It had jQuery code integrated which I also liked (see for example; http://blog.primefaces.org/?p=2226). PrimeFaces has a large set of ready to use examples; http://www.primefaces.org/showcase/ui/home.jsf. This would save me a lot of work.

Back-end integration

For the database I wanted to have as little work as possible. I don't like to program things like relations, constraints, data structures, etc in the database; I was developing a web-application and the data-model, implemented as Java entities, should be sufficient definition. I chose to use the Java Persistence API (JPA) in combination with the pre-installed/configured Derby database. For the JPA implementation I chose EclipseLink since it was selected by default. Hibernate was also one of the options. I didn't do any research on the best choice for me since I wanted to quickly start developing and this worked out of the box.

Developing

The EntityManager and EntityManagerFactory

I read several tutorials and went developing. I started out with different tutorials such as  http://hendrosteven.wordpress.com/2008/03/06/simple-jpa-application-with-netbeans/ and http://www.objectdb.com/tutorial/jpa/netbeans/web. In these tutorials I encountered two very interesting and much discussed classes; EntityManagerFactory and EntityManager. These two classes make it possible to link the data model entities to the persistent storage.

Then it became a little bit more complicated. Where should I do things like starting a transaction? What should be the scope of my transaction? How should I manage the lifecycle of the EntityManager and the EntityManagerFactory? Since the transaction code could be reused among entities I decided not to put it in my entity classes. I first created an EntityUtils class (replace entity with a specific entity name) which would do entity management such as starting transactions, deleting, creating entities, performing queries on the entities. Then I found the following list of patterns; http://blog.xebia.com/2009/07/13/jpa-implementation-patterns-wrap-up/. My EntityUtils class was actually a DAO! (Data Access Object). I refactored my EntityUtils class and created a common (using type-variables) abstract class for DAO's. I still had to manually manage the transactions though.

Then I found the following tutorial;http://netbeans.org/kb/docs/javaee/javaee-gettingstarted.html. I could generate almost all the code I had previously written. Manual transaction management was not required. The DAO was however called ServiceFacade. Most likely because of the different role the class has when comparing for example Sevlet implementations to JSF implementations. Based on this last tutorial I could finish my webapplication and had an efficient mechanism to generate most of the code required to use the database from my JSF application.

I have not provided my application as an example for download since the tutorial mentioned would provide a better description. Also I will need to expand the application a little further to convince myself I'm developing efficient reusable stable code.

Conclusion

These conclusions are based on my personal experiences.

- use Netbeans instead of Eclipse in you want more functionality (being not specific here on purpose) to work out of the box
- checkout different frameworks for your JSF implementation. PrimeFaces has a lot of easily usable online examples and jQuery integration
- start a research on the EntityManager and EntityManagerFactory. using them efficiently in a Servlet differs from an efficient implementation in a JSF page.
- there is a lot of information available. getting to the relevant parts required effort

The following document contains a lot of information on J2EE; http://docs.oracle.com/javaee/6/tutorial/doc/javaeetutorial6.pdf. To get something working quickly however http://netbeans.org/kb/docs/javaee/javaee-gettingstarted.html is recommended.

Monday, February 4, 2013

Comparing Weblogic filestore backed JMS with Oracle AQ backed JMS

In this blog post I will look at two methods of creating JMS queues with durable subscribers in Oracle Weblogic Server 11g. One backed by a persistent file store and the other backed by an Oracle AQ implementation. I will compare several properties of these implementations such as configuration required, performance and maintaining subscribers. I'll describe the issues I encountered during the setup and how I've solved them. I've performed the tests on the Oracle supplied Virtualbox image; http://www.oracle.com/technetwork/middleware/soasuite/learnmore/vmsoa-172279.html

Configuration

First I'll shortly describe the different implementations used. Then I'll show how the queues can be tested using SOAP UI.

AQ implementation

To setup the JMS implementation which is backed by an Oracle AQ, the following steps need to be performed (which are described in more detail on; http://docs.oracle.com/cd/E21764_01/integration.1111/e10231/adptr_jms.htm#BABBBGGB (in 8.4.8));

In the database
- create user
- grant user required rights
- create multi consumer queue table
- create queue using queue table
- start created queue

In the Weblogic server
- create datasource
- create JMSModule
- create foreign server inside JMSModule
- configure foreign server for using datasource and specify queue (destination)
- create connection factory inside foreign server
- create durable subscriptions (using PL/SQL scripts)

Filestore implementation

This is described on; http://middlewaremagic.com/weblogic/?p=4403

The steps to perform are in short;
- create a FileStore
- create JMSServer
- create JMSModule
- create connection factory inside JMSModule
- create subdeployment inside JMSModule
- create topic using FileStore and target to subdeployment
- create durable subscriptions (using Weblogic console)

Using SOAP UI to test the implementations

This is described on; http://learnwithpavan.wordpress.com/2012/02/06/hermesjms-configuration-for-weblogic-11g/. Below are some pointers and screenshots to help get this working.

HermesJMS

First configure HermesJMS; add weblogic.jar (from <MIDDLEWARE_HOME>\wlserver_10.3\server\lib) to the HermesJMS claspath in the hermes.bat file.

Next under the providers tab when creating a session, add the weblogic.jar again.


Then restart HermesJMS and add Weblogic as loader and specify the properties.

To be able to use this configuration in SOAP UI, check the tutorial on; http://www.soapui.org/JMS/working-with-jms-messages.html

To configure in Hermes the two implementations, in the binding you need to specify the connection factory. For the Oracle AQ implementation this is defined as Remote JNDI name under the foreign server in the created JMSModule.


For the Filestore implementation, this is defined directly under the JMSModule  as JNDI name. This is of course if you followed the mentioned descriptions.


I encountered in HermesJMS the following error;
weblogic hermesjms javax.jms.JMSException: Could not create InitialContext: t3://127.0.0.1:7001: Bootstrap to 127.0.0.1/127.0.0.1:7001 failed. It is likely that the remote side declared peer gone on this JVM

This was caused by a network connection which could not be correctly established in my local setup. To avoid similar issues (VM running locally, DHCP server and suspend/resume actions of the VM, misconfigured hosts file, bridged VM network adapter, Windows firewall, etc...), I decided to install SOAP UI inside the VM which worked fine.

I found that the binding setting in which the connectionfactory was specified, did not cause different results when running Discover in HermesJMS (see below).


SOAP UI

In SOAP UI you can add a JMS endpoint to an existing service by right-clicking the binding and selecting 'Add JMS endpoint'. Here you can use the HermesJMS configuration which you have created earlier.

SOAP UI does not need to have weblogic.jar in the classpath. It determines the classpath required based on the HermesJMS config.


After the endpoint is added, it can be tested the same way as a regular webservice. You can check if the service can publish and receive to/from the created topic.


The above part is for the Filestore JMS. When trying the AQ JMS however I got the following exception;

weblogic.jms.common.InvalidDestinationException: [JMSClientExceptions:055090]Foreign destination

This was caused by setting the wrong connection factory in the HermesJMS configuration. I changed the binding value to aqjms/testForeignConnectionFactory (the connection factory defined in the foreign server configuration). Then I encountered various other errors which apparently were already encountered by others (http://jianmingli.com/wp/?p=2950). Also I later found that Oracle has provided a working, well documented, example of this; http://www.oracle.com/technetwork/middleware/weblogic/aq-jms-demo-171733.zip. Checking the documentation in the example is worthwhile!

At first I got the following exception; java.lang.UnsupportedOperationException: Remote JDBC disabled

I fixed it as followed; locate setDomainEnv.sh and set WLS_JDBC_REMOTE_ENABLED to true and restart the server (see for example; https://forums.oracle.com/forums/thread.jspa?threadID=989569).

The next error I encountered was;

Caused by: java.rmi.MarshalException: error marshalling return; nested exception is:
    java.io.NotSerializableException: oracle.jdbc.driver.T4CConnection
    at weblogic.rjvm.ResponseImpl.unmarshalReturn(ResponseImpl.java:237)
    at weblogic.rmi.internal.BasicRemoteRef.invoke(BasicRemoteRef.java:223)
    ... 19 more
Caused by: java.io.NotSerializableException: oracle.jdbc.driver.T4CConnection
    at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1164)
    at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:330)

This is fixed by setting the JDBC properties not using JNDI lookup in the foreign server to obtain the datasource.


Then I was able to send a message in HermesJMS. The next error I encountered is in SOAP UI;

Fri Feb 01 06:36:08 PST 2013:ERROR:oracle.jms.AQjmsException: ORA-24047: invalid agent name durableSubscriptionaqjms/TestX, agent name should be of the form NAME
ORA-06512: at "SYS.DBMS_AQADM_SYS", line 6270
ORA-06512: at line 1
ORA-06512: at "SYS.DBMS_AQJMS", line 129
ORA-06512: at line 1

I decided to add a subscriber in the database;

DECLARE
   subscriber          sys.aq$_agent;
BEGIN
   subscriber := sys.aq$_agent('SUBSCRIBER', null, null);
   DBMS_AQADM.ADD_SUBSCRIBER(
      queue_name         => 'JMSTOPIC',
      subscriber         =>  subscriber);
END;
/
COMMIT;

If you forget to commit, the below part will make SOAP UI hang!

Specify the subscriber while enqueueing;


Finally it worked... Now I could compare and test the different implementations.

Results

There are of course a lot of properties which can be compared. I choose performance, management of durable subscribers and monitoring of messages. I didn't look into it in much detail but it should give some idea of major differences between the implementations.

Performance

AQ JMS

Load test in SOAP UI. First I tested the AQ JMS implementation.

With 50 threads; avg response; 449.82ms, failure rate; 90.17%


With one thread; avg response; 76.05ms, failure rate; 0%


More then one thread lead to slow responses and a high error rate. I don't believe the failure rate is something which can't be fixed with some tweaking of settings but I haven't looked into it.

Filestore JMS

I should be honest here. I've set the Client ID policy to unrestricted and the Subscription Sharing Policy to Sharable. See below.


With 50 threads; avg response 58.79ms, failure rate; 0.018%


With one thread; avg response 20.37ms, failure rate; 0%


Maintaining the subscribers

When using AQ JMS, the durable subscribers can be maintained by using the Oracle Enterprise Manager and by using PL/SQL scripts. When using the Filestore implementation, the durable subscribers can be maintained from the Weblogic console and by using WLST.

AQ JMS
Support from the Oracle Enterprise manager; http://docs.oracle.com/cd/B28359_01/server.111/b28420/manage.htm#i1005928

Adding subscribers can also be achieved by scripting them with PL/SQL;

Adding;
DECLARE
   subscriber          sys.aq$_agent;
BEGIN
   subscriber := sys.aq$_agent('SUBSCRIBER', null, null);
   DBMS_AQADM.ADD_SUBSCRIBER(
      queue_name         => 'JMSTOPIC',
      subscriber         =>  subscriber);
END;

/

Removing;
DECLARE
   subscriber       sys.aq$_agent;
BEGIN
   subscriber := sys.aq$_agent ('SUBSCRIBER', null, null);
   DBMS_AQADM.REMOVE_SUBSCRIBER(
      queue_name => 'JMSTOPIC',
      subscriber => subscriber);
END;
/


Filestore JMS
Weblogic interface

This can also be scripted by using WLST e.g.  http://www.javamonamour.org/2011/09/wlst-deleting-durable-subscribers.html

Checking messages

AQ JMS
AQ JMS messages can be checked by looking at the queue tables and viewes;


Also the queue behavior can be altered using for example SQLDeveloper (http://www.oracle.com/technetwork/developer-tools/sql-developer/overview/index.html).

Filestore JMS
Messages can be viewed from the Weblogic console. The behavior of the queue can also be altered from the console.
Conclusion

Initial configuration required
AQ JMS requires configuration in the Oracle database and in the Weblogic console. A filestore JMS implementation requires only (and less) configuration in the weblogic server. Configuration of testing tools required more work (mainly due to the errors encountered) in SOAP UI /HermesJMS for the AQ JMS implementation then for the Filestore JMS implementation. If you know how exactly you have to configure SOAPUI/HermesJMS, the work required on the client side is similar.

Performance
The performance of the Filestore JMS implementation was significantly better (about 7 times) when using 50 threads. When using one thread, the performance of the Filestore JMS implementation was about 3 times better. The failure rate for the AQ JMS implementation when using 50 threads was very high. This is likely due to a configuration error or wrong test case.

Subscribers and managing messages
Managing subscribers is similar for both implementations in complexity, GUI support and scripting options. Messages can be managed similarly in both implementations. I've not looked into error handling and re-enqueueing options.