Webinterface REST/JSON vs Middleware SOAP/XML

Introduction

JavaScript

Webdevelopers use a lot of JavaScript on the clientside. There are things which JavaScript is good at and there are things you're better of not doing in JavaScript. JSON and REST are often used by JavaScript developers and XML and SOAP are shunned.

JSON

JSON (see http://www.json.org/), the JavaScript Object Notation, is an easy lightweight notation which can be used to transfer JavaScript objects; objects can be transported as strings. JSON is a lot easier for JavaScript then for example XML. JSON has some query languages (such as XQuery / XPATH for XML) which are still work in progress but seem to work to some extend; http://stackoverflow.com/questions/777455/is-there-a-query-language-for-json. For Java there are libraries available to make working with JSON more easy (such as JSON-lib and Jackson JSON).

REST

Since webinterfaces nowadays often require asynchronous interaction with the server to for example validate values in forms without requiring a submit of the entire form, services on the server are often deployed.

For service interaction, JavaScript can easily do HTTP POST and GET requests but it is harder to create entire SOAP messages. An example on how asynchronous HTTP requests can be done from JavaScript; http://rest.elkstein.org/2008/02/using-rest-in-javascript.html

REST services (http://en.wikipedia.org/wiki/Representational_state_transfer) are an alternative to SOAP services. REST services often do not have an interface defined (WSDL and WADL are disputable for REST services); there is often less strict message exchange (if you want to validate messages or throw exceptions, you have to do it yourself, they are application specific and not inherent to the protocol). For the sake of quick (webinterface) development, less strict standards can be a choice.

Oracle SOA and webinterfaces

This is an Oracle SOA / Java blog, so where do those fit in? Well, Middleware often has to link in some way to the Frontend or Frontend oriented backend systems. The Oracle SOA Middleware is heavily XML/SOAP based and the Frontend likes JSON and REST services a lot better. This is a gap which needs to be bridged.

SOAP and REST services

Oracle supplies an HTTP binding adapter which can be used to send and receive HTTP requests. See http://docs.oracle.com/cd/E25054_01/dev.1111/e10224/sca_bindingcomps.htm#autoId3 for more information. This adapter allows sending/receiving HTTP GET and POST requests and is capable of receiving XML from REST services. The Spring component can also be used for REST service interaction and since it is custom Java code which is used, it can more easily be expanded with additional functionality; http://technology.amis.nl/2009/12/16/soa-suite-11g-using-spring-component-to-mimic-http-binding-and-integrate-restful-services/

JSON and XML

Java

When examining the differences between JSON and XML, one major problem arises; there can be no strict fixed conversion between the two. See for example the below article on a user forum of one of the better known/performing JSON libraries (Jackson JSON); http://jackson-users.ning.com/forum/topics/xml-to-json-conversion-using; JSON and XML information models are fundamentally incompatible. Jackson JSON is good at converting JSON to Java objects and the other way around. Converting to XML however will most likely not be implemented because of the difference in information models.

JSON-lib (http://json-lib.sourceforge.net/) however does allow a conversion of JSON to XML and the other way around (although not reversible) with relatively little code. Of course certain assumptions need to be made to allow the conversion to work. If you're interested, you can read more about this on; http://www.xml.com/pub/a/2006/05/31/converting-between-xml-and-json.html

XSLT

Going from XML to JSON is specific and can be modeled using XSLT to be able to support different conversion patterns. See http://controlfreak.net/xml-to-json-in-xslt-a-toolkit/ for some useful examples.

Conversion JSON to XML and XML to JSON webservice

I wanted to be able to easily convert JSON from the webinterface and REST services (Java code) to XML in order to be more flexible in BPEL with for example XPATH and XSLT. When I have XML, I can use XPATH to create the relevant JSON again if the default transformation is insufficient. I created a webservice based on JSON-lib to do the conversion. I used JAX WS to make developing/exposing the methods as a webservice easy. The code required is the following;

package ms.testapp.jsonxml;

import java.io.InputStream;
import java.io.InputStreamReader;

import javax.jws.WebMethod;
import javax.jws.WebParam;
import javax.jws.WebResult;
import javax.jws.WebService;

import net.sf.json.JSON;
import net.sf.json.JSONArray;
import net.sf.json.JSONObject;
import net.sf.json.JSONSerializer;
import net.sf.json.xml.XMLSerializer;

import org.apache.commons.lang.StringEscapeUtils;

@WebService
public class JsonXml {
   
    @WebMethod
    @WebResult(name = "xml")
    public String JsonToXml(@WebParam(name = "namespace")String namespace, @WebParam(name = "rootelement")String rootelement,@WebParam(name = "jsonstring")String jsonStr) {
            XMLSerializer serializer = new XMLSerializer();
            serializer.setRootName(rootelement);
            serializer.addNamespace("", namespace);
            JSON json = JSONSerializer.toJSON(jsonStr);
            String xml = serializer.write(json);
           
            return xml;
    }

    @WebMethod
    @WebResult(name = "json")
    public String EscapedXmlToJson(@WebParam(name = "xmlstring")String xmlStr) {
        XMLSerializer xmlSerializer = new XMLSerializer();
        JSON json = xmlSerializer.read(StringEscapeUtils.unescapeXml(xmlStr)); 
        return( json.toString() ); 
    }
   
    @WebMethod(exclude=true)
    public String XmlToJson(@WebParam(name = "xmlstring")String xmlStr) {
        XMLSerializer xmlSerializer = new XMLSerializer(); 
        JSON json = xmlSerializer.read( xmlStr ); 
        return( json.toString() ); 
    }
/*
    public static void main(String[] args) {
        String Json = "{'foo':'bar',\n" +
        " 'coolness':2.0,\n" +
        " 'altitude':39000,\n" +
        " 'pilot':{'firstName':'Buzz',\n" +
        "          'lastName':'Aldrin'},\n" +
        " 'mission':'apollo 11'}";
        String Xml = new JsonXml().JsonToXml("http://www.google.com","root",Json);
        System.out.println(Json);
        System.out.println(Xml);
        Json = new JsonXml().XmlToJson(Xml);
        System.out.println(Json);
        String escapedXml = StringEscapeUtils.escapeXml(Xml);
        System.out.println(escapedXml);
        Json = new JsonXml().EscapedXmlToJson(escapedXml);
        System.out.println(Json);
       
    }
*/
}

The full code is available here; https://dl.dropbox.com/u/6693935/blog/JsonXmlApp.zip. There is also a JAR deployment profile (in addition to the WAR deployment profile) in the package and the dependencies are included. For Java developers, it is recommended (for performance reasons to avoid webservice overhead) to use the Jar deployment profile. From BPEL (in a service oriented landscape) it is more easy to have the functionality available as a webservice. The main method has been used as a sort of unittest.

Please mind, the conversion is not reversible and specific.I personally think the JSON to XML conversion is more useful then the XML to JSON conversion since that can be achieved more specifically with XSLT. I have not exposed the XmlToJson method since it's hard to call the method as a webservice with an XML parameter containing a non-escaped XML fragment.

The method of converting JSON to XML, has the benefit of not requiring an XSD schema describing the XML for the conversion and not requiring a Java object representation of the data. This has the benefit that one service/class is enough to do the conversions different messages (at least JSON to XML).

Further suggestions; JSON and BPEL

When you want to use the XML created by converting JSON in BPEL, you need to do the following;
- create a schema for the created XML message and include it in a used WSDL file
- create a variable using this schema
- use ora:parseEscapedXML to assign the output of the JsonToXml to the variable
- use the variable however you like

Because this can be cumbersome (creating the schema from the XML message, JDeveloper has a wizard but still...), it's advisable to create another webservice with two input variables; JSON string and JSON path expression returning the result of the JSON path expression on the JSON string. This can easily be accomplished by using for example http://code.google.com/p/json-path/. This way if you want to use a single variable from the JSON string, you can query for it directly and you don't need to create an XSD schema for the result of the JSON to XML conversion since it is a simple string. Use can use something like; String result = JsonPath.read(jsonstring, jsonpathstring).toString(). This is however not recommended if you need a lot of variables from the JSON string since calling a webservice is also overhead and might cost performance.

Packaging the above JSON samples in an XPATH library (see for example http://docs.oracle.com/cd/E23943_01/dev.1111/e10224/bp_appx_functs.htm) might be a better solution then wrapping them in webservices since it further increases their ease of use and performance in BPEL. If the services need to be re-used outside of BPEL by for example other middleware products, webservices might be the better solution.

Best method of using database sequences from BPEL; native sequencing or Tapi?

Introduction

In this example I will create a simple BPEL process; a Data Service, which will insert a record in a table and return the Id of the created entry. I will compare two methods to achieve this

1 TAPI (Table API) + before insert trigger for Id generation

2 Direct insert from BPEL DbAdapter and using native sequencing for Id generation

I will (in general terms) compare both methods on the amount of developer time required to achieve the method and the maintainability. I will also do a performance test to compare both methods.

I've used the following VM for this test; http://www.oracle.com/technetwork/middleware/soasuite/learnmore/vmsoa-172279.html. I've created a database user testuser. In the Weblogic console I've created a JDBC datasource and ConnectionFactory. Also I've created two simple synchronous BPEL processes which have the same activities to make sure the overhead for the processes is equal. These processes can be downloaded here; https://dl.dropbox.com/u/6693935/blog/TestSequencing.zip

Implementation

TAPI + before insert trigger

I created a simple table, sequence and a before insert trigger;

CREATE SEQUENCE "TESTUSER"."TESTTEXT_SEQ" MINVALUE 1 INCREMENT BY 1 START WITH 1;
/
CREATE TABLE "TESTUSER"."TESTTEXT"
  (
    "ID"   NUMBER NOT NULL ENABLE,
    "TEXT" VARCHAR2(256 BYTE),
    CONSTRAINT "TESTTEXT_PK" PRIMARY KEY ("ID") ENABLE
  );
 /
CREATE OR REPLACE TRIGGER "TESTUSER"."BI_TESTTEXT" BEFORE
  INSERT ON TESTTEXT FOR EACH ROW BEGIN IF (:NEW.ID IS NULL) THEN
  SELECT TESTTEXT_SEQ.NEXTVAL INTO :NEW.ID FROM DUAL;
END IF;
END;
/
ALTER TRIGGER "TESTUSER"."BI_TESTTEXT" ENABLE;

I created a TAPI for the table as followed;

The result was;

CREATE OR REPLACE
PACKAGE TESTTEXT_tapi
IS
type TESTTEXT_tapi_rec
IS
  record
  (
    TEXT TESTTEXT.TEXT%type ,
    ID TESTTEXT.ID%type );
type TESTTEXT_tapi_tab
IS
  TABLE OF TESTTEXT_tapi_rec;
  -- insert
PROCEDURE ins(
    p_TEXT IN TESTTEXT.TEXT%type DEFAULT NULL ,
    p_ID   IN TESTTEXT.ID%type );
  -- update
PROCEDURE upd(
    p_TEXT IN TESTTEXT.TEXT%type DEFAULT NULL ,
    p_ID   IN TESTTEXT.ID%type );
  -- delete
PROCEDURE del(
    p_ID IN TESTTEXT.ID%type );
END TESTTEXT_tapi;
/
CREATE OR REPLACE
PACKAGE body TESTTEXT_tapi
IS
  -- insert
PROCEDURE ins(
    p_TEXT IN TESTTEXT.TEXT%type DEFAULT NULL ,
    p_ID   IN TESTTEXT.ID%type )
IS
BEGIN
  INSERT INTO TESTTEXT
    ( TEXT ,ID
    ) VALUES
    ( p_TEXT ,p_ID
    );
END;
-- update
PROCEDURE upd
  (
    p_TEXT IN TESTTEXT.TEXT%type DEFAULT NULL ,
    p_ID   IN TESTTEXT.ID%type
  )
IS
BEGIN
  UPDATE TESTTEXT SET TEXT = p_TEXT WHERE ID = p_ID;
END;
-- del
PROCEDURE del(
    p_ID IN TESTTEXT.ID%type )
IS
BEGIN
  DELETE FROM TESTTEXT WHERE ID = p_ID;
END;
END TESTTEXT_tapi;

This TAPI was insufficient for my test since there was no link to the sequence used and no procedure/function returning the Id. It would be nice if Oracle could enhance this feature a little in the future to make it more flexible. SQL Developer however should not become a new 'Headstart'! (remember this product from the Designer days?)

For the purpose of this test I've updated the insert procedure to;

PROCEDURE ins(
    p_TEXT IN TESTTEXT.TEXT%type DEFAULT NULL ,
    p_ID   IN OUT TESTTEXT.ID%type )
IS
BEGIN
  INSERT INTO TESTTEXT
    ( TEXT ,ID
    ) VALUES
    ( p_TEXT ,p_ID
    ) returning id into p_ID;
END;

Direct insert from BPEL DbAdapter and using native sequencing

In order to use native sequencing, you have to specify the sequence in the DbAdapter wizard in JDeveloper;

Also when you want to use native sequencing, the sequencePreallicationSize (DbAdapter, connectionfactory setting) needs to be equal to the increment of the sequence;

If you don't do this, you will encounter the following error;

java.lang.Exception: oracle.sysman.emSDK.webservices.wsdlapi.SoapTestException: Client received SOAP Fault from server : Exception occured when binding was invoked. Exception occured during invocation of JCA binding: "JCA Binding execute of Reference operation 'insert' failed due to: DBWriteInteractionSpec Execute Failed Exception. insert failed. Descriptor name: [insert_DB.Testtext]. Caused by Exception [EclipseLink-7027] (Eclipse Persistence Services - 2.3.1.v20111018-r10243): org.eclipse.persistence.exceptions.ValidationException Exception Description: The sequence named [TESTTEXT_SEQ] is setup incorrectly. Its increment does not match its pre-allocation size.. Please see the logs for the full DBAdapter logging output prior to this exception. This exception is considered not retriable, likely due to a modelling mistake. ". The invoked JCA adapter raised a resource exception. Please examine the above error message carefully to determine a resolution

Mind that when this error occurs, the sequence is already increased however no entry is saved in the table!

Performance test

To make the test more representative, I've disabled the database trigger for the native sequencing test. After each test, I've truncated the table and done drastic cleaning on the dehydration store as described on; http://www.emarcel.com/soa-suit/152-deleteinstancessoasuite11gwls. I've done a SOAP UI loadtest with the following settings; 5 threads, no spread/random, one message every 100 ms per thread and a run for 60 seconds.

Results native sequencing

Results TAPI + before insert trigger

Conclusion

Performance

As can be seen from the performance test, using a TAPI is significantly faster then using native sequencing. More requests can be processed in the same time and the average response time is less.

Maintainability

By using a TAPI, the database table and the BPEL process are not tightly coupled. This increases flexibility. For example, it would be easy to implement logging in the TAPI or handle changes in the table layout without effecting the BPEL process using the TAPI. There is however more (database) code involved. This code needs to be maintained.

Work required

When using a TAPI, the base can be generated and small changes will need to be made in order to make it useful code. When using native sequencing, no database procedures are required; less code is used. The logic concerning the sequence is handled by the DbAdapter instead of database code. Do not forget to set the sequencePreallocationSize property on the applicationserver.

Overall

Using a TAPI provides best performance and flexibility. However it requires more work to create and there will be more code to maintain. Using native sequencing, the amount of database code can be limited; the DbAdapter will handle the sequence part.