A reusable solution for the conversion between JSON and XML

Two popular information exchange notations/languages are Json and XML. In order to provide interoperability between frameworks using different mechanisms of information exchange, a conversion between the two is required. There is however not a strict way to provide a conversion; http://jackson-users.ning.com/forum/topics/xml-to-json-conversion-using.

I've seen several solutions in practice to provide the conversion for example;
- using a service bus to make the translation from XML to JSON and the other way around
- using a Java HttpServlet with JAX-B to provide a mapping

There are also other solutions such as embedding a Java JSON client in the BPEL code; http://technology.amis.nl/2009/12/15/the-oracle-soa-suite-11g-httpbinding-or-another-way-to-call-restful-services-from-soa-composite-applications/

Those solutions however require work per service and are thus not very reusable. When the service landscape consists of a lot of services, this creates an extra mapping layer which needs maintenance.

A reusable solution

JSON is structured (http://www.w3resource.com/JSON/structures.php; objects, arrays and values). The conversion from JSON to XML (and the other way around) can be fixed and a single message definition (XSD) can be provided.

In order to provide a single reusable conversion from JSON to XML and XML to JSON, an XML schema is required which is capable of containing JSON structures. Luckily, the internet is a big place and I could find several examples of such XSD's such as; http://xml.calldei.com/JsonXML. This way, only a single set of JAX-B classes can be used and only two pieces of mapping code to make the transformation work. If you have a different Json message (sending or receiving), it can be mapped to the same XML so no additional programming is required for individual services.

Based on the schema, I used JAX-B to generate Java classes. Then I used Jackson Json processor to create Json (http://jackson.codehaus.org/). I wrote a mapping from the JAX-B objects to JsonNodes and the other way around. Next I exposed this as a webservice so it could for example easily be called from BPEL.

Implementation

I managed to create a reversible transformation from Json to a fixed Xml schema and the other way around. Also I've provided a test method to check if the conversion works (the following two conversions lead to the original result; JSON -> XML -> JSON) for specific Json messages. Keep in mind that JSON strings need to be enclosed in ". This allows Json to be used in XML based systems for request and response handling. Also, because the schema is fixed, it can easily be used in XML based systems to build JSON messages.

I've used recursive methods to walk the XML and JSON trees respectively and convert them to the other type. The code is not thoroughly tested so should not be used carelessly in production environments.

The example webservice can be downloaded here (JDeveloper 11.1.1.6 project); https://dl.dropbox.com/u/6693935/blog/JsonXml.zip

Conclusion

When combined the above with the SocketAdapter functionality as described in http://javaoraclesoa.blogspot.nl/2012/12/receiving-json-requests-in-oracle-bpel.html the Hello World JSON BPEL process is not farfetched anymore. I didn't manage to complete this yet however. I've spend some time on getting the SocketAdapter to work with the XSL (request/reply) transformations. I came to the conclusion that usage of the socket XSLT functions is hard. One of the issues I encountered is when the server should start sending back a reply. Also getting the body from the HTTP message could be better programmed out in Java (using the Content-Length HTTP header to obtain the correct body).

The other way around; converting an XML to a fixed Json message and back again to obtain the original result, is more difficult. An example of a method of converting XML to Json can be found on; http://javaoraclesoa.blogspot.nl/2012/07/webinterface-restjson-vs-middleware.html. This transformation is however not reversible.

Receiving JSON requests in Oracle BPEL by using the SocketAdapter

Some frameworks prefer the use of JSON, such as webinterface frameworks, GIS frameworks or mobile app frameworks. In order to integrate and provide for example service orchestration, Oracle SOA Suite can be used. Oracle SOA Suite however does not have a JSONAdapter in the current release.

Using the HTTPAdapter is not an option to provide JSON support. The Oracle HTTPAdapter does not support receiving and sending JSON (only XML). It can however be used to send a HTTP GET request with parameters which is how REST services are often called. See for example; http://shrikworld.blogspot.nl/2011/04/http-adapter-in-soa-11g.html

The SocketAdapter however does not have the XML limitations of the HTTPAdapter. In this post I describe how the SocketAdapter can be used to receive a JSON message from an HTTP Post request.

Implementation

SocketAdapter

To use the  SocketAdapter for this usecase requires some parsing of the HTTP communication which is usually abstracted. It is possible to use XSLT or a custom Java class to parse the request received from the adapter and transform it to XML (http://docs.oracle.com/cd/E21764_01/integration.1111/e10231/adptr_sock.htm#BABHECAG). I decided to try the XSLT option.

I downloaded an example from; http://java.net/projects/oraclesoasuite11g/pages/SocketAdapters

In this example I needed to change the following to get it to work;
- update adf-config.xml with the correct path of the jdeveloper/integration folder
- make sure bpm-infra.jar is in the classpath of the server

Based on the configuration of the SocketAdapter connection factory I tried connecting to port 12110 with telnet.

In the logfile I saw the following;

- Socket Adapter ClientProcessor:run() Error occured in processing client request
- Socket Stylesheet Parsing Error.
Error while trying to parse the stylesheet.
Please ensure that that the stylesheet exists and is configured properly. Please cross check the extension functions. Contact Oracle support if error is not fixable. 

This was as expected since I didn't provide the correct test message.

When I opened in my browser; http://localhost:12110 it did work however. I noticed the input was;

0.9,*/*

I wondered where this came from. I created a custom Java socket listener based on http://docs.oracle.com/javase/tutorial/networking/sockets/readingWriting.html and http://zerioh.tripod.com/ressources/sockets.html and did the request on the listener. The result was;

client>GET / HTTP/1.1
client>Host: localhost:2004
client>User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:16.0) Gecko/20100101 Firefox/16.0
client>Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
client>Accept-Language: en-US,en;q=0.5
client>Accept-Encoding: gzip, deflate
client>Connection: keep-alive

The request.xsl from the example was as followed;

<?xml version="1.0" encoding="windows-1252" ?>
<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
                xmlns:ora="http://www.oracle.com/XSL/Transform/java/"
                xmlns:socket="http://www.oracle.com/XSL/Transform/java/oracle.tip.adapter.socket.ProtocolTranslator"
                xmlns="http://xmlns.oracle.com/HelloWorld">
 <!-- Root template -->
 <xsl:template match="//input">
  <!-- Input coming from the browser would look like
  GET /input=sagar; HTTP/1.1.
  dummy variable to read the input= text -->
  <xsl:variable name="variable1"
                select="socket:socketRead('terminated', 'terminatedBy==')"/>
               
  <!-- Read the input string terminated by ;and build the input xml -->
  <HelloWorldProcessRequest>
   <input>
    <xsl:value-of select="socket:socketRead('terminated', 'terminatedBy=;')"/>
   </input>
  </HelloWorldProcessRequest>
 </xsl:template>
</xsl:stylesheet>


The SocketAdapter took the part from the first = sign to the first ; after that. This was confirmed by looking at the log set to trace level.

When trying again with telnet with the input;
input=Maarten;

It worked and I got an instance of my process. I was however not yet able to receive JSON from a HTTP POST request. First I needed an example request;

POST /request HTTP/1.1
Accept: application/jsonrequest
Content-Length: 123
Content-Type: application/jsonrequest
Host: localhost

{"user":"test","var":7,"t":"something","stuff":123}

I needed to extract everything after the first empty line (I'm not sure yet if this is always the case in HTTP messages!) to the end of the message. This way if the JSON message was multiline, it would also work. To do that I needed to understand the XPath function socket:socketRead used in the XSLT transformation used by the SocketAdapter when receiving the message

I found the following (http://docs.oracle.com/cd/E21764_01/integration.1111/e10231/adptr_sock.htm#BABHECAG);

"By using StyleReader, which is exposed by the NXSD framework, to read and write from the socket stream using the following methods: socketRead(nxsdStyle:String, nxsdStyleAttributes:String):String"

I remembered from the Native Format Builder (which also uses the NXSD framework) some codes I could use in the terminatedBy clause. The resulting XSLT for my usecase was;

<?xml version="1.0" encoding="windows-1252" ?>
<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
                xmlns:ora="http://www.oracle.com/XSL/Transform/java/"
                xmlns:socket="http://www.oracle.com/XSL/Transform/java/oracle.tip.adapter.socket.ProtocolTranslator"
                xmlns="http://xmlns.oracle.com/HelloWorld">
 <!-- Root template -->
 <xsl:template match="//input">
  <!-- Input coming from the browser would look like
  GET /input=sagar; HTTP/1.1.
  dummy variable to read the input= text -->
  <xsl:variable name="variable1"
                select="socket:socketRead('terminated', 'terminatedBy=${eol}${eol}')"/>
              
  <!-- Read the input string terminated by ;and build the input xml -->
  <HelloWorldProcessRequest>
   <input>
    <xsl:value-of select="socket:socketRead('terminated', 'terminatedBy=${eof}')"/>
   </input>
  </HelloWorldProcessRequest>
 </xsl:template>
</xsl:stylesheet>

With this XSLT to parse the request I could obtain the JSON String from a HTTP POST request. This is the first step in creating a BPEL JSON HelloWorld service.

JsonPath

The parsing or querying of a JSON String can be done in several ways such as using the JsonPath (http://code.google.com/p/json-path/). Below is code for an example webservice which can be used.

package ms.testapp.soa.utils;

import com.jayway.jsonpath.JsonPath;

import javax.jws.WebParam;
import javax.jws.WebResult;
import javax.jws.WebService;

@WebService
public class JsonPathUtils {
   
    @WebResult(name = "result")
    public String ExecuteJsonPath(@WebParam(name = "jsonstring") String jsonstring, @WebParam(name = "jsonpath")String jsonpath) {
        String result = JsonPath.read(jsonstring, jsonpath).toString();
        return result;
    }
   
    public JsonPathUtils() {
        super();
    }
/*   
    public static void main(String[] args) {
        JsonPathUtils myPath = new JsonPathUtils();
        System.out.println(myPath.ExecuteJsonPath("{ \"store\": {\n" +
        "    \"book\": [ \n" +
        "      { \"category\": \"reference\",\n" +
        "        \"author\": \"Nigel Rees\",\n" +
        "        \"title\": \"Sayings of the Century\",\n" +
        "        \"price\": 8.95\n" +
        "      },\n" +
        "      { \"category\": \"fiction\",\n" +
        "        \"author\": \"Evelyn Waugh\",\n" +
        "        \"title\": \"Sword of Honour\",\n" +
        "        \"price\": 12.99,\n" +
        "        \"isbn\": \"0-553-21311-3\"\n" +
        "      }\n" +
        "    ],\n" +
        "    \"bicycle\": {\n" +
        "      \"color\": \"red\",\n" +
        "      \"price\": 19.95\n" +
        "    }\n" +
        "  }\n" +
        "}", "$.store.book[1].author"));
    }
*/
}

Conclusion

I haven't finished the HelloWorld BPEL JSON service implementation yet. What I've found however is that it requires some work to get relatively simple functionality.

Not only does it require some work, but to make the code reusuable is also a challenge. You will need routing mechanisms which sounds a lot like where the OSB is really good at.
JSON is often used when performance is important. JSON messages can be smaller then XML messages and a strict message or interface definition (such as for XML XSD/WSDL) is not a requirement. Also the (Java) frameworks required to work with JSON are often relatively easy to implement making quick development possible. When using the above solution, several components are used which cause additional overhead (such as the SocketAdapter and XSLT transformations). This is expensive when considering there can be a relatively light and easy implementation by using the OSB or custom Java code per integration.