Showing posts with label soap. Show all posts
Showing posts with label soap. Show all posts

Monday, April 25, 2022

Apache NiFi: JSON to SOAP

Apache NiFi is a powerful open source integration product. A challenge you might encounter when integrating systems is that one system can produce JSON messages and the other has a SOAP API available. In this blog post I'll show how you can use NiFi to convert JSON input to a SOAP service call. This involves abstracting an AVRO schema for the JSON, converting it to XML and transforming the XML to a SOAP message. 

In this example I'm using several publicly available websites. You should of course be careful. Do not copy/paste sensitive XML or JSON on these sites!

Friday, January 26, 2018

Automate calls to SOAP and REST webservices using simple Python scripts

Probably not many people will tell you running batches over webservices is a good idea. Sometimes though, it can be handy to have a script available to generate webservice calls based on a template message with variables and automate processing the response messages. In addition, if you have a large number of calls, executing the calls in parallel might save you a lot of time if your service platform can handle the concurrency.

Friday, August 11, 2017

Oracle Mobile Cloud Service integration options

Oracle Mobile Cloud Service has a lot of options which allows it to integrate with other services and systems. Since it runs JavaScript on Node.js for custom APIs, it is very flexible.

Some features allow it to extent its own functionality such as the Firebase configuration option to send notifications to mobile devices, while for example the connectors allow wizard driven integration with other systems. The custom API functionality running on a recent Node.js version ties it all together. In this blog article I'll provide a quick overview and some background of the integration options of MCS.

MCS is very well documented here and there are many YouTube video's available explaining/demonstrating various MCS features here. So if you want to know more, I suggest looking at those.


Thursday, August 11, 2016

Node.js: My first SOAP service

I created a simple HelloWorld SOAP service running on Node.js. Why did I do that? I wanted to try if Node.js was a viable solution to use as middleware layer in an application landscape. Not all clients can call JSON services. SOAP is still very common. If Node.js is to be considered for such a role, it should be possible to host SOAP services on it. My preliminary conclusion is that it is possible to host SOAP services on Node.js but you should carefully consider how you want to do this.

I tried to create the SOAP service in two distinct ways.
  • xml2js. This Node.js module allows transforming XML to JSON and back. The JSON which is created can be used to easily access content with JavaScript. This module is fast and lightweight, but does not provide specific SOAP functionality.
  • soap. This Node.js module provides some abstractions and features which make working with SOAP easier. The module is specifically useful when calling SOAP services (when Node.js is the client). When hosting SOAP services, the means to control the specific response to a call are limited (or undocumented)
Using both modules, I encountered some challenges which I will describe and how (and if) I solved them. You can find my sample code here.


Monday, February 23, 2015

Oracle introduces API Manager!

Oracle has introduced a new product; API Manager (you can find the official documentation here). API Manager is an important addition to the already impressive Oracle SOA stack. In this article I'll explain what this new product does and how it helps in managing your API's. I will focus on the features and benefits you can have of this product and will also elaborate a little about my current experiences with it.


Monday, May 12, 2014

MockServer: Easy mocking of HTTP(S) services (e.g. SOAP or JSON)

Testing services as an atomic entity can be difficult. Especially if these services are part of a call chain or call other services. Often in such cases mock services are developed to reduce test dependencies and exclude services which are not interesting to the specific test case. For example, I'm testing service A. Service A calls service B. I'm not interested in service B (or service B is maintained by another department on which I don't want to depend). I would mock service B when testing service A in this case. There are several methods to create mock services. These methods however are mostly not easily usable by testers since they require developing/coding mock services. Testers would benefit from being able to create their own mock services in order to create different tests for a specific service.

In this blog post I provide a brief introduction and describe some features of MockServer. An open source product which can be used to mock services. For a more detailed article (with more examples) you can look at the following written by my colleague Robert van Mölken: http://technology.amis.nl/2014/03/06/functional-boundary-testing-of-a-service-based-environment-using-mockserver/

The below image was taken from http://www.mock-server.com/

Friday, July 13, 2012

Webinterface REST/JSON vs Middleware SOAP/XML

Introduction

JavaScript

Webdevelopers use a lot of JavaScript on the clientside. There are things which JavaScript is good at and there are things you're better of not doing in JavaScript. JSON and REST are often used by JavaScript developers and XML and SOAP are shunned.

JSON

JSON (see http://www.json.org/), the JavaScript Object Notation, is an easy lightweight notation which can be used to transfer JavaScript objects; objects can be transported as strings. JSON is a lot easier for JavaScript then for example XML. JSON has some query languages (such as XQuery / XPATH for XML) which are still work in progress but seem to work to some extend; http://stackoverflow.com/questions/777455/is-there-a-query-language-for-json. For Java there are libraries available to make working with JSON more easy (such as JSON-lib and Jackson JSON).

REST

Since webinterfaces nowadays often require asynchronous interaction with the server to for example validate values in forms without requiring a submit of the entire form, services on the server are often deployed.

For service interaction, JavaScript can easily do HTTP POST and GET requests but it is harder to create entire SOAP messages. An example on how asynchronous HTTP requests can be done from JavaScript; http://rest.elkstein.org/2008/02/using-rest-in-javascript.html

REST services (http://en.wikipedia.org/wiki/Representational_state_transfer) are an alternative to SOAP services. REST services often do not have an interface defined (WSDL and WADL are disputable for REST services); there is often less strict message exchange (if you want to validate messages or throw exceptions, you have to do it yourself, they are application specific and not inherent to the protocol). For the sake of quick (webinterface) development, less strict standards can be a choice.

Oracle SOA and webinterfaces

This is an Oracle SOA / Java blog, so where do those fit in? Well, Middleware often has to link in some way to the Frontend or Frontend oriented backend systems. The Oracle SOA Middleware is heavily XML/SOAP based and the Frontend likes JSON and REST services a lot better. This is a gap which needs to be bridged.

SOAP and REST services

Oracle supplies an HTTP binding adapter which can be used to send and receive HTTP requests. See http://docs.oracle.com/cd/E25054_01/dev.1111/e10224/sca_bindingcomps.htm#autoId3 for more information. This adapter allows sending/receiving HTTP GET and POST requests and is capable of receiving XML from REST services. The Spring component can also be used for REST service interaction and since it is custom Java code which is used, it can more easily be expanded with additional functionality; http://technology.amis.nl/2009/12/16/soa-suite-11g-using-spring-component-to-mimic-http-binding-and-integrate-restful-services/

JSON and XML

Java

When examining the differences between JSON and XML, one major problem arises; there can be no strict fixed conversion between the two. See for example the below article on a user forum of one of the better known/performing JSON libraries (Jackson JSON); http://jackson-users.ning.com/forum/topics/xml-to-json-conversion-using; JSON and XML information models are fundamentally incompatible. Jackson JSON is good at converting JSON to Java objects and the other way around. Converting to XML however will most likely not be implemented because of the difference in information models.

JSON-lib (http://json-lib.sourceforge.net/) however does allow a conversion of JSON to XML and the other way around (although not reversible) with relatively little code. Of course certain assumptions need to be made to allow the conversion to work. If you're interested, you can read more about this on; http://www.xml.com/pub/a/2006/05/31/converting-between-xml-and-json.html

XSLT

Going from XML to JSON is specific and can be modeled using XSLT to be able to support different conversion patterns. See http://controlfreak.net/xml-to-json-in-xslt-a-toolkit/ for some useful examples.

Conversion JSON to XML and XML to JSON webservice

I wanted to be able to easily convert JSON from the webinterface and REST services (Java code) to XML in order to be more flexible in BPEL with for example XPATH and XSLT. When I have XML, I can use XPATH to create the relevant JSON again if the default transformation is insufficient. I created a webservice based on JSON-lib to do the conversion. I used JAX WS to make developing/exposing the methods as a webservice easy. The code required is the following;

package ms.testapp.jsonxml;

import java.io.InputStream;
import java.io.InputStreamReader;

import javax.jws.WebMethod;
import javax.jws.WebParam;
import javax.jws.WebResult;
import javax.jws.WebService;

import net.sf.json.JSON;
import net.sf.json.JSONArray;
import net.sf.json.JSONObject;
import net.sf.json.JSONSerializer;
import net.sf.json.xml.XMLSerializer;

import org.apache.commons.lang.StringEscapeUtils;

@WebService
public class JsonXml {
   
    @WebMethod
    @WebResult(name = "xml")
    public String JsonToXml(@WebParam(name = "namespace")String namespace, @WebParam(name = "rootelement")String rootelement,@WebParam(name = "jsonstring")String jsonStr) {
            XMLSerializer serializer = new XMLSerializer();
            serializer.setRootName(rootelement);
            serializer.addNamespace("", namespace);
            JSON json = JSONSerializer.toJSON(jsonStr);
            String xml = serializer.write(json);
           
            return xml;
    }

    @WebMethod
    @WebResult(name = "json")
    public String EscapedXmlToJson(@WebParam(name = "xmlstring")String xmlStr) {
        XMLSerializer xmlSerializer = new XMLSerializer();
        JSON json = xmlSerializer.read(StringEscapeUtils.unescapeXml(xmlStr)); 
        return( json.toString() ); 
    }
   
    @WebMethod(exclude=true)
    public String XmlToJson(@WebParam(name = "xmlstring")String xmlStr) {
        XMLSerializer xmlSerializer = new XMLSerializer(); 
        JSON json = xmlSerializer.read( xmlStr ); 
        return( json.toString() ); 
    }
/*
    public static void main(String[] args) {
        String Json = "{'foo':'bar',\n" +
        " 'coolness':2.0,\n" +
        " 'altitude':39000,\n" +
        " 'pilot':{'firstName':'Buzz',\n" +
        "          'lastName':'Aldrin'},\n" +
        " 'mission':'apollo 11'}";
        String Xml = new JsonXml().JsonToXml("http://www.google.com","root",Json);
        System.out.println(Json);
        System.out.println(Xml);
        Json = new JsonXml().XmlToJson(Xml);
        System.out.println(Json);
        String escapedXml = StringEscapeUtils.escapeXml(Xml);
        System.out.println(escapedXml);
        Json = new JsonXml().EscapedXmlToJson(escapedXml);
        System.out.println(Json);
       
    }
*/
}

The full code is available here; https://dl.dropbox.com/u/6693935/blog/JsonXmlApp.zip. There is also a JAR deployment profile (in addition to the WAR deployment profile) in the package and the dependencies are included. For Java developers, it is recommended (for performance reasons to avoid webservice overhead) to use the Jar deployment profile. From BPEL (in a service oriented landscape) it is more easy to have the functionality available as a webservice. The main method has been used as a sort of unittest.

Please mind, the conversion is not reversible and specific.I personally think the JSON to XML conversion is more useful then the XML to JSON conversion since that can be achieved more specifically with XSLT. I have not exposed the XmlToJson method since it's hard to call the method as a webservice with an XML parameter containing a non-escaped XML fragment.

The method of converting JSON to XML, has the benefit of not requiring an XSD schema describing the XML for the conversion and not requiring a Java object representation of the data. This has the benefit that one service/class is enough to do the conversions different messages (at least JSON to XML).

Further suggestions; JSON and BPEL

When you want to use the XML created by converting JSON in BPEL, you need to do the following;
- create a schema for the created XML message and include it in a used WSDL file
- create a variable using this schema
- use ora:parseEscapedXML to assign the output of the JsonToXml to the variable
- use the variable however you like

Because this can be cumbersome (creating the schema from the XML message, JDeveloper has a wizard but still...), it's advisable to create another webservice with two input variables; JSON string and JSON path expression returning the result of the JSON path expression on the JSON string. This can easily be accomplished by using for example http://code.google.com/p/json-path/. This way if you want to use a single variable from the JSON string, you can query for it directly and you don't need to create an XSD schema for the result of the JSON to XML conversion since it is a simple string. Use can use something like; String result = JsonPath.read(jsonstring, jsonpathstring).toString(). This is however not recommended if you need a lot of variables from the JSON string since calling a webservice is also overhead and might cost performance.

Packaging the above JSON samples in an XPATH library (see for example http://docs.oracle.com/cd/E23943_01/dev.1111/e10224/bp_appx_functs.htm) might be a better solution then wrapping them in webservices since it further increases their ease of use and performance in BPEL. If the services need to be re-used outside of BPEL by for example other middleware products, webservices might be the better solution.