Oracle SOA Suite 11g and 12c: Determining composite dependencies to the level of individual operations

In large companies, often there are many services and dependencies between services. It is important to track service dependencies in order to for example estimate the impact of changes. Design documents or architecture views can be used for this but as everybody knows, there is often a gap between theory and practice (design and implementation).

In this blog post I provide code to determine dependencies between composites to the level of operation calls. In order to achieve this, I'll parse the composite.xml files, JCA files (used by adapters) and also the BPEL and BPMN files in order to determine the operations. The script can be used for SOA Suite 11g and 12c composites.

The above picture shows different parts of which a composite is composed and how they are linked. The script first determines references. The references specify which external services are called. Then by using wires, the relevant components are determined. Based on the component type, specific logic is used to extract the operation. Not shown in this picture is how database dependencies can also be determined by the script by parsing the JCA files specified in the reference. If you're in a hurry, you can go to the 'Executing the script' part directly and skip the explanation.

Composites

This blog will focus on composites (which can contain components like BPEL and BPM) using the shared SOA infrastructure and not on for example the Service Bus. Composites use the Software Component Architecture (http://www.oasis-opencsa.org/sca) to wire different components together in a consistent way. Oracle uses XML files to describe composites, components and references. These files can easily be parsed and correlated.

Introduction composite.xml

The main file describing a composite is the composite.xml file. Below is a small sample of a HelloWorld composite containing a single component, a BPEL process.

 <?xml version="1.0" encoding="UTF-8" ?>  
 <!-- Generated by Oracle SOA Modeler version 12.1.3.0.0 at [5-9-14 11:37]. -->  
 <composite name="HelloWorld"  
       revision="1.0"  
       label="2014-09-05_11-37-30_542"  
       mode="active"  
       state="on"  
       xmlns="http://xmlns.oracle.com/sca/1.0"  
       xmlns:xs="http://www.w3.org/2001/XMLSchema"  
       xmlns:wsp="http://schemas.xmlsoap.org/ws/2004/09/policy"  
       xmlns:orawsp="http://schemas.oracle.com/ws/2006/01/policy"  
       xmlns:ui="http://xmlns.oracle.com/soa/designer/"  
       xmlns:sca-ext="http://xmlns.oracle.com/sca/1.0-ext">  
  <import namespace="http://xmlns.oracle.com/SOAApplication/HelloWorld/HelloWorld" location="WSDLs/HelloWorld.wsdl"  
      importType="wsdl"/>  
  <service name="helloworld_client_ep" ui:wsdlLocation="WSDLs/HelloWorld.wsdl">  
   <interface.wsdl interface="http://xmlns.oracle.com/SOAApplication/HelloWorld/HelloWorld#wsdl.interface(HelloWorld)"/>  
   <binding.ws port="http://xmlns.oracle.com/SOAApplication/HelloWorld/HelloWorld#wsdl.endpoint(helloworld_client_ep/HelloWorld_pt)"/>  
  </service>  
  <property name="productVersion" type="xs:string" many="false">12.1.3.0.0</property>  
  <property name="compositeID" type="xs:string" many="false">50fcf028-89ea-46bd-b865-bdee0ff8e1d8</property>  
  <component name="HelloWorld" version="2.0">  
   <implementation.bpel src="BPEL/HelloWorld.bpel"/>  
   <componentType>  
    <service name="helloworld_client" ui:wsdlLocation="WSDLs/HelloWorld.wsdl">  
     <interface.wsdl interface="http://xmlns.oracle.com/SOAApplication/HelloWorld/HelloWorld#wsdl.interface(HelloWorld)"/>  
    </service>  
   </componentType>  
   <property name="bpel.config.transaction" type="xs:string" many="false">required</property>  
  </component>  
  <wire>  
   <source.uri>helloworld_client_ep</source.uri>  
   <target.uri>HelloWorld/helloworld_client</target.uri>  
  </wire>  
 </composite>  

As can be seen, the composite contains properties, services, component definitions and wires which link components to either services or references. This composite has one service: helloworld_client_ep and no references.

Web service references

When I create a new composite which calls this one (HelloWorldCaller), an example can be seen of a reference in the composite of the newly created process.

  <reference name="HelloWorldSOAPReference"  
        ui:wsdlLocation="https://soa2admin2.example.com:8002/soa-infra/services/default/HelloWorld!1.0/WSDLs/HelloWorld.wsdl">  
   <interface.wsdl interface="http://xmlns.oracle.com/SOAApplication/HelloWorld/HelloWorld#wsdl.interface(HelloWorld)"/>  
   <binding.ws port="http://xmlns.oracle.com/SOAApplication/HelloWorld/HelloWorld#wsdl.endpoint(helloworld_client_ep/HelloWorld_pt)"  
         location="https://soa2admin2.example.com:8002/soa-infra/services/default/HelloWorld/helloworld_client_ep?WSDL"  
         soapVersion="1.1">  
    <property name="weblogic.wsee.wsat.transaction.flowOption" type="xs:string" many="false">WSDLDriven</property>  
   </binding.ws>  
  </reference>  

This reference contains information on the development time WSDL to be used (ui:wsdlLocation) and the concrete binding at runtime (binding.ws). The ui:wsdlLocation should of course be an MDS path in a production environment (see https://blogs.oracle.com/aia/entry/aia_11g_best_practices_for_dec).

The name of the service to be called can be abstracted in various ways from for example the WSDL path if the service is hosted on the SOA infrastructure. On different components the URL can be manually set. Since the method to determine the service name differs per customer/technology, I will not further elaborate on this (service virtualization is a best practice). You can however in most cases use the composite.xml file to determine the service called. This is required in order to link services together, which is necessary in order to visualize dependencies. In my script I've used a method to determine the service name from the service WSDL: wsdl_to_servicename.

Determining the operation for BPEL processes

The operation is specified in the invoke action inside a BPEL process. The invoke action partnerLink attribute contains the name of the partnerlink. This partnerlink name corresponds to the name of the reference in the composite.xml. This can be used to select specific operations.

Determining the operation for BPMN processes

BPMN processes are structured differently then BPEL processes. Here Conversations are defined in the BPMN XML file. The Conversations are referred to by Conversationals which have a ServiceCallConversationalDefinition. This definition contains the operation which is called.

JCA references

JCA references are references which call JCA adapters such as the JMSAdapter, the DbAdapter or other adapters. The references in this case are not webservices and the binding in the composite looks a bit different.

  <reference name="HelloWorldSysdateDbReference" ui:wsdlLocation="WSDLs/HelloWorldSysdateDbReference.wsdl">  
   <interface.wsdl interface="http://xmlns.oracle.com/pcbpel/adapter/db/SOAApplication/HelloWorldSysdate/HelloWorldSysdateDbReference#wsdl.interface(HelloWorldSysdateDbReference_ptt)"/>  
   <binding.jca config="Adapters/HelloWorldSysdateDbReference_db.jca"/>  
   <property name="jca.retry.count" type="xs:int" many="false" override="may">4</property>  
   <property name="jca.retry.interval" type="xs:int" many="false" override="may">1</property>  
   <property name="jca.retry.backoff" type="xs:int" many="false" override="may">2</property>  
   <property name="jca.retry.maxInterval" type="xs:int" many="false" override="may">120</property>  
  </reference>  

In this case, the important information is contained in the file referenced by the binding.jca element. The JCA file specified contains the following information in my case.

 <adapter-config name="HelloWorldSysdateDbReference" adapter="db" wsdlLocation="../WSDLs/HelloWorldSysdateDbReference.wsdl" xmlns="http://platform.integration.oracle/blocks/adapter/fw/metadata">  
  <connection-factory UIConnectionName="testuser" location="eis/DB/testuser"/>  
  <endpoint-interaction portType="HelloWorldSysdateDbReference_ptt" operation="HelloWorldSysdateDbReference">  
   <interaction-spec className="oracle.tip.adapter.db.DBStoredProcedureInteractionSpec">  
    <property name="SchemaName" value="TESTUSER"/>  
    <property name="PackageName" value="UTILS"/>  
    <property name="ProcedureName" value="GET_SYSDATE"/>  
    <property name="GetActiveUnitOfWork" value="false"/>  
   </interaction-spec>  
  </endpoint-interaction>  
 </adapter-config>  

From this JCA file, I can determine the connection factory, package and procedure name. Do keep in mind that you should check if the adapter="db" and the interaction-spec className attribute is oracle.tip.adapter.db.DBStoredProcedureInteractionSpec if you want to check the PackageName property. Other JCA adapters can be parsed in a similar way.

Executing the script

The script can be downloaded here: https://dl.dropboxusercontent.com/u/6693935/blog/soaparser.zip

In order to execute it, you need Python 2.7 (I have not checked if it works with higher or lower versions). I have used PyDev plugin in Eclipse Luna to develop it. I'm no expert Python programmer so do not use this script as an example on how you should write correct Python scripts. You need to change the path to the root where composites can be searched for recursively.

As you can see in the bottom right part of the screenshot, my testcase contained three services. HelloWorld, HelloWorldCaller which called the process operation on HelloWorld and HelloWorldDb which called the database procedure TESTUSER.UTILS.GET_SYSDATE

Conclusion

In this blog I have supplied a script to analyse composite dependencies. Some reservations are in order though.

Custom code
Of course, when using interesting specific custom logic to dynamically determine calls at runtime, code to determine dependencies also needs to be written specifically for the implementation. In such cases this script will not suffice.

No runtime dependencies
The script does not analyse a runtime situation but code present in a certain directory. If services have their own lifecycle, there might not be a specific directory which contains the state of a complete environment.

No BPEL 10g
For BPEL 10g, the bpel.xml file and the adapter WSDL files can be analyzed in a similar way. At the time of writing, I hope not many customers are still using 10g though.

Further

The method of parsing code to determine dependencies can also be used to extent to other technologies. Such as frontend applications. They can be linked to the services providing a more complete image of the application landscape.

When using a simple visualization tool such as Graphviz (http://www.graphviz.org/) you can visualize the dependencies. It requires the dependencies to be provided in a specific format, the DOT language. This language is relatively easy to generate from a script. Such an image can be very interesting to developers, architects and designers to determine who uses a certain service.