Monday, August 27, 2012

Building BPEL / MDS applications using Jenkins and Ant

This post describes a tryout (not production ready) of a manually triggered Ant build of Oracle SOA applications using Jenkins and custom Ant scripts. I've used and expanded on the scripts mentioned in;


'builds' globally consist of several steps (details can differ among different customers)

- selecting what to build. this can be based for example on a tag or status/property of a source in a version control system or a selection of tasks with a specific status (if issue management and version control are linked).
- preparing a workarea / workspace based on the above
- triggering a build (manually or automated by monitoring a version control system)
- archiving the build ('releasing')

This post will have a limited scope since putting the above into place is customer specific and requires some time (a project) to achieve a production ready setup. I will manually trigger a build for a prepared workarea. This can of course be expanded. I've chosen to call Ant from Jenkins. The following post describes how Maven can be used to wrap Ant; Ant wraps Java classes to achieve deployment. Jenkins has plug-ins to directly call Ant tasks and has easy integration in the GUI (Ant output parsing). I do not see the added benefit of wrapping Ant in Maven.

I've installed Jenkins as described on;

I've chosen for Jenkins instead of Hudson based on

The Buildserver installed is an Ubuntu 12.04 server 64 bit installation.

I've adapted the scripts used in my previous buildtool post slightly for this purpose. I've used a JDeveloper installation instead of 'manually' copying JAR files. This has the benefit that JDeveloper can easily be upgraded and plugins can more easily be installed using the GUI. This provides a more steady and flexible base for the buildtool then manually copying JAR files.

MDS and adf-config.xml

Also for the purpose of MDS dependencies I've expanded the build.xml file to write an application specific adf-config.xml and copy it to the relevant application directory. The adf-config is copied to the application directory and it contains placeholders; MDS_REPOSITORY and MDS_APPLICATION. These are replaced in the build.xml using;

Copy to application

                <mkdir dir="${proj.compositeDir}/${proj.compositeName}/SCA-INF/classes/META-INF"/>
                <copy file="${base.dir}/adf-config.xml" tofile="${proj.compositeDir}/${proj.compositeName}/SCA-INF/classes/META-INF/adf-config.xml"/>
Replace placeholders
                <replace file="${proj.compositeDir}/${proj.compositeName}/SCA-INF/classes/META-INF/adf-config.xml" token="MDS_REPOSITORY" value="${mds.repository}"/>
                <replace file="${proj.compositeDir}/${proj.compositeName}/SCA-INF/classes/META-INF/adf-config.xml" token="MDS_APPLICATION" value="${mds.application}"/>

Buildtool setup

I've installed JDeveloper in /home/oracle/Middleware11116. I've used the standard installer. I did however encounter the following problem when I tried to call the 32 bit JDeveloper installer on a 64 bit OS;

I've created a buildtool directory /home/oracle/Middleware11116buildtool. In this directory I've created a symlink Middleware to /home/oracle/Middleware11116. The directory structure is as followed;

-rw-rw-r--  1 oracle oracle  1911 Aug 21 09:40 adf-config.xml
drwxrwxr-x  3 oracle oracle  4096 Aug 23 16:40 build-area
-rw-rw-r--  1 oracle oracle    83 Aug 25 13:06 build.num
-rw-rw-r--  1 oracle oracle   925 Aug 25 12:59
drwxrwxr-x 16 oracle oracle  4096 Aug 25 13:06 builds
-rw-rw-r--  1 oracle oracle 21961 Aug 25 10:54 build.xml
-rwxrwxr-x  1 oracle oracle   449 Aug 23 13:32 deployAll
-rw-rw-r--  1 oracle oracle   407 Aug 14 09:05 deployAll.bat
-rw-rw-r--  1 oracle oracle   261 Aug  5 18:07
drwxrwxr-x  2 oracle oracle  4096 Aug 23 13:33 junit
drwxrwxr-x  2 oracle oracle  4096 Aug 23 13:16 lib
drwxrwxr-x  2 oracle oracle  4096 Aug 25 13:06 logs
lrwxrwxrwx  1 oracle oracle    28 Aug 23 13:19 Middleware -> /home/oracle/Middleware11116

You can download the scripts here;

Jenkins setup

After the Jenkins installation I've added the Jenkins user to the oracle group (/etc/group) in order to access the buildtool, Ant, JDK, etc. After having started Jenkins, first I've configured the JDK and Ant;

I've set these to the JDeveloper JDK and Ant

Then I've created a multiconfiguration project

I've done this in order to be able to allow different configurations to be deployed at a later stage. I've not tried this yet though.

Then I've selected the JDK ant Ant version to be used. Important here is;

- specify the base dir in which Ant is executed; base.dir=/home/oracle/Middleware11116buildtool. this makes sure Ant can find the required scripts and of course the!
- specify the build.xml file; /home/oracle/Middleware11116buildtool/build.xml.


After the task is configured, it can be run. This results in the below
The different Ant tasks which are executed can be navigated to by clicking at the left side of the Console output on the specific task. Also the result is analyzed; in this case of course BUILD SUCCESFUL.

Friday, August 10, 2012

Issues and solutions when testing and developing asynchronous BPEL processes

Webservices, when developed, need testing. Some difficulties with testing webservices are the following;

- asynchronous services are not easily tested outside Fusion Middleware Control if there is no service available handling the correlation
- often it is not visible directly what the raw HTTP message is which is transmitted. HTTP headers are often hidden from the monitoring tooling. Also in case for example the Oracle HTTP binding adapter is used to do a JSON call, the adapter proxies the message. This causes only the adapter call to be visible in the audit trail in the Enterprise Manager.
- metrics for load tests are often needed. the start situation needs to be reproduced for subsequent tests and thus these metrics need to allow resetting

In this post I will describe solutions to these issues. A collegue Marcel Bellinga has provided suggestions on resetting of the metrics, usage of the JDeveloper HTTP Analyzer and has solved the issue when calling an asynchronous proces using a synchronous one in JDeveloper


I've created a synchronous HelloWorldAsync service. This service is wrapped by a synchronous HelloWorldAsyncCaller service. I want to know exactly (including HTTP headers) which message is send to the HelloWorldAsync service by the HelloWorldAsyncCaller service. In addition to this I want to be able to test the HelloWorldAsync service from SOAP-UI and collect metrics on the requests on the application server (SOAP UI can provide load tests but those are influenced by network latency and I'm not interested to take that also into account since it is time and customer dependant).

One first thing to mind in this setup is the following; when calling an asynchronous service from a synchronous one with a project created in JDeveloper, you will encounter the following error;

[2012-08-10T01:09:32.472-07:00] [AdminServer] [ERROR] [] [] [tid: [ACTIVE].ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)'] [userId: <anonymous>] [ecid: 11d1def534ea1be0:7c9113cc:1390f864b2c:-8000-0000000000000579,0:9] [ HelloWorldAsyncCaller_pt] [APP: soa-infra] [composite_name: HelloWorldAsyncCaller] [ fabric] [ helloworldasynccaller_client_ep] [ soa-infra] No response found for the reply for conversation id urn:99E85AE0E2C211E1BF3487E7CAA738F8

And then

[2012-08-10T01:09:32.836-07:00] [AdminServer] [ERROR] [OWS-04086] [oracle.webservices.service] [tid: [ACTIVE].ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)'] [userId: <anonymous>] [ecid: 11d1def534ea1be0:7c9113cc:1390f864b2c:-8000-0000000000000579,0:9] [APP: soa-infra] [composite_name: HelloWorldAsyncCaller] javax.xml.rpc.soap.SOAPFaultException: Waiting for response has timed out. The conversation id is null. Please check the process instance for detail.[[

This is caused by the following setting in the composite.xml file;

  <component name="HelloWorldAsyncCaller" version="2.0">
    <implementation.bpel src="HelloWorldAsyncCaller.bpel"/>
    <property name="bpel.config.transaction" type="xs:string" many="false">required</property>

If this line is removed or changed to

    <property name="bpel.config.transaction" type="xs:string" many="false">requiresNew</property>

This error will not occur.

This property (Transaction in the below screenshot) is created when creating a synchronous BPEL process in JDeveloper In previous versions ( was checked), the property was not added by default.

Asynchronous correlation issues with parallel execution in foreach

In order to increase performance, parallel execution in the foreach construct in BPEL 2.0 can be used. In a synchronous process when calling asynchronous services inside a loop in which all branches are executed simultaneously, the 'Conflicting receive' error can occur. To avoid this, correlation sets can be used. Using correlation sets is however relatively complex and requires the response message to contain correlation information in the message content. In order to avoid having to use correlation sets to achieve this, it is of course possible to wrap the asynchronous service in a synchronous one as described above (if the asynchronous service doesn't take too long) and call the synchronous process from the loop in the calling synchronous process (Synchronous process with loop -> synchronous process wrapper -> asynchronous process). It is however a better solution to define the partnerlink on the level of the scope which is executed in parallel. If you define the same partnerlink in global BPEL scope and in the scope which is executed in parallel, you can assign values to the request/response messages more easily. If a partnerlink is defined locally only, this is not always possible.


Monitoring service calls

JDeveloper has has an HTTP Analyzer and it is possible to specify the HTTP Analyzer as proxy in Fusion Middleware Control. This causes webservice requests to be proxied by the analyzer and allows the complete request to be monitored.

Start the HTTP Analyzer

Specify the IP of the machine running the HTTP Analyzer as HTTP proxy

 Test the webservice and look at the results

I was however surprised I couldn't see the HelloWorldAsyncCaller HTTP requests to the HelloWorldAsync process. I created a simple JAX-WS webservice instead to test this. Actually, this was correct; there were no HTTP requests to find here. If a BPEL process calls another BPEL process running in the same container, the transformation to HTTP is not done, which has of course benefits for performance. Load balancing however is something to think about as it requires more then just setting the load balancer as HTTP proxy in the services.

Testing asynchronous services with SOAP-UI

SOAP-UI ( is a tool which is often used for testing of webservices. Testing of asynchronous services is explained on;

Key is creating a test suite with two steps; a request and a mock response. The request contains WS-Adressing headers and a ReplyTo indication so the response is send to the Mock Response listener in SOAP UI.

Things to mind in the previously mentioned post; the specified path in step 8 under 'Testing it' should correspond to the ReplyTo path specified in the WS-Adressing section of the request. Also the ReplyTo address should point to the machine running SOAP UI. If the development environment is not running locally, the IP address of your machine should be specified here.

Process metrics

A servlet is available to analyze process metrics. See section 6.4.1 of for more information on this DMS Spy servlet. It can be started by going to http://<host>:<port>/dms/Spy.

How these metrics can be reset, can be found on;

An example URL for resetting metrics; http://localhost:7001/dms/Spy?operation=reset&format=raw&cache=refreshall&name=/dev_soa_osb/AdminServer/soainfra/default/HelloWorldAsyncCaller&recurse=all

The sample projects can be downloaded here;

Sunday, August 5, 2012

Standalone Buildtool for deployment of Oracle SOA Suite composites


SOA Suite 11g allows deployment to environments from JDeveloper. The people installing software often don't have JDeveloper available or configured correctly. Often there is the requirement to provide a standalone package for installation which can be executed without too much configuration by someone with relatively little knowledge about what he/she is installing.

There have already been numerous posts about build automation for Oracle SOA 11g applications, such as the excellent post from Edwin Biemond; and

In this post I've created a standalone Buildtool for SOA Suite This Buildtool allows compilation and deployment of various SOA artifacts. It is heavily based on however provides some added value such as;

- it's fully based on libraries
- can be used standalone without the need to install JDeveloper or Weblogic Server
- allows applications to be packaged/configured separately and independently
- provides a script to recreate most of the Buildtool from scratch (hopefully making upgrades to the buildtool more easy)
- this buildtool compiles and installs composites. compilation is thus not dependent on the client providing a jar file; the sources can be used as source for the build.


Install JDeveloper

Download and install JDeveloper Install the following plugins from

-          Oracle BPM Studio 11g
-          Oracle SOA Composite Editor

This will be used as the start-point for the Buildtool.

Copy sources

The below DOS batch script copies the required libraries to the TARGETDIR (build tool base directory). The SOURCEDIR is a locally installed JDeveloper

set SOURCEDIR=D:\dev\Middleware11116
set TARGETDIR=D:\dev\buildtool11116

rem buildtool generator
rem Maarten Smeets, 2012-08-05

mkdir %TARGETDIR%\logs
mkdir %TARGETDIR%\junit
mkdir %TARGETDIR%\builds

rem wlserver_10.3/server/lib
mkdir %TARGETDIR%\Middleware\wlserver_10.3\server\lib
xcopy /i %SOURCEDIR%\wlserver_10.3\server\lib\*.jar %TARGETDIR%\Middleware\wlserver_10.3\server\lib

rem jdeveloper\ant
mkdir %TARGETDIR%\Middleware\jdeveloper\ant
xcopy /s %SOURCEDIR%\jdeveloper\ant %TARGETDIR%\Middleware\jdeveloper\ant

rem jdeveloper\bin
mkdir %TARGETDIR%\Middleware\jdeveloper\bin
xcopy /s %SOURCEDIR%\jdeveloper\bin %TARGETDIR%\Middleware\jdeveloper\bin

rem jdeveloper\soa\modules
mkdir %TARGETDIR%\Middleware\jdeveloper\soa\modules
xcopy /s %SOURCEDIR%\jdeveloper\soa\modules %TARGETDIR%\Middleware\jdeveloper\soa\modules

rem jdeveloper\uddi
mkdir %TARGETDIR%\Middleware\jdeveloper\uddi
xcopy /s %SOURCEDIR%\jdeveloper\uddi %TARGETDIR%\Middleware\jdeveloper\uddi

rem jdk160_24
rem jdk160_21
mkdir %TARGETDIR%\Middleware\jdk160_24
xcopy /s %SOURCEDIR%\jdk160_24 %TARGETDIR%\Middleware\jdk160_24

rem ant-contrib.jar
mkdir %TARGETDIR%\lib
xcopy %SOURCEDIR%\modules\net.sf.antcontrib_1.1.0.0_1-0b2\lib\* %TARGETDIR%\lib

rem oracle_common\webservices
mkdir %TARGETDIR%\Middleware\oracle_common\webservices
xcopy /s %SOURCEDIR%\oracle_common\webservices %TARGETDIR%\Middleware\oracle_common\webservices

rem oracle_common/modules
mkdir %TARGETDIR%\Middleware\oracle_common\modules\oracle.adf.model_11.1.1
mkdir %TARGETDIR%\Middleware\oracle_common\modules\oracle.adf.share.ca_11.1.1
mkdir %TARGETDIR%\Middleware\oracle_common\modules\oracle.adf.share_11.1.1
mkdir %TARGETDIR%\Middleware\oracle_common\modules\oracle.bali.share_11.1.1
mkdir %TARGETDIR%\Middleware\oracle_common\modules\oracle.dms_11.1.1
mkdir %TARGETDIR%\Middleware\oracle_common\modules\oracle.fabriccommon_11.1.1
mkdir %TARGETDIR%\Middleware\oracle_common\modules\oracle.javacache_11.1.1
mkdir %TARGETDIR%\Middleware\oracle_common\modules\oracle.jmx_11.1.1
mkdir %TARGETDIR%\Middleware\oracle_common\modules\oracle.jps_11.1.1
mkdir %TARGETDIR%\Middleware\oracle_common\modules\oracle.jrf_11.1.1
mkdir %TARGETDIR%\Middleware\oracle_common\modules\oracle.mds_11.1.1
mkdir %TARGETDIR%\Middleware\oracle_common\modules\oracle.nlsrtl_11.1.0
mkdir %TARGETDIR%\Middleware\oracle_common\modules\oracle.odl_11.1.1
mkdir %TARGETDIR%\Middleware\oracle_common\modules\oracle.webservices_11.1.1
mkdir %TARGETDIR%\Middleware\oracle_common\modules\oracle.wsm.common_11.1.1
mkdir %TARGETDIR%\Middleware\oracle_common\modules\oracle.xdk_11.1.0
mkdir %TARGETDIR%\Middleware\oracle_common\modules\oracle.xmlef_11.1.1
mkdir %TARGETDIR%\Middleware\oracle_common\modules\datadirect_4.1

xcopy /s %SOURCEDIR%\oracle_common\modules\oracle.adf.model_11.1.1 %TARGETDIR%\Middleware\oracle_common\modules\oracle.adf.model_11.1.1
xcopy /s %SOURCEDIR%\oracle_common\modules\oracle.adf.share.ca_11.1.1 %TARGETDIR%\Middleware\oracle_common\modules\oracle.adf.share.ca_11.1.1
xcopy /s %SOURCEDIR%\oracle_common\modules\oracle.adf.share_11.1.1 %TARGETDIR%\Middleware\oracle_common\modules\oracle.adf.share_11.1.1
xcopy /s %SOURCEDIR%\oracle_common\modules\oracle.bali.share_11.1.1 %TARGETDIR%\Middleware\oracle_common\modules\oracle.bali.share_11.1.1
xcopy /s %SOURCEDIR%\oracle_common\modules\oracle.dms_11.1.1 %TARGETDIR%\Middleware\oracle_common\modules\oracle.dms_11.1.1
xcopy /s %SOURCEDIR%\oracle_common\modules\oracle.fabriccommon_11.1.1 %TARGETDIR%\Middleware\oracle_common\modules\oracle.fabriccommon_11.1.1
xcopy /s %SOURCEDIR%\oracle_common\modules\oracle.javacache_11.1.1 %TARGETDIR%\Middleware\oracle_common\modules\oracle.javacache_11.1.1
xcopy /s %SOURCEDIR%\oracle_common\modules\oracle.jmx_11.1.1 %TARGETDIR%\Middleware\oracle_common\modules\oracle.jmx_11.1.1
xcopy /s %SOURCEDIR%\oracle_common\modules\oracle.jps_11.1.1 %TARGETDIR%\Middleware\oracle_common\modules\oracle.jps_11.1.1
xcopy /s %SOURCEDIR%\oracle_common\modules\oracle.jrf_11.1.1 %TARGETDIR%\Middleware\oracle_common\modules\oracle.jrf_11.1.1
xcopy /s %SOURCEDIR%\oracle_common\modules\oracle.mds_11.1.1 %TARGETDIR%\Middleware\oracle_common\modules\oracle.mds_11.1.1
xcopy /s %SOURCEDIR%\oracle_common\modules\oracle.nlsrtl_11.1.0 %TARGETDIR%\Middleware\oracle_common\modules\oracle.nlsrtl_11.1.0
xcopy /s %SOURCEDIR%\oracle_common\modules\oracle.odl_11.1.1 %TARGETDIR%\Middleware\oracle_common\modules\oracle.odl_11.1.1
xcopy /s %SOURCEDIR%\oracle_common\modules\oracle.webservices_11.1.1 %TARGETDIR%\Middleware\oracle_common\modules\oracle.webservices_11.1.1
xcopy /s %SOURCEDIR%\oracle_common\modules\oracle.wsm.common_11.1.1 %TARGETDIR%\Middleware\oracle_common\modules\oracle.wsm.common_11.1.1
xcopy /s %SOURCEDIR%\oracle_common\modules\oracle.xdk_11.1.0 %TARGETDIR%\Middleware\oracle_common\modules\oracle.xdk_11.1.0
xcopy /s %SOURCEDIR%\oracle_common\modules\datadirect_4.1 %TARGETDIR%\Middleware\oracle_common\modules\datadirect_4.1

xcopy /i %SOURCEDIR%\oracle_common\modules\oracle.ucp_11.1.0.jar %TARGETDIR%\Middleware\oracle_common\modules
xcopy /i %SOURCEDIR%\oracle_common\modules\org.apache.commons.beanutils_1.6.jar %TARGETDIR%\Middleware\oracle_common\modules
xcopy /i %SOURCEDIR%\oracle_common\modules\org.apache.commons.digester_1.8.jar %TARGETDIR%\Middleware\oracle_common\modules
xcopy /i %SOURCEDIR%\oracle_common\modules\org.apache.commons.logging_1.0.4.jar %TARGETDIR%\Middleware\oracle_common\modules
xcopy /i %SOURCEDIR%\oracle_common\modules\org.springframework_2.5.jar %TARGETDIR%\Middleware\oracle_common\modules
xcopy /i %SOURCEDIR%\oracle_common\modules\oracle.classloader_11.1.1.jar %TARGETDIR%\Middleware\oracle_common\modules
xcopy /i %SOURCEDIR%\oracle_common\modules\oracle.http_client_11.1.1.jar %TARGETDIR%\Middleware\oracle_common\modules
xcopy /i %SOURCEDIR%\oracle_common\modules\oracle.logging-utils_11.1.1.jar  %TARGETDIR%\Middleware\oracle_common\modules
xcopy /i %SOURCEDIR%\oracle_common\modules\commonj.sdo_2.1.0.jar  %TARGETDIR%\Middleware\oracle_common\modules
xcopy /i %SOURCEDIR%\oracle_common\modules\com.bea.core.xml.xmlbeans_2.1.0.0_2-5-1.jar  %TARGETDIR%\Middleware\oracle_common\modules
xcopy /i %SOURCEDIR%\oracle_common\modules\  %TARGETDIR%\Middleware\oracle_common\modules

rem modules

mkdir %TARGETDIR%\Middleware\modules

xcopy /i %SOURCEDIR%\modules\com.bea.core.apache.commons.collections_3.2.0.jar %TARGETDIR%\Middleware\modules
xcopy /i %SOURCEDIR%\modules\com.bea.core.apache.commons.lang_2.1.0.jar %TARGETDIR%\Middleware\modules
rem xcopy /i %SOURCEDIR%\modules\com.bea.core.weblogic.workmanager_1.9.0.0.jar %TARGETDIR%\Middleware\modules
xcopy /i %SOURCEDIR%\modules\com.bea.core.weblogic.workmanager_1.10.0.0.jar %TARGETDIR%\Middleware\modules
rem xcopy /i %SOURCEDIR%\modules\com.bea.core.xml.beaxmlbeans_2.2.0.0_2-5-1.jar %TARGETDIR%\Middleware\modules
xcopy /i %SOURCEDIR%\modules\com.bea.core.xml.beaxmlbeans_2.3.0.0_2-5-1.jar %TARGETDIR%\Middleware\modules
xcopy /i %SOURCEDIR%\modules\commonj.sdo_1.0.0.0_1-0.jar %TARGETDIR%\Middleware\modules
xcopy /i %SOURCEDIR%\modules\glassfish.jaxb_1.0.0.0_2-1-12.jar %TARGETDIR%\Middleware\modules
xcopy /i %SOURCEDIR%\modules\ %TARGETDIR%\Middleware\modules
xcopy /i %SOURCEDIR%\modules\javax.xml.rpc_1.2.1.jar %TARGETDIR%\Middleware\modules
rem xcopy /i %SOURCEDIR%\modules\org.eclipse.persistence_1.0.0.0_2-1.jar %TARGETDIR%\Middleware\modules
xcopy /i %SOURCEDIR%\modules\org.eclipse.persistence_1.1.0.0_2-1.jar %TARGETDIR%\Middleware\modules

Some script files were taken from and updated. The updated scriptfiles can be downloaded here;


In my case, the buildtool base dir is; D:\dev\buildtool11116. Here you can create a directory build-area (as specified in the file). In this directory you can create a structure like for example;

The structure contains an example project, a deployment profile and configuration files. This is what the developer has to provide. The person installing the software has to update the following files;
- update deployAll.bat to reflect the correct locations (once)
- update to reflect the environment which is used (it is recommended the installer keeps per environment a separate file)

When deployAll.bat is executed, log files are generated in the logs directory and the deployed applications are put in the builds directory.

The build.xml is capable of calling unit tests, but since I got the following error which I couldn't fix quickly enough;

  [scatest] java.lang.NoClassDefFoundError: weblogic/security/subject/AbstractSubject


D:\dev\buildtool11116\Middleware\jdeveloper\bin\ant-sca-test.xml:111: Problem executing test cases

I've decided to put this functionality in comments and maybe look at it later. wlclient.jar contains this class and it should be in the classpath.

The example build-area provides a simple deployment of a SCA composite. The supplied build.xml provides functionality for MDS deployments and partitions (Edwin Biemond has described this functionality on his blog site). Some minor changes to the script might be needed to make sure deployment of the MDS is from the build-area folder so developers can more easily provide the required files. You can download the entire buildtool here;