Showing posts with label buildtool. Show all posts
Showing posts with label buildtool. Show all posts

Monday, August 27, 2012

Building BPEL / MDS applications using Jenkins and Ant

This post describes a tryout (not production ready) of a manually triggered Ant build of Oracle SOA applications using Jenkins and custom Ant scripts. I've used and expanded on the scripts mentioned in; http://javaoraclesoa.blogspot.nl/2012/08/standalone-buildtool-for-deployment-of.html

Introduction

'builds' globally consist of several steps (details can differ among different customers)

- selecting what to build. this can be based for example on a tag or status/property of a source in a version control system or a selection of tasks with a specific status (if issue management and version control are linked).
- preparing a workarea / workspace based on the above
- triggering a build (manually or automated by monitoring a version control system)
- archiving the build ('releasing')

This post will have a limited scope since putting the above into place is customer specific and requires some time (a project) to achieve a production ready setup. I will manually trigger a build for a prepared workarea. This can of course be expanded. I've chosen to call Ant from Jenkins. The following post describes how Maven can be used to wrap Ant; http://redstack.wordpress.com/2011/03/15/getting-started-with-continuous-integration-for-soa-projects/. Ant wraps Java classes to achieve deployment. Jenkins has plug-ins to directly call Ant tasks and has easy integration in the GUI (Ant output parsing). I do not see the added benefit of wrapping Ant in Maven.

I've installed Jenkins as described on; https://wiki.jenkins-ci.org/display/JENKINS/Installing+Jenkins+on+Ubuntu

I've chosen for Jenkins instead of Hudson based on http://stackoverflow.com/questions/4973981/how-to-choose-between-hudson-and-jenkins.

The Buildserver installed is an Ubuntu 12.04 server 64 bit installation.
 
Implementation

I've adapted the scripts used in my previous buildtool post slightly for this purpose. I've used a JDeveloper installation instead of 'manually' copying JAR files. This has the benefit that JDeveloper can easily be upgraded and plugins can more easily be installed using the GUI. This provides a more steady and flexible base for the buildtool then manually copying JAR files.

MDS and adf-config.xml

Also for the purpose of MDS dependencies I've expanded the build.xml file to write an application specific adf-config.xml and copy it to the relevant application directory. The adf-config is copied to the application directory and it contains placeholders; MDS_REPOSITORY and MDS_APPLICATION. These are replaced in the build.xml using;

Copy to application

                <mkdir dir="${proj.compositeDir}/${proj.compositeName}/SCA-INF/classes/META-INF"/>
                <copy file="${base.dir}/adf-config.xml" tofile="${proj.compositeDir}/${proj.compositeName}/SCA-INF/classes/META-INF/adf-config.xml"/>
    
Replace placeholders
               
                <replace file="${proj.compositeDir}/${proj.compositeName}/SCA-INF/classes/META-INF/adf-config.xml" token="MDS_REPOSITORY" value="${mds.repository}"/>
                <replace file="${proj.compositeDir}/${proj.compositeName}/SCA-INF/classes/META-INF/adf-config.xml" token="MDS_APPLICATION" value="${mds.application}"/>

Buildtool setup

I've installed JDeveloper in /home/oracle/Middleware11116. I've used the standard installer. I did however encounter the following problem when I tried to call the 32 bit JDeveloper installer on a 64 bit OS; http://stackoverflow.com/questions/2716702/no-such-file-or-directory-error-when-executing-a-binary

I've created a buildtool directory /home/oracle/Middleware11116buildtool. In this directory I've created a symlink Middleware to /home/oracle/Middleware11116. The directory structure is as followed;

-rw-rw-r--  1 oracle oracle  1911 Aug 21 09:40 adf-config.xml
drwxrwxr-x  3 oracle oracle  4096 Aug 23 16:40 build-area
-rw-rw-r--  1 oracle oracle    83 Aug 25 13:06 build.num
-rw-rw-r--  1 oracle oracle   925 Aug 25 12:59 build.properties
drwxrwxr-x 16 oracle oracle  4096 Aug 25 13:06 builds
-rw-rw-r--  1 oracle oracle 21961 Aug 25 10:54 build.xml
-rwxrwxr-x  1 oracle oracle   449 Aug 23 13:32 deployAll
-rw-rw-r--  1 oracle oracle   407 Aug 14 09:05 deployAll.bat
-rw-rw-r--  1 oracle oracle   261 Aug  5 18:07 dev.jndi.properties
drwxrwxr-x  2 oracle oracle  4096 Aug 23 13:33 junit
drwxrwxr-x  2 oracle oracle  4096 Aug 23 13:16 lib
drwxrwxr-x  2 oracle oracle  4096 Aug 25 13:06 logs
lrwxrwxrwx  1 oracle oracle    28 Aug 23 13:19 Middleware -> /home/oracle/Middleware11116

You can download the scripts here; http://dl.dropbox.com/u/6693935/blog/buildtoolscriptsincmds.zip

Jenkins setup

After the Jenkins installation I've added the Jenkins user to the oracle group (/etc/group) in order to access the buildtool, Ant, JDK, etc. After having started Jenkins, first I've configured the JDK and Ant;

I've set these to the JDeveloper JDK and Ant

Then I've created a multiconfiguration project


I've done this in order to be able to allow different configurations to be deployed at a later stage. I've not tried this yet though.


Then I've selected the JDK ant Ant version to be used. Important here is;

- specify the base dir in which Ant is executed; base.dir=/home/oracle/Middleware11116buildtool. this makes sure Ant can find the required scripts and of course the build.properties!
- specify the build.xml file; /home/oracle/Middleware11116buildtool/build.xml.

Running

After the task is configured, it can be run. This results in the below
The different Ant tasks which are executed can be navigated to by clicking at the left side of the Console output on the specific task. Also the result is analyzed; in this case of course BUILD SUCCESFUL.

Sunday, August 5, 2012

Standalone Buildtool for deployment of Oracle SOA Suite 11.1.1.6 composites

Introduction

SOA Suite 11g allows deployment to environments from JDeveloper. The people installing software often don't have JDeveloper available or configured correctly. Often there is the requirement to provide a standalone package for installation which can be executed without too much configuration by someone with relatively little knowledge about what he/she is installing.

There have already been numerous posts about build automation for Oracle SOA 11g applications, such as the excellent post from Edwin Biemond; http://biemond.blogspot.nl/2009/09/deploy-soa-suite-11g-composite.html and http://redstack.wordpress.com/2011/03/15/getting-started-with-continuous-integration-for-soa-projects/

In this post I've created a standalone Buildtool for SOA Suite 11.1.1.6. This Buildtool allows compilation and deployment of various SOA artifacts. It is heavily based on http://biemond.blogspot.nl/2009/09/deploy-soa-suite-11g-composite.html however provides some added value such as;

- it's fully based on 11.1.1.6 libraries
- can be used standalone without the need to install JDeveloper or Weblogic Server
- allows applications to be packaged/configured separately and independently
- provides a script to recreate most of the Buildtool from scratch (hopefully making upgrades to the buildtool more easy)
- this buildtool compiles and installs composites. compilation is thus not dependent on the client providing a jar file; the sources can be used as source for the build.

Implementation

Install JDeveloper

Download and install JDeveloper 11.1.1.6. Install the following plugins from http://www.oracle.com/ocom/groups/public/@otn/documents/webcontent/156082.xml

-          Oracle BPM Studio 11g
-          Oracle SOA Composite Editor

This will be used as the start-point for the Buildtool.

Copy sources

The below DOS batch script copies the required libraries to the TARGETDIR (build tool base directory). The SOURCEDIR is a locally installed JDeveloper 11.1.1.6.

set SOURCEDIR=D:\dev\Middleware11116
set TARGETDIR=D:\dev\buildtool11116


rem buildtool generator 11.1.1.6
rem Maarten Smeets, 2012-08-05


mkdir %TARGETDIR%\logs
mkdir %TARGETDIR%\junit
mkdir %TARGETDIR%\builds

rem wlserver_10.3/server/lib
mkdir %TARGETDIR%\Middleware\wlserver_10.3\server\lib
xcopy /i %SOURCEDIR%\wlserver_10.3\server\lib\*.jar %TARGETDIR%\Middleware\wlserver_10.3\server\lib


rem jdeveloper\ant
mkdir %TARGETDIR%\Middleware\jdeveloper\ant
xcopy /s %SOURCEDIR%\jdeveloper\ant %TARGETDIR%\Middleware\jdeveloper\ant


rem jdeveloper\bin
mkdir %TARGETDIR%\Middleware\jdeveloper\bin
xcopy /s %SOURCEDIR%\jdeveloper\bin %TARGETDIR%\Middleware\jdeveloper\bin


rem jdeveloper\soa\modules
mkdir %TARGETDIR%\Middleware\jdeveloper\soa\modules
xcopy /s %SOURCEDIR%\jdeveloper\soa\modules %TARGETDIR%\Middleware\jdeveloper\soa\modules


rem jdeveloper\uddi
mkdir %TARGETDIR%\Middleware\jdeveloper\uddi
xcopy /s %SOURCEDIR%\jdeveloper\uddi %TARGETDIR%\Middleware\jdeveloper\uddi


rem jdk160_24
rem 11.1.1.4: jdk160_21
mkdir %TARGETDIR%\Middleware\jdk160_24
xcopy /s %SOURCEDIR%\jdk160_24 %TARGETDIR%\Middleware\jdk160_24


rem ant-contrib.jar
mkdir %TARGETDIR%\lib
xcopy %SOURCEDIR%\modules\net.sf.antcontrib_1.1.0.0_1-0b2\lib\* %TARGETDIR%\lib


rem oracle_common\webservices
mkdir %TARGETDIR%\Middleware\oracle_common\webservices
xcopy /s %SOURCEDIR%\oracle_common\webservices %TARGETDIR%\Middleware\oracle_common\webservices


rem oracle_common/modules
mkdir %TARGETDIR%\Middleware\oracle_common\modules\oracle.adf.model_11.1.1
mkdir %TARGETDIR%\Middleware\oracle_common\modules\oracle.adf.share.ca_11.1.1
mkdir %TARGETDIR%\Middleware\oracle_common\modules\oracle.adf.share_11.1.1
mkdir %TARGETDIR%\Middleware\oracle_common\modules\oracle.bali.share_11.1.1
mkdir %TARGETDIR%\Middleware\oracle_common\modules\oracle.dms_11.1.1
mkdir %TARGETDIR%\Middleware\oracle_common\modules\oracle.fabriccommon_11.1.1
mkdir %TARGETDIR%\Middleware\oracle_common\modules\oracle.javacache_11.1.1
mkdir %TARGETDIR%\Middleware\oracle_common\modules\oracle.jmx_11.1.1
mkdir %TARGETDIR%\Middleware\oracle_common\modules\oracle.jps_11.1.1
mkdir %TARGETDIR%\Middleware\oracle_common\modules\oracle.jrf_11.1.1
mkdir %TARGETDIR%\Middleware\oracle_common\modules\oracle.mds_11.1.1
mkdir %TARGETDIR%\Middleware\oracle_common\modules\oracle.nlsrtl_11.1.0
mkdir %TARGETDIR%\Middleware\oracle_common\modules\oracle.odl_11.1.1
mkdir %TARGETDIR%\Middleware\oracle_common\modules\oracle.webservices_11.1.1
mkdir %TARGETDIR%\Middleware\oracle_common\modules\oracle.wsm.common_11.1.1
mkdir %TARGETDIR%\Middleware\oracle_common\modules\oracle.xdk_11.1.0
mkdir %TARGETDIR%\Middleware\oracle_common\modules\oracle.xmlef_11.1.1
mkdir %TARGETDIR%\Middleware\oracle_common\modules\datadirect_4.1


xcopy /s %SOURCEDIR%\oracle_common\modules\oracle.adf.model_11.1.1 %TARGETDIR%\Middleware\oracle_common\modules\oracle.adf.model_11.1.1
xcopy /s %SOURCEDIR%\oracle_common\modules\oracle.adf.share.ca_11.1.1 %TARGETDIR%\Middleware\oracle_common\modules\oracle.adf.share.ca_11.1.1
xcopy /s %SOURCEDIR%\oracle_common\modules\oracle.adf.share_11.1.1 %TARGETDIR%\Middleware\oracle_common\modules\oracle.adf.share_11.1.1
xcopy /s %SOURCEDIR%\oracle_common\modules\oracle.bali.share_11.1.1 %TARGETDIR%\Middleware\oracle_common\modules\oracle.bali.share_11.1.1
xcopy /s %SOURCEDIR%\oracle_common\modules\oracle.dms_11.1.1 %TARGETDIR%\Middleware\oracle_common\modules\oracle.dms_11.1.1
xcopy /s %SOURCEDIR%\oracle_common\modules\oracle.fabriccommon_11.1.1 %TARGETDIR%\Middleware\oracle_common\modules\oracle.fabriccommon_11.1.1
xcopy /s %SOURCEDIR%\oracle_common\modules\oracle.javacache_11.1.1 %TARGETDIR%\Middleware\oracle_common\modules\oracle.javacache_11.1.1
xcopy /s %SOURCEDIR%\oracle_common\modules\oracle.jmx_11.1.1 %TARGETDIR%\Middleware\oracle_common\modules\oracle.jmx_11.1.1
xcopy /s %SOURCEDIR%\oracle_common\modules\oracle.jps_11.1.1 %TARGETDIR%\Middleware\oracle_common\modules\oracle.jps_11.1.1
xcopy /s %SOURCEDIR%\oracle_common\modules\oracle.jrf_11.1.1 %TARGETDIR%\Middleware\oracle_common\modules\oracle.jrf_11.1.1
xcopy /s %SOURCEDIR%\oracle_common\modules\oracle.mds_11.1.1 %TARGETDIR%\Middleware\oracle_common\modules\oracle.mds_11.1.1
xcopy /s %SOURCEDIR%\oracle_common\modules\oracle.nlsrtl_11.1.0 %TARGETDIR%\Middleware\oracle_common\modules\oracle.nlsrtl_11.1.0
xcopy /s %SOURCEDIR%\oracle_common\modules\oracle.odl_11.1.1 %TARGETDIR%\Middleware\oracle_common\modules\oracle.odl_11.1.1
xcopy /s %SOURCEDIR%\oracle_common\modules\oracle.webservices_11.1.1 %TARGETDIR%\Middleware\oracle_common\modules\oracle.webservices_11.1.1
xcopy /s %SOURCEDIR%\oracle_common\modules\oracle.wsm.common_11.1.1 %TARGETDIR%\Middleware\oracle_common\modules\oracle.wsm.common_11.1.1
xcopy /s %SOURCEDIR%\oracle_common\modules\oracle.xdk_11.1.0 %TARGETDIR%\Middleware\oracle_common\modules\oracle.xdk_11.1.0
xcopy /s %SOURCEDIR%\oracle_common\modules\datadirect_4.1 %TARGETDIR%\Middleware\oracle_common\modules\datadirect_4.1


xcopy /i %SOURCEDIR%\oracle_common\modules\oracle.ucp_11.1.0.jar %TARGETDIR%\Middleware\oracle_common\modules
xcopy /i %SOURCEDIR%\oracle_common\modules\org.apache.commons.beanutils_1.6.jar %TARGETDIR%\Middleware\oracle_common\modules
xcopy /i %SOURCEDIR%\oracle_common\modules\org.apache.commons.digester_1.8.jar %TARGETDIR%\Middleware\oracle_common\modules
xcopy /i %SOURCEDIR%\oracle_common\modules\org.apache.commons.logging_1.0.4.jar %TARGETDIR%\Middleware\oracle_common\modules
xcopy /i %SOURCEDIR%\oracle_common\modules\org.springframework_2.5.jar %TARGETDIR%\Middleware\oracle_common\modules
xcopy /i %SOURCEDIR%\oracle_common\modules\oracle.classloader_11.1.1.jar %TARGETDIR%\Middleware\oracle_common\modules
xcopy /i %SOURCEDIR%\oracle_common\modules\oracle.http_client_11.1.1.jar %TARGETDIR%\Middleware\oracle_common\modules
xcopy /i %SOURCEDIR%\oracle_common\modules\oracle.logging-utils_11.1.1.jar  %TARGETDIR%\Middleware\oracle_common\modules
xcopy /i %SOURCEDIR%\oracle_common\modules\commonj.sdo_2.1.0.jar  %TARGETDIR%\Middleware\oracle_common\modules
xcopy /i %SOURCEDIR%\oracle_common\modules\com.bea.core.xml.xmlbeans_2.1.0.0_2-5-1.jar  %TARGETDIR%\Middleware\oracle_common\modules
xcopy /i %SOURCEDIR%\oracle_common\modules\com.bea.core.weblogic.security.identity_1.1.2.1.jar  %TARGETDIR%\Middleware\oracle_common\modules

rem modules


mkdir %TARGETDIR%\Middleware\modules


xcopy /i %SOURCEDIR%\modules\com.bea.core.apache.commons.collections_3.2.0.jar %TARGETDIR%\Middleware\modules
xcopy /i %SOURCEDIR%\modules\com.bea.core.apache.commons.lang_2.1.0.jar %TARGETDIR%\Middleware\modules
rem 11.1.1.4
rem xcopy /i %SOURCEDIR%\modules\com.bea.core.weblogic.workmanager_1.9.0.0.jar %TARGETDIR%\Middleware\modules
xcopy /i %SOURCEDIR%\modules\com.bea.core.weblogic.workmanager_1.10.0.0.jar %TARGETDIR%\Middleware\modules
rem 11.1.1.4
rem xcopy /i %SOURCEDIR%\modules\com.bea.core.xml.beaxmlbeans_2.2.0.0_2-5-1.jar %TARGETDIR%\Middleware\modules
xcopy /i %SOURCEDIR%\modules\com.bea.core.xml.beaxmlbeans_2.3.0.0_2-5-1.jar %TARGETDIR%\Middleware\modules
xcopy /i %SOURCEDIR%\modules\commonj.sdo_1.0.0.0_1-0.jar %TARGETDIR%\Middleware\modules
xcopy /i %SOURCEDIR%\modules\glassfish.jaxb_1.0.0.0_2-1-12.jar %TARGETDIR%\Middleware\modules
xcopy /i %SOURCEDIR%\modules\javax.management.j2ee_1.0.jar %TARGETDIR%\Middleware\modules
xcopy /i %SOURCEDIR%\modules\javax.xml.rpc_1.2.1.jar %TARGETDIR%\Middleware\modules
rem 11.1.1.4
rem xcopy /i %SOURCEDIR%\modules\org.eclipse.persistence_1.0.0.0_2-1.jar %TARGETDIR%\Middleware\modules
xcopy /i %SOURCEDIR%\modules\org.eclipse.persistence_1.1.0.0_2-1.jar %TARGETDIR%\Middleware\modules

Some script files were taken from https://github.com/biemond/soa_tools/tree/master/ant and updated. The updated scriptfiles can be downloaded here; https://dl.dropbox.com/u/6693935/blog/buildtool11116scriptfiles.zip

Usage

In my case, the buildtool base dir is; D:\dev\buildtool11116. Here you can create a directory build-area (as specified in the build.properties file). In this directory you can create a structure like for example; https://dl.dropbox.com/u/6693935/blog/build-area.zip

The structure contains an example project, a deployment profile and configuration files. This is what the developer has to provide. The person installing the software has to update the following files;
- update deployAll.bat to reflect the correct locations (once)
- update build.properties to reflect the environment which is used (it is recommended the installer keeps per environment a separate build.properties file)

When deployAll.bat is executed, log files are generated in the logs directory and the deployed applications are put in the builds directory.

The build.xml is capable of calling unit tests, but since I got the following error which I couldn't fix quickly enough;

  [scatest] java.lang.NoClassDefFoundError: weblogic/security/subject/AbstractSubject

and

D:\dev\buildtool11116\Middleware\jdeveloper\bin\ant-sca-test.xml:111: Problem executing test cases
weblogic/security/subject/AbstractSubject

I've decided to put this functionality in comments and maybe look at it later. wlclient.jar contains this class and it should be in the classpath.

The example build-area provides a simple deployment of a SCA composite. The supplied build.xml provides functionality for MDS deployments and partitions (Edwin Biemond has described this functionality on his blog site). Some minor changes to the script might be needed to make sure deployment of the MDS is from the build-area folder so developers can more easily provide the required files. You can download the entire 11.1.1.6 buildtool here; https://dl.dropbox.com/u/6693935/blog/buildtool11116.zip.