Saturday, March 17, 2012

Oracle SOA Suite Cluster part 1


Introduction

Most customers use clustered production environments for the SOA Suite to provide (amongst others) high availability and failover. The clustered environments are often only available in production and not in development or test. For developers, working with clustered environments, provide several challenges such as processes running simultaneously on different machines which can cause timing issues and locks.

In order to be able to identify these challenges and tackle them before they occur in the production environment, I wanted to setup a small cluster environment myself. What I wanted to achieve was that I could use the IP of the load balancer to deploy processes and that deployed processes (composites) could refer to the load balancer instead of to individual instances. In this way when a process calls another process, it could be on another Weblogic server instance.

I was interested in how the Weblogic instances would communicate and what would be synchronized between the instances. I choose to use one database for both Weblogic servers since I’m not much interested in the database functioning. Of course in a production environment, you would use a RAC setup for the database as well.

Below is a simple image of the initial idea. This might change when I learn more about this setup and the options available!


Building such a setup requires installation of
- Server and OS
- Database
- Load balancer
- Weblogic Server
- SOA Suite

Because this is a lot of material to cover, I've split this post up into several parts.

Setup

Server and Operating System

For the servers I've used Oracle Virtualbox. For the operating system I've used Oracle Enterprise Linux (5.8 x86_64). Make sure your swap file is at least 4Gb. Otherwise the installers (amongst others of the application server and database) might start complaining at a later stage. I've used 30Gb dynamically expanding HD images. 30Gb will be more then enough for my setup and since it is dynamically expanding, the images on disk will remain a lot smaller if disk space is not used. For memory I've done the following (my desktop PC only has 12Gb RAM);

- server DB: 3Gb
- server MW1: Adminserver + Managed domain 1 3Gb
- server MW2: Managed domain 2 2Gb

Also I've used DHCP (setup on a local router) to assign static IP's and hostnames based on MAC address. I've encountered several problems during the database installation which might have been related to this choice.

Several other things to keep in mind during the installation of Oracle Enterprise Linux;

Public Yum repository
Several packages are required (for the database installation and the Virtualbox guest additions) which can be downloaded from Oracle's public Yum repository. Oracle has provided usage instructions on http://public-yum.oracle.com.

I've used the following repositories; ol5_u8_base, ol5_UEK_base. After this you can do (as root);

yum install gcc make automake autoconf kernel-headers kernel-devel elfutils-libelf-devel gcc-c++ libstdc++-devel sysstat libaio-devel glibc-devel binutils compat-libstdc++-33 elfutils-libelf-devel gcc gcc-c++ glibc glibc-common libaio libaio-devel libgcc libstdc++

Then install the guest additions. Don't forget to add the oracle user to vboxsf in order to make using de VirtualBox shares easier.

Java installation
I've used a 64 bit JVM installation; http://www.oracle.com/technetwork/java/javase/downloads/jdk-6u31-download-1501634.html and I've added the below lines to /etc/profile

export JAVA_HOME=/usr/java/jdk1.6.0_31
pathmunge $JAVA_HOME/bin

Hosts file
I cloned the above base installation 2 times so I had my 3 servers. One would be used for the database and load balancer and the other two for the Weblogic server installations. 

Add your short hostname at the end of the /etc/hosts file if you use DHCP to get an IP address. 

Database

When using Oracle XE 11g database during a previous installation (11.1.1.6), I encountered several errors during the installation of the Oracle Enterprise Scheduler schema. I didn't save screenshots of this. To avoid future problems and to provide a more production like environment, I decided to use 11gR2 (11.2.0.3.0) Enterprise Edition database. This however requires more actions then installation of an XE database. 

In the below part I describe the actions I had to take after the initial installation to get the database to work quickly. I'm no expert on this topic!

I first installed the database using basic options (no RAC). The installer can generate scripts to update settings required for the database installation on your system. This will not solve all requirement issues, but makes the installation a lot easier.

/etc/profile
I've added the following lines to /etc/profile

export ORACLE_UNQNAME=orcl
export ORACLE_HOME=/u01/app/oracle/product/11.2.0/dbhome_1
export ORACLE_SID=orcl
pathmunge $ORACLE_HOME/bin

Hosts file
My /etc/hosts file looks like this;
127.0.0.1               localhost.localdomain localhost
192.168.1.160       misakidb

chkconfig --levels 345 sendmail off

Startup script
I changed /etc/oratab;
orcl:/u01/app/oracle/product/11.2.0/dbhome_1:Y

I used the following to create a base startup script for the database. XE installations provide this automatically but the Enterprise Edition doesn’t; http://www.oracle-base.com/articles/linux/AutomatingDatabaseStartupAndShutdownOnLinux.php.

For my environment I ended up with the following (I've added the starting of the Enterprise Manager to the script)

#!/bin/sh
# chkconfig: 345 99 10
# description: Oracle auto start-stop script.
#
# Set ORA_HOME to be equivalent to the $ORACLE_HOME
# from which you wish to execute dbstart and dbshut;
#
# Set ORA_OWNER to the user id of the owner of the
# Oracle database in ORA_HOME.

ORA_HOME=/u01/app/oracle/product/11.2.0/dbhome_1
ORA_OWNER=oracle

if [ ! -f $ORA_HOME/bin/dbstart ]
then
    echo "Oracle startup: cannot start"
    exit
fi

case "$1" in
    'start')
        # Start the Oracle databases:
        # The following command assumes that the oracle login
        # will not prompt the user for any values
        su - $ORA_OWNER -c "$ORA_HOME/bin/dbstart $ORA_HOME"
        su - $ORA_OWNER -c "$ORA_HOME/bin/emctl start dbconsole"
        touch /var/lock/subsys/dbora
        ;;
    'stop')
        # Stop the Oracle databases:
        # The following command assumes that the oracle login
        # will not prompt the user for any values
        su - $ORA_OWNER -c "$ORA_HOME/bin/emctl stop dbconsole"
        su - $ORA_OWNER -c "$ORA_HOME/bin/dbshut $ORA_HOME"
        rm -f /var/lock/subsys/dbora
        ;;
esac

Database configuration
I used http://www.oracle-base.com/articles/misc/OracleNetworkConfiguration.php to get a working tnsnames.ora, sqlnet.ora and listener.ora. I ended up with the following;

tnsnames.ora
ORCL.WORLD =
  (DESCRIPTION =
    (ADDRESS = (PROTOCOL = TCP)(HOST = misakidb)(PORT = 1521))
    (CONNECT_DATA =
      (SERVICE_NAME = ORCL.WORLD)
    )
  )

sqlnet.ora
NAMES.DIRECTORY_PATH= (TNSNAMES, ONAMES, HOSTNAME)
NAMES.DEFAULT_DOMAIN = WORLD
ADR_BASE = /u01/app/oracle

listener.ora
LISTENER =
  (DESCRIPTION_LIST =
    (DESCRIPTION =
      (ADDRESS = (PROTOCOL = TCP)(HOST = misakidb)(PORT = 1521))
    )
  )

SID_LIST_LISTENER =
  (SID_LIST =
    (SID_DESC =
      (GLOBAL_DBNAME = ORCL.WORLD)
      (ORACLE_HOME = /u01/app/oracle/product/11.2.0/dbhome_1)
      (SID_NAME = ORCL)
    )
  )

ADR_BASE_LISTENER = /u01/app/oracle