Inform the DBA if Weblogic can't connect to a database. Automated by using Jenkins, Maven and WLST.

As described in a previous post (http://javaoraclesoa.blogspot.nl/2013/08/wlst-obtaining-parameters-recovering.html), by using a WLST script it is possible to check if a Weblogic server can connect to all databases which are configured in datasources. In this post I'll describe how this script can be used in Jenkins to automatically inform the DBA to fix their database issues. The focus will be on the left part of the below image.

The SSHExec Ant task is used to connect to a Linux/Unix machine where the Weblogic server is running. The SSHExec Ant task then executes the WLST script which checks datasource/database availability (for every managed server). To call the task with different parameters for the different environments, Maven profiles are used in a wrapper POM file. This POM is then called by using the Maven invoker plugin for every profile. In order to not make the build fail on the first database which is down, the log file in Jenkins is parsed afterwards using the Logparser plugin (https://wiki.jenkins-ci.org/display/JENKINS/Log+Parser+Plugin).

The Maven project, resume.py script and log parsing rules can be downloaded here; https://dl.dropboxusercontent.com/u/6693935/blog/checkdatasources.zip

Implementation

Invoker Maven POM

The invoker Maven POM is shown below. This POM calls the POM which actually does the work and contains the profile information (${basedir}/sshutils/pom.xml).

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>

<groupId>ms.maven.test.caller</groupId>
<artifactId>sshutils_caller</artifactId>
<version>1.0-SNAPSHOT</version>
<packaging>jar</packaging>

<name>sshutils_caller</name>
<url>http://maven.apache.org</url>

<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
</properties>

<build>
<plugins>
<plugin>
<artifactId>maven-invoker-plugin</artifactId>
                <version>1.8</version>
<configuration> 
<goals> 
<goal>validate</goal> 
</goals> 
<streamLogs>true</streamLogs>
                    <cloneProjectsTo>${project.build.directory}/sshutils</cloneProjectsTo>
</configuration> 
<executions> 
<execution> 
<id>sshutils_dev</id> 
<goals> 
<goal>run</goal> 
</goals>
<phase>validate</phase> 
<configuration> 
<pom>${basedir}/sshutils/pom.xml</pom> 
<profiles>
<profile>dev</profile>
</profiles>
</configuration> 
</execution>
<execution> 
<id>sshutils_tst</id> 
<goals> 
<goal>run</goal> 
</goals>
<phase>validate</phase> 
<configuration> 
<pom>${basedir}/sshutils/pom.xml</pom> 
<profiles>
<profile>tst</profile>
</profiles>
</configuration> 
</execution>
</executions> 
</plugin>
</plugins>
</build>
</project>

Profile POM

This POM is called for every profile. It logs in by using an SSH connection on the server running Weblogic and executes the script which checks the databases.

To be able to see stderr on the console,  2&gt;&amp;1 is used after the commandline.

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>

<groupId>ms.maven.test.profile</groupId>
<artifactId>sshutils</artifactId>
<version>1.0-SNAPSHOT</version>
<packaging>jar</packaging>

<name>sshutils</name>
<url>http://maven.apache.org</url>

<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
</properties>

<profiles>
<profile>
<id>dev</id>
            <properties>
                <hostname>localhost</hostname>
<sshport>2222</sshport>
<ux_username>oracle</ux_username>
<ux_password>oracle</ux_password>
<weblogic_username>weblogic</weblogic_username>
<weblogic_password>welcome1</weblogic_password>
<wlst_path>/oracle/fmwhome/wlserver_10.3/common/bin/wlst.sh</wlst_path>
<script_path>/home/oracle/resume.py</script_path>
            </properties>
    </profile>
<profile>
<id>tst</id>
            <properties>
                <hostname>localhost</hostname>
<sshport>2222</sshport>
<ux_username>oracle</ux_username>
<ux_password>oracle</ux_password>
<weblogic_username>weblogic</weblogic_username>
<weblogic_password>welcome1</weblogic_password>
<wlst_path>/oracle/fmwhome/wlserver_10.3/common/bin/wlst.sh</wlst_path>
<script_path>/home/oracle/resume.py</script_path>
</properties>
    </profile>
</profiles>

<dependencies>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>3.8.1</version>
<scope>test</scope>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<inherited>false</inherited>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-antrun-plugin</artifactId>
<version>1.6</version>
<executions>
<execution>
<id>sshexec</id>
<phase>validate</phase>
<configuration>
<target>
<echo>Connecting to ${hostname}</echo>
<!-- calls deploy script -->
<sshexec host="${hostname}" port="${sshport}" trust="yes"
               username="${ux_username}" password="${ux_password}"
               command="${wlst_path} ${script_path} --username=${weblogic_username} --password=${weblogic_password}  2&gt;&amp;1" verbose="true"
        failonerror="false"/>

<!-- SSH -->
<taskdef name="sshexec"
               classname="org.apache.tools.ant.taskdefs.optional.ssh.SSHExec"
               classpathref="maven.plugin.classpath" />
</target>
</configuration>
<goals>
<goal>run</goal>
</goals>
</execution>
</executions>
<dependencies>
<dependency>
<groupId>commons-net</groupId>
<artifactId>commons-net</artifactId>
<version>1.4.1</version>
</dependency>
<dependency>
<groupId>ant</groupId>
<artifactId>ant-commons-net</artifactId>
<version>1.6.5</version>
</dependency>
<dependency>
<groupId>ant</groupId>
<artifactId>ant-jsch</artifactId>
<version>1.6.5</version>
</dependency>
<dependency>
<groupId>com.jcraft</groupId>
<artifactId>jsch</artifactId>
<version>0.1.29</version>
</dependency>
</dependencies>
</plugin>
</plugins>
</build>
</project>

Log parser plugin

The Jenkins logparser plugin allows parsing of the console output after the run has completed. This allows the build to fail after every datasource has been checked instead of the default behavior (which is overwritten in the profile POM by failonerror="false"); after the first failed datasource.

The log parser plugin can easily be installed in Jenkins and requires a parsing rules file. In this case, I created a simple file;

error /ERROR/
error /Connection refused/

This changes the status of the build to error if the text ERROR or Connection refused is encountered. Connection refused can occur when the SSH connection fails. In Jenkins this looks like below;

If all databases referred to by datasources can be logged in on all available Weblogic servers, the status is succesful. This looks like below;

If a database is not available, it looks like;

The Jenkins Job can be configured to notify people on Build failure. If the job is scheduled to for example every morning just before the DBA arrives, if you're lucky, the problems are solved before you want to start your workday.