Jenkins Pipeline: SonarQube and OWASP Dependency-Check

The OWASP top 10 has listed the following vulnerability for several years (at least in 2013 and 2017): using components with known vulnerabilities. But software nowadays can be quite complex consisting of many dependencies. How do you know the components and versions of those components do not contain known vulnerabilities? Luckily the OWASP foundation has also provided a dependency-check tool with plugins for various languages to make detecting this more easy. In this blog post I'll show how you can incorporate this in a Jenkins pipeline running on Kubernetes and using Jenkens and SonarQube to display the results of the scan.

Kubernetes: Building and deploying a Java service with Jenkins

Kubernetes has become the de facto container orchestration platform to run applications on. Java applications are no exception to this. When using a PaaS provider to give you a hosted Kubernetes, sometimes that provider also provides a CI/CD solution. However this is not always the case. When hosting Kubernetes yourself, you also need to implement a CI/CD solution.

Jenkins is a popular tool to use when implementing CI/CD solutions. Jenkins can also run quite easily in a Kubernetes environment. When you have Jenkins installed, you need to have a Git repository to deploy your code from, a Jenkins pipeline definition, tools to wrap your Java application in a container, a container registry to deploy your container to and some files to describe how the container should be deployed and run on Kubernetes. In this blog post I'll describe a simple end-to-end solution to deploy a Java service to Kubernetes. This is a minimal example so there is much room for improvement. It is meant to get you started quickly.

SOA Suite 12c: Best practices for project structure and deployment

Efficient usage of version control has specific requirements to allow identification of versions and synchronous development on different branches. Design time you will want to have your Service Bus projects in a single application in order to allow usage of shared objects. At deploy-time or when creating a release, you want to group SCA composites together with Service Bus projects. How do you combine these different requirements?

In this article I'll describe several practices and considerations which can help you structuring your version control and artifact repository. The main challenge is finding a workable balance between the amount/complexity your deployment scripts and developer productivity / focus on business value. A lot of scripts (large investment) can make it easy for developers on the short term, however those scripts can easily become a burden.

If you are just looking for some good practices to structure your version control and artifact repository, look at the list below. If however you want to know why I think certain things are good and bad practice, read on.

SOA Suite 12c: Collect & Deploy SCA composites & Service Bus artifacts using Maven

An artifact repository has many benefits for collaboration and governance of artifacts. In this blog post I will illustrate how you can fetch SCA composites and Service Bus artifacts from an artifact repository and deploy them. The purpose of this exercise is to show that you do not need loads of custom scripts to do these simple tasks. Why re-invent a wheel when Oracle already provides it?

This example has been created for SOA Suite 12.1.3. This will not work as-is for 11g and earlier since they lack Maven support for SOA Suite artifacts. In order to start using Maven to do command-line deployments, you need to have some Oracle artifacts in your repository. See http://biemond.blogspot.nl/2014/06/maven-support-for-1213-service-bus-soa.html on how to put them there. I have used two test projects which were already in the repository. A SCA composite called HelloWorld_1.0 and a Service Bus project also called HelloWorld_1.0. In my example, the SCA composite is in the GroupId nl.amis.smeetsm.composite and the Service Bus project is in the GroupId nl.amis.smeetsm.servicebus.

Deploying SOA Suite 12c artifacts from Nexus

SOA Suite 12c introduces Maven support to build and deploy artifacts. Oracle has provided extensive documentation on this. Also there already are plenty of blog posts describing how to do this. I will not repeat those posts (only shortly describe the steps). What I couldn't find quickly enough though was how to deploy artifacts from an artifact repository to an environment. This is a task often done by provisioning software such as Puppet or Jenkins. Sometimes though you want to do this from a command-line. In this post I'll briefly describe steps required to get your Continuous Delivery efforts going and how to deploy an artifact from the Nexus repository to a SOA Suite runtime environment.

Sonatype Nexus: Retrieving artifacts using the REST API or Apache Ivy

Sonatype Nexus is an often used artifact repository. In a previous blog post I have shown an example how Maven can be used to assemble and release artifacts to Nexus. In this blog post I will describe two ways how artifacts can be fetched from the Nexus repository; by using the REST API and by using Apache Ivy.

Java Unit testing: Mocking JNDI, final classes, statics and private parts

To improve quality of an application, it is a good practice to provide unit tests together with the code which is written. This makes it easier to determine the impact of changes and fix them early during development.

Writing unit tests can be challenging. Often applications run inside an application server which provides services and a unit test runs outside of the server. Applications can depend on the services provided by the application server which makes testing outside of this scope difficult. One of those services is JNDI. JNDI makes it possible for an application to look up for example a datasource. Luckily JNDI can be provided outside of application server scope.

To make mocking methods in unit tests easier, the Mockito framework (https://code.google.com/p/mockito/) can be used. Mockito however uses a subclassing mechanism which does not work when dealing with final classes. Also static and private methods cannot be mocked because of this. PowerMock provides solutions for those issues; https://code.google.com/p/powermock/.

Sending HTML reports of JMeter integration test results from Maven

In this post I'll describe how I automated executing integration tests by using JMeter for several environments and sending integration test reports (HTML).

To accomplish this I've used Maven with several plugins;

- jmeter-maven-plugin (http://wiki.apache.org/jmeter/JMeterMavenPlugin) for executing JMeter tests from Maven
- xml-maven-plugin (http://mojo.codehaus.org/xml-maven-plugin/) for transforming the resulting JTL file to a readable HTML report
- exec-maven-plugin (http://mojo.codehaus.org/exec-maven-plugin/) for calling scripts
- maven-postman-plugin (http://doc.fortysix.ch/maven/maven-postman-plugin/) for sending e-mails

I've also used conditional profiles which act on the result from previous build steps to only send a mail in case of errors. In the following post it is described you shouldn't be doing this; http://www.blackbuild.com/how-to-really-use-maven-profiles-without-endangering-your-karma/ ('Using a profile should not result in a different artifact than building without the profile.') but I did it anyway because it seemed useful in this case.

I encountered several challenges which I will describe in this post and how I fixed them. The sample code can be downloaded here; https://dl.dropboxusercontent.com/u/6693935/blog/jmeter-test.zip

Inform the DBA if Weblogic can't connect to a database. Automated by using Jenkins, Maven and WLST.

As described in a previous post (http://javaoraclesoa.blogspot.nl/2013/08/wlst-obtaining-parameters-recovering.html), by using a WLST script it is possible to check if a Weblogic server can connect to all databases which are configured in datasources. In this post I'll describe how this script can be used in Jenkins to automatically inform the DBA to fix their database issues. The focus will be on the left part of the below image.

The SSHExec Ant task is used to connect to a Linux/Unix machine where the Weblogic server is running. The SSHExec Ant task then executes the WLST script which checks datasource/database availability (for every managed server). To call the task with different parameters for the different environments, Maven profiles are used in a wrapper POM file. This POM is then called by using the Maven invoker plugin for every profile. In order to not make the build fail on the first database which is down, the log file in Jenkins is parsed afterwards using the Logparser plugin (https://wiki.jenkins-ci.org/display/JENKINS/Log+Parser+Plugin).

The Maven project, resume.py script and log parsing rules can be downloaded here; https://dl.dropboxusercontent.com/u/6693935/blog/checkdatasources.zip