Saturday, March 16, 2019

Using Python to performancetest an Oracle DB

Performance testing is a topic with many opinions and complexities. You can not do it in a way which will make everyone happy. It is not straightforward to compare measures before and after a change. Environments are often not stable (without change in itself and its environment). When performing a test, the situation at the start or end of the test are also often not the same. For example the test might write data in a database.

There are various ways to look at performance. You can look at user experience, generate load similar to what application usage produces or you can do more basic things like query performance. What will you be looking at? Resource consumption and throughput are the usual suspects.

I'll look at a simple example in this blog post. I'll change database parameters and look at throughput of various actions which are regularly performed on databases. This takes away the complexity of distributed systems. I used a single Python script for this which can be downloaded here.


Summary of conclusions: Exposing database functionality using a DAD is not so much influenced by the tested settings. Setting FILESYSTEMIO_OPTIONS to SETALL improved the performance of almost all database actions. This has also been observed at different customers. Disabling Transparent HugePages and enabling the database to use HugePages seemed to have little effect. PL/SQL native compilation also did not cause a massive improvement. From the tested settings FILESYSTEMIO_OPTIONS is the easiest to apply. Query performance and actions involving a lot of data improved with all (and any of) these settings.

Tuesday, February 26, 2019

Filesystem events to Elasticsearch / Kibana through Kafka Connect / Kafka

Filesystem events are useful to monitor. They can indicate a security breach. They can also help  understanding how a complex system works by looking at the files it reads and writes.

When monitoring events, you can expect a lot of data to be generated quickly. The events might be interesting to process for different systems and at a different pace. Also it would be nice if you could replay events from the start or a specific moment. Enter Kafka. In order to put the filesystem events in Kafka (from an output file), the Kafka Connect FileSourceConnector is used. In order to get the data from Kafka to Elasticsearch, the Kafka Connect ElasticsearchSinkConnector is used. Both connectors can be used without Enterprise license.

Saturday, February 16, 2019

Minikube on KVM on Linux Mint 19.1

In a previous blog post I wrote about running Minikube on Windows. I ended with the suggestion that getting Minikube working might be much easier on Linux. Thus I installed Linux Mint (as dual-boot) on my laptop and gave it a shot. The steps I took to get it working are described here.

Friday, February 15, 2019

Some challenges with Oracle Reports 12.2.1.3

Oracle Reports has been around for a long time and future versions will most likely not be created  (see here). Hence this is going to be my first and also last blog post on this product. Installing Reports is not an easy task. It requires several steps which are not well documented. This blog post contains a few pointers. The main source of inspiration is here.

Sunday, February 10, 2019

Minikube on Windows. Hyper-V vs Vagrant/VirtualBox

Kubernetes is a system for running and coordinating containerized applications across a cluster of machines. Minikube runs a single-node Kubernetes cluster and can be used for local development. In this blog post I'll compare 2 different ways to get a working Minikube environment on Windows based on experience with a workshop which we've created. One based on using Vagrant and VirtualBox (in which an Ubuntu environment is created) and one which uses Hyper-V (and an out of the box Minikube Linux distribution running on top). Do mind that many of the things in this blog post are a personal opinion.

Saturday, December 8, 2018

JVM performance: OpenJ9 uses least memory. GraalVM most. OpenJDK distributions differ.

In a previous blog post I created a setup to compare JVM performance of several JVMs. I received some valuable feedback on the measures I conducted and requests to add additional JVMs. In this second post I'll look at some more JVMs and I've added some measures like process memory usage and startup time. Also I've automated the test and reduced the complexity of the setup by removing haproxy and testing a single JVM at a time.

Friday, November 23, 2018

Comparing JVM performance; Zulu, OpenJDK, Oracle JDK, GraalVM CE

There are many different choices for a JVM for your Java application. Which would be the best to use? This depends on various factors. Performance being an important one. Solid performance research however is difficult. In this blog post I'll describe a setup I created to perform tests on different JVMs at the same time and show some interesting results! I also looked at the effect of resource isolation (assigning specific CPUs and memory to the process). This effect was negligible. My test application consisted of a reactive (non-blocking) Spring Boot REST application and I've used Prometheus to poll the JVMs and Grafana for visualization.

Below is an image of the used setup. Everything was running in Docker containers except SoapUI.