Apache Camel + Spring Boot: Different components to expose HTTP endpoints

Apache Camel is an open source integration framework that allows you to integrate technologically diverse systems using a large library of components. A common use-case is to service HTTP based endpoints. Those of course come in several flavors and there is quite a choice in components to use.

In this blog post I'll take a look at what is available and how they differ with respect to flexibility to define multiple hosts, ports and URLs to host from using a single CamelContext. Depending on your use-case you will probably be using one of these. You can find my sample project here.

Java Microservices: What do you need to tweak to optimize throughput and response times

Performance tuning usually goes something like followed:

• a performance problem occurs
• an experienced person knows what is probably the cause and suggests a specific change
• baseline performance is determined, the change is applied, and performance is measured again
• if the performance has improved compared to the baseline, keep the change, else revert the change
• if the performance is now considered sufficient, you're done. If not, return to the experienced person to ask what to change next and repeat the above steps

This entire process can be expensive. Especially in complex environments where the suggestion of an experienced person is usually a (hopefully well informed) guess. This probably will require quite some iterations for the performance to be sufficient. If you can make these guesses more accurate by augmenting this informed judgement, you can potentially tune more efficiently.

In this blog post I'll try to do just that. Of course a major disclaimer applies here since every application, environment, hardware, etc is different. The definition of performance and how to measure it is also something which you can have different opinions on. In short what I've done is look at many different variables and measuring response times and throughput of minimal implementations of microservices for every combination of those variables. I fed all that data to a machine learning model and asked the model which variables it used to do predictions of performance with. I also presented on this topic at UKOUG Techfest 2019 in Brighton, UK. You can view the presentation here.

Oracle Database: Write arbitrary log messages to the syslog from PL/SQL

Syslog is a standard for message logging, often employed in *NIX environments. It allows separation of the software that generates messages, the system that stores them, and the software that reports and analyzes them. Each message is labeled with a facility code, indicating the software type generating the message, and assigned a severity level.

In *NIX systems syslog messages often end up in /var/log/messages. You can configure these messages to be forwarded to remote syslog daemons. Also a pattern which often seen is that the local log files are monitored and processed by an agent.

Oracle database audit information can be send to the syslog daemon. See for example the audit functionality. If you however want to use a custom format in the syslog or write an entry to the syslog which is not related to an audit action, this functionality will not suffice. How to achieve this without depending on the audit functionality is described in this blog post. PL/SQL calls database hosted Java code. This code executes an UDP call to the local syslog. You can find the code here.

Calling an Oracle DB stored procedure from Spring Boot using Apache Camel

There are different ways to create data services. The choice for a specific technology to use, depends on several factors inside the organisation which wishes to realize these services. In this blog post I'll provide a minimal sample on how you can use Spring Boot with Apache Camel to call an Oracle database procedure which returns the result of an SQL query as an XML. You can browse the code here.

Microservice framework startup time on different JVMs

When developing microservices, a fast startup time is useful. It can for example reduce the amount of time a rolling upgrade of instances takes and reduce build time thus shortening development cycles. When running your code using a 'serverless' framework such as for example Knative or FnProject, scaling and getting the first instance ready is faster.

When you want to reduce startup time, an obvious thing to look at is ahead of time (AOT) compilation such as provided as an early adopter plugin as part of GraalVM. Several frameworks already support this out of the box such as Helidon SE, Quarkus and Micronaut. Spring will probably follow with version 5.3 Q2 2020. AOT code, although it is fast to startup, still shows differences per framework. Which framework produces the native executable which is fastest to start?

If you need specific libraries which cannot be natively compiled (not even when using the Tracing Agent), using Java the old-fashioned JIT way is also an option. You will not achieve start-up times near AOT start-up times but by choosing the right framework and JVM, it can still be acceptable.

In this blog post I'll provide some measures which I did on start-up times of minimal implementations of several frameworks and an implementation with only Java SE. I've looked at both JIT and AOT (wherever this was possible) and ran the code on different JVMs.

Apache Camel and Spring Boot: Calling multiple services in parallel and merging results

Sometimes you have multiple services you want to call at the same time and merge their results when they're all in (or after a timeout). In Enterprise Integration Patterns (EIP) this is a Splitter followed by an Aggregator. I wanted to try and implement this in Spring Boot using Apache Camel so I did. Since this is my first quick try at Apache Camel, I might not have followed much best practices. I used sample code from Baeldungs blog, combined it with this sample of sending parallel requests using Futures. You can browse my code here.

A transparent Spring Boot REST service to expose Oracle Database logic

Sometimes you have an Oracle database which contains a lot of logic and you want to expose specific logic as REST services. There are a variety of ways to do this. The most obvious one to consider might be Oracle REST Data Services. It is quite powerful and supports multiple authentication mechanisms like OAuth. Another option might be using the database embedded PL/SQL gateway This gateway however is deprecated for APEX and difficult to tune (believe me, I know).

Sometimes there are specific requirements which make the above solutions not viable. For example if you have complex custom authentication logic implemented elsewhere which might be difficult to translate to ORDS or the embedded PL/SQL gateway. ORDS also runs in stand-alone in a Docker container but this is not so easy for the PL/SQL gateway. Also if you are looking for a product or framework which can be used for multiple flavors of database, these solutions might be too Oracle specific.

You can consider creating your own custom service in for example Java. The problem here however is that it is often tightly coupled with the implementation. If for example parameters of a database procedure are mapped to Java objects or a translation from a view to JSON takes place in the service, there is often a tight coupling between the database code and the service.

In this blog post I'll provide a solution for a transparent Spring Boot REST service which forwards everything it receives to the database for further processing without this tight coupling, only to to a generic database procedure to handle all REST requests. The general flow of the solution is as follows:

• The service receives an HTTP request from a client
• Service translates the HTTP request to an Oracle database REST_REQUEST_TYPE object type
• Service calls the Oracle database over JDBC with this Object
• The database processes the REST_REQUEST_TYPE and creates a REST_RESPONSE_TYPE Object
• The database returns the REST_RESPONSE_TYPE Object to the service
• The service translates the REST_RESPONSE_TYPE Object to an HTTP response
• The HTTP response is returned to the client

Graceful shutdown of forked workers in Python and JavaScript running in Docker containers

You might encounter a situation where you want to fork a script during execution. For example if the amount of forks is dependent on user input or another specific situation. I encountered such a situation in which I wanted to put load on a service using multiple concurrent processes. In addition, when running in a docker container, only the process with PID=1 receives a SIGTERM signal. If it has terminated, the worker processes receive a SIGKILL signal and are not allowed a graceful shutdown. In order to do a graceful shutdown of the worker processes, the main process needs to manage them and only exit after the worker processes have terminated gracefully. Why do you want processes to be terminated gracefully? In my case because I store performance data in memory (disk is too slow) and only write the data to disk when the test has completed.

This seems relatively straightforward, but there are some challenges. Also I implemented this in JavaScript running on Node and in Python. Python and JavaScript handle forking differently.

Performance! 3 reasons to stick to Java 8 for the moment

It is a smart thing to move to newer versions of Java! Support such as security updates and new features are just two of them but there are many more. Performance might be a reason to stick to Java 8 though. In this blog post I'll show some results of performance tests I have conducted showing Java 11 has slower startup times and slightly slower throughput compared to Java 8 when using the same Java code. Native images (a GraalVM feature) have greatly reduced startup time and memory usage at the cost of throughput. You can only compile Java 8 byte-code to a native image though (at the moment).

6 tips to make your life with Vagrant even better!

HashiCorp Vagrant is a great tool to quickly get up and running with a development environment. In this blog post I'll give some tips to make your life with Vagrant even better! You can find an example which uses these tips here.

Using Python to performancetest an Oracle DB

Performance testing is a topic with many opinions and complexities. You can not do it in a way which will make everyone happy. It is not straightforward to compare measures before and after a change. Environments are often not stable (without change in itself and its environment). When performing a test, the situation at the start or end of the test are also often not the same. For example the test might write data in a database.

There are various ways to look at performance. You can look at user experience, generate load similar to what application usage produces or you can do more basic things like query performance. What will you be looking at? Resource consumption and throughput are the usual suspects.

I'll look at a simple example in this blog post. I'll change database parameters and look at throughput of various actions which are regularly performed on databases. This takes away the complexity of distributed systems. I used a single Python script for this which can be downloaded here.

Summary of conclusions: Exposing database functionality using a DAD is not so much influenced by the tested settings. Setting FILESYSTEMIO_OPTIONS to SETALL improved the performance of almost all database actions. This has also been observed at different customers. Disabling Transparent HugePages and enabling the database to use HugePages seemed to have little effect. PL/SQL native compilation also did not cause a massive improvement. From the tested settings FILESYSTEMIO_OPTIONS is the easiest to apply. Query performance and actions involving a lot of data improved with all (and any of) these settings.

Filesystem events to Elasticsearch / Kibana through Kafka Connect / Kafka

Filesystem events are useful to monitor. They can indicate a security breach. They can also help  understanding how a complex system works by looking at the files it reads and writes.

When monitoring events, you can expect a lot of data to be generated quickly. The events might be interesting to process for different systems and at a different pace. Also it would be nice if you could replay events from the start or a specific moment. Enter Kafka. In order to put the filesystem events in Kafka (from an output file), the Kafka Connect FileSourceConnector is used. In order to get the data from Kafka to Elasticsearch, the Kafka Connect ElasticsearchSinkConnector is used. Both connectors can be used without Enterprise license.

Minikube on KVM on Linux Mint 19.1

In a previous blog post I wrote about running Minikube on Windows. I ended with the suggestion that getting Minikube working might be much easier on Linux. Thus I installed Linux Mint (as dual-boot) on my laptop and gave it a shot. The steps I took to get it working are described here.

Some challenges with Oracle Reports 12.2.1.3

Oracle Reports has been around for a long time and future versions will most likely not be created  (see here). Hence this is going to be my first and also last blog post on this product. Installing Reports is not an easy task. It requires several steps which are not well documented. This blog post contains a few pointers. The main source of inspiration is here.

Minikube on Windows. Hyper-V vs Vagrant/VirtualBox

Kubernetes is a system for running and coordinating containerized applications across a cluster of machines. Minikube runs a single-node Kubernetes cluster and can be used for local development. In this blog post I'll compare 2 different ways to get a working Minikube environment on Windows based on experience with a workshop which we've created. One based on using Vagrant and VirtualBox (in which an Ubuntu environment is created) and one which uses Hyper-V (and an out of the box Minikube Linux distribution running on top). Do mind that many of the things in this blog post are a personal opinion.