One of the benefits of using Docker is quick and easy provisioning. I wanted to find out first-hand if this could help me get an Oracle Enterprise Edition database quickly up and running for use in a development environment. Oracle provides Docker images for its Standard and Enterprise Edition database in the Oracle Container Registry. Lucas Jellema has already provided two blogs on this (here and here) which have been a useful starting point. In this blog I'll describe some of the choices to make and challenges I encountered. To summarize, I'm quite happy with the Docker images in the registry as they provide a very easy way to automate the install of an EE database. You can find a Vagrant provisioning shell script with the installation of Docker and Docker commands to execute here and a description on how to use it here.
Articles containing tips, tricks and nice to knows related to IT stuff I find interesting. Also serves as online memory.
Friday, December 29, 2017
Sunday, December 10, 2017
10 reasons NOT to implement Blockchain now
A secure distributed ledger with smart contract capabilities not requiring a bank as an intermediary! Also a single source of truth with complete traceability. Definitely something we want! Blockchain technology promises to make this possible. Blockchain became famous through cryptocurrency like Bitcoin and Ethereum. The technology could also be considered to replace B2B functionality. With new technologies it is not a bad idea to look at pro's and con's before starting an implementation. Blockchain is the new kid on the block and there is not much experience yet on how well he will play with others and will mature. In this blog I summarize some of my concerns concerning blockchain of which I hope will be solved in due time.
Regarding new/emerging technologies in the integration space, I'm quite open to investigate the potential value which they can offer. I'm a great proponent of for example Kafka, the highly scalable streaming platform and Docker to host microservices. However, I've been to several conferences and did some research online regarding blockchain and I'm sceptical. I definitely don't claim to be an expert on this subject so please correct me if I'm wrong! Also, this is my personal opinion. It might deviate from my employers and customers views.
Most of the issues discussed here are valid for public blockchains. Private blockchains are of course more flexible since they can be managed by companies themselves. You can for example more easily migrate private blockchains to a new blockchain technology or fix issues with broken smart contracts. These do require management tooling, scripts and enough developers / operations people around your private blockchain though. I don't think it is a deploy and go solution just yet.
Regarding new/emerging technologies in the integration space, I'm quite open to investigate the potential value which they can offer. I'm a great proponent of for example Kafka, the highly scalable streaming platform and Docker to host microservices. However, I've been to several conferences and did some research online regarding blockchain and I'm sceptical. I definitely don't claim to be an expert on this subject so please correct me if I'm wrong! Also, this is my personal opinion. It might deviate from my employers and customers views.
Most of the issues discussed here are valid for public blockchains. Private blockchains are of course more flexible since they can be managed by companies themselves. You can for example more easily migrate private blockchains to a new blockchain technology or fix issues with broken smart contracts. These do require management tooling, scripts and enough developers / operations people around your private blockchain though. I don't think it is a deploy and go solution just yet.
Tuesday, October 17, 2017
Quickly create a Virtualbox development VM with XE DB using Kickstart, Packer, Vagrant
The topic of quickly creating an Oracle development VM is not new. Several years ago Edwin Biemond and Lucas Jellema have written several blogs about this and have given presentations about the topics at various conferences. You can also download ready made Virtualbox images from Oracle here and specifically for SOA Suite here.
Over the years I have created a lot (probably 100+) of virtual machines manually. For SOA Suite, the process of installing the OS, installing the database, installing WebLogic Server, installing SOA Suite itself can be quite time consuming and boring if you have already done it so many times. Finally my irritation has passed the threshold that I need to automate it! I wanted easily recreate a clean environment with a new version of specific software. This blog is a start; provisioning an OS and installing the XE database on it. It might seem a lot but this blog contains the knowledge of two days work. This indicates it is relatively easy to get started with these things.
I decided to start from scratch and first create a base Vagrant box using Packer which uses Kickstart. Kickstart is used to configure the OS of the VM such as disk partitioning scheme, root password and initial packages. Packer makes using Kickstart easy and allows easy creation of a Vagrant base box. After the base Vagrant box was created, I can use Vagrant to create the Virtualbox machine, configure it and do additional provisioning such as in this case installing the Oracle XE database.
Over the years I have created a lot (probably 100+) of virtual machines manually. For SOA Suite, the process of installing the OS, installing the database, installing WebLogic Server, installing SOA Suite itself can be quite time consuming and boring if you have already done it so many times. Finally my irritation has passed the threshold that I need to automate it! I wanted easily recreate a clean environment with a new version of specific software. This blog is a start; provisioning an OS and installing the XE database on it. It might seem a lot but this blog contains the knowledge of two days work. This indicates it is relatively easy to get started with these things.
I decided to start from scratch and first create a base Vagrant box using Packer which uses Kickstart. Kickstart is used to configure the OS of the VM such as disk partitioning scheme, root password and initial packages. Packer makes using Kickstart easy and allows easy creation of a Vagrant base box. After the base Vagrant box was created, I can use Vagrant to create the Virtualbox machine, configure it and do additional provisioning such as in this case installing the Oracle XE database.
Sunday, September 24, 2017
Oracle SOA and WebLogic: Overview of key and keystore configuration
Keystores and the keys within can be used for security on the transport layer and application layer in Oracle SOA Suite and WebLogic Server. Keystores hold private keys (identity) but also public certificates (trust). This is important when WebLogic / SOA Suite acts as the server but also when it acts as the client. In this blog post I'll explain the purpose of keystores, the different keystore types available and which configuration is relevant for which keystore purpose.
Wednesday, August 23, 2017
R and the Oracle database: Using dplyr / dbplyr with ROracle in Windows 10
R uses data extensively. Data often resides in a database. In this blog I will describe installing and using dplyr, dbplyr and ROracle on Windows 10 to access data from an Oracle database and use it in R.
Friday, August 11, 2017
Oracle Mobile Cloud Service integration options
Oracle Mobile Cloud Service has a lot of options which allows it to integrate with other services and systems. Since it runs JavaScript on Node.js for custom APIs, it is very flexible.
Some features allow it to extent its own functionality such as the Firebase configuration option to send notifications to mobile devices, while for example the connectors allow wizard driven integration with other systems. The custom API functionality running on a recent Node.js version ties it all together. In this blog article I'll provide a quick overview and some background of the integration options of MCS.
MCS is very well documented here and there are many YouTube video's available explaining/demonstrating various MCS features here. So if you want to know more, I suggest looking at those.
Some features allow it to extent its own functionality such as the Firebase configuration option to send notifications to mobile devices, while for example the connectors allow wizard driven integration with other systems. The custom API functionality running on a recent Node.js version ties it all together. In this blog article I'll provide a quick overview and some background of the integration options of MCS.
MCS is very well documented here and there are many YouTube video's available explaining/demonstrating various MCS features here. So if you want to know more, I suggest looking at those.
Wednesday, July 26, 2017
Oracle Mobile Cloud Service (MCS) and Integration Cloud Service (ICS): How secure is your TLS connection?
In a previous blog I have explained which what cipher suites are, the role they play in establishing SSL connections and have provided some suggestions on how you can determine which cipher suite is a strong cipher suite. In this blog post I'll apply this knowledge to look at incoming connections to Oracle Mobile Cloud Service and Integration Cloud Service. Outgoing connections are a different story altogether. These two cloud services do not allow you control of cipher suites to the extend as for example Oracle Java Cloud Service and you are thus forced to use the cipher suite Oracle has chosen for you.
Why should you be interested in TLS? Well, 'normal' application authentication uses tokens (like SAML, JWT, OAuth). Once an attacker obtains such a token (and no additional client authentication is in place), it is more or less free game for the attacker. An important mechanism which prevents the attacker from obtaining the token is TLS (Transport Layer Security). The strength of the provided security depends on the choice of cipher suite. The cipher suite is chosen by negotiation between client and server. The client provides options and the server chooses the one which has its preference.
Disclaimer: my knowledge is not at the level that I can personally exploit the liabilities in different cipher suites. I've used several posts I found online as references. I have used the OWASP TLS Cheat Sheet extensively which provides many references for further investigation should you wish.
Why should you be interested in TLS? Well, 'normal' application authentication uses tokens (like SAML, JWT, OAuth). Once an attacker obtains such a token (and no additional client authentication is in place), it is more or less free game for the attacker. An important mechanism which prevents the attacker from obtaining the token is TLS (Transport Layer Security). The strength of the provided security depends on the choice of cipher suite. The cipher suite is chosen by negotiation between client and server. The client provides options and the server chooses the one which has its preference.
Disclaimer: my knowledge is not at the level that I can personally exploit the liabilities in different cipher suites. I've used several posts I found online as references. I have used the OWASP TLS Cheat Sheet extensively which provides many references for further investigation should you wish.
Monday, July 3, 2017
SSL/TLS: How to choose your cipher suite
For SSL/TLS connections, cipher suites determine for a major part how secure the connection will be. A cipher suite is a named combination of authentication, encryption, message authentication code (MAC) and key exchange algorithms used to negotiate the security settings (here). But what does this mean and how do you choose a secure cipher suite? The area of TLS is quite extensive and I cannot cover it in its entirety in a single blog post but I will provide some general recommendations based on several articles researched online. At the end of the post I'll provide some suggestions for strong ciphers for JDK8.
Friday, June 2, 2017
Oracle SOA Suite: Want performance? Don't log so much and clean up your database!
The Oracle SOA Suite infrastructure, especially composites, use the database intensively. Not only are the process definitions stored in the database, also a lot of audit information gets written there. The SOA infrastructure database, if not well managed, will grow and will eventually have detrimental effects on performance. In this blog post I will give some quick suggestions that will help you increase performance of your SOA Suite infrastructure on the database side by executing some simple scripts. These are some suggestions I have seen work at different customers. Not only do they help managing the SOA Suite data in the database, they will also lead to better SOA Suite performance.
Sunday, May 7, 2017
Oracle SOA Suite: Two-way SSL with TLS1.2 made easy (slightly less complicated)
Transport layer security is not an easy topic. Many blogs have been written about this already. Surprisingly though, I did not find a single blog which was more or less complete and provided me with everything I needed to know to get this working on SOA Suite 12.2.1. In this blog I try to make the topic more easy to understand and provide a complete end to end example.
Suppose you only want an implementation and do not care much about the explanation, you can skip the 'Some basics' section, only execute the commands in bold in the 'Lets get started!' section and the steps in the 'WebLogic and SOA Suite' section. Do take into consideration any existing SSL related configuration on your own system.
Saturday, April 22, 2017
R: Utilizing multiple CPUs
R is a great piece of software to perform statistical analyses. Computing power can however be a limitation. R by default uses only a single CPU. In almost every machine, multiple CPUs are present, so why not utilize them?
Friday, April 7, 2017
Oracle Mobile Cloud Service (MCS): An introduction to API security: Basic Authentication and OAuth2
As an integration/backend developer, when starting a project using Mobile Cloud Service, it is important to have some understanding of what this MBaaS (Mobile Backend as a Service) has to offer in terms of security features. This is important in order to be able to configure and test MCS. In this blog I will give examples on how to configure and use the basic authentication and OAuth2 features which are provided to secure APIs. You can read the Oracle documentation (which is quite good for MCS!) on this topic here.
Monday, March 27, 2017
Machine learning: Getting started with random forests in R
According to Gartner, machine learning is on top of the hype cycle at the peak of inflated expectations. There is a lot of misunderstanding about what machine learning actually is and what it can be done with it.
Machine learning is not as abstract as one might think. If you want to get value out of known data and do predictions for unknown data, the most important challenge is asking the right questions and of course knowing what you are doing, especially if you want to optimize your prediction accuracy.
In this blog I'm exploring an example of machine learning. The random forest algorithm. I'll provide an example on how you can use this algorithm to do predictions. In order to implement a random forest, I'm using R with the randomForest library and I'm using the iris dataset which is provided by the R installation.
Machine learning is not as abstract as one might think. If you want to get value out of known data and do predictions for unknown data, the most important challenge is asking the right questions and of course knowing what you are doing, especially if you want to optimize your prediction accuracy.
In this blog I'm exploring an example of machine learning. The random forest algorithm. I'll provide an example on how you can use this algorithm to do predictions. In order to implement a random forest, I'm using R with the randomForest library and I'm using the iris dataset which is provided by the R installation.
Monday, March 20, 2017
Oracle SOA Suite: Find that composite instance!
When executing BPM or BPEL processes, they are usually executed in the context of a specific entity. Sometimes you want to find instances involved with a specific entity. There are different ways to make this easy. You can for example use composite instance titles or sensors and set them to a unique identifier for your entity. If they have not been used, you can check the audit trail. However, manually checking the audit trail, especially if there are many instances, can be cumbersome. Also if different teams use different standards or standards have evolved over time, there might not be a single way to look for your entity identifier in composite instances. You want to automate this.
It is of course possible to write Java or WLST code and use the API to gather all relevant information. It would however require fetching large amounts of data from the SOAINFRA database to analyse. Fetching all that data into WLST or Java and combining it, would not be fast. I've created a database package / query which performs this feat directly on the 11g SOAINFRA database (and most likely with little alteration on 12c).
It is of course possible to write Java or WLST code and use the API to gather all relevant information. It would however require fetching large amounts of data from the SOAINFRA database to analyse. Fetching all that data into WLST or Java and combining it, would not be fast. I've created a database package / query which performs this feat directly on the 11g SOAINFRA database (and most likely with little alteration on 12c).
Monday, February 6, 2017
Oracle Service Bus: Produce messages to a Kafka topic
Oracle Service Bus is a powerful tool to provide features like transformation, throttling, virtualization of messages coming from different sources. There is a (recently opensourced!) Kafka transport available for Oracle Service Bus (see here). Oracle Service Bus can thus be used to do all kinds of interesting things to messages coming from Kafka topics. You can then produce altered messages to other Kafka topics and create a decoupled processing chain. In this blog I provide an example on how to use Oracle Service Bus to produce messages to a Kafka topic.
Saturday, February 4, 2017
Oracle Service Bus: Pipeline alerts in Splunk using SNMP traps
Oracle Service Bus provides a reporting activity called Alert. The OSB pipeline alerts use a persistent store. This store is file based. Changing the persistent store to JDBC based, does not cause pipeline alerts to be stored in a database instead of on disk. When the persistent store on disk becomes large, opening pipeline alerts in the Enterprise Manager (12c) or Service Bus console (11g) can suffer from poor performance. If you put an archive setting on pipeline alerts (see here), the space from the persistent store on disk is not reduced when alerts get deleted. You can compact the store to reduce space (see here), but this requires the store to be offline and this might require shutting down the Service Bus. This can be cumbersome to do often and is not good for your availability.
If you do not want to use the EM / SB console or have the issues with the filestore, there is an alternative. Pipeline alerts can produce SNMP traps. SNMP traps can be forwarded by a WebLogic SNMP Agent to an SNMP Manager. This manager can store the SNMP traps in a file and Splunk can monitor the file. Splunk makes searching alerts and visualizing them easy. In this blog I will describe the steps needed to get a minimal setup with SNMP traps going and how to see the pipeline alerts in Splunk.
If you do not want to use the EM / SB console or have the issues with the filestore, there is an alternative. Pipeline alerts can produce SNMP traps. SNMP traps can be forwarded by a WebLogic SNMP Agent to an SNMP Manager. This manager can store the SNMP traps in a file and Splunk can monitor the file. Splunk makes searching alerts and visualizing them easy. In this blog I will describe the steps needed to get a minimal setup with SNMP traps going and how to see the pipeline alerts in Splunk.
Thursday, January 12, 2017
WebLogic Server: Logging the SOAP action in the access.log
WebLogic Server allows you to customize your access.log. This can be very powerful if you want to monitor for example service response times in a tool like Splunk (see here). When working with SOAP services though, especially those with many operations, it can be insufficient to monitor services to the level of the individual endpoint. You want to also know with which intent the endpoint is called. In this blog I will show how this can be achieved.
Monday, January 9, 2017
Oracle Mobile Cloud Service (MCS). Implementing custom APIs using JavaScript on Node.js.
Oracle Mobile Cloud Service is a mobile backend as a service. MCS does its magic by providing a lot of features to make implementing mobile services easy such as (among many other) authentication, logging/analytics, lookups and calling other services. There are also features available to make integration with mobile clients easy such as providing an easy way to implement push notifications.
Personally I think one of the most powerful features of MCS is the ability to write custom JavaScript code and use that as an API implementation. This custom code can (among the regular JavaScript features) call MCS connectors and platform services. This provides a lot of flexibility in defining API behavior.
In this blog I will show how you can use this custom Node.js code to create an end to end example. I will use a RAML file to define my interface. Next I will define a connector in MCS to call the OpenWeatherMap API. This API returns (amongst other things) the temperature at a location in Kelvin. I want to define my own custom result message (with the temperature in Celsius) which better matches the requirements of my mobile client. I will use a custom JavaScript implementation to call the connector which calls the OpenWeatherMap API and create a custom response message from the result.
The described example is not suitable for a production implementation and is based on limited experience (and watching some really nice YouTube presentations). It is provided to give an idea on how to get started easily with a simple working example.
Personally I think one of the most powerful features of MCS is the ability to write custom JavaScript code and use that as an API implementation. This custom code can (among the regular JavaScript features) call MCS connectors and platform services. This provides a lot of flexibility in defining API behavior.
In this blog I will show how you can use this custom Node.js code to create an end to end example. I will use a RAML file to define my interface. Next I will define a connector in MCS to call the OpenWeatherMap API. This API returns (amongst other things) the temperature at a location in Kelvin. I want to define my own custom result message (with the temperature in Celsius) which better matches the requirements of my mobile client. I will use a custom JavaScript implementation to call the connector which calls the OpenWeatherMap API and create a custom response message from the result.
The described example is not suitable for a production implementation and is based on limited experience (and watching some really nice YouTube presentations). It is provided to give an idea on how to get started easily with a simple working example.