Node.js and Oracle NoSQL Database

Oracle NoSQL Database is an interesting option to consider when you want a schemaless, fast, scale-able database which can provide relaxed (eventual) consistency. Oracle provides a Node.js driver for this database. In this blog I'll describe how to install Oracle NoSQL database and how to connect to it from a Node.js application.

The Node.js driver provided by Oracle is currently in preview version 3.3.7. It uses NoSQL client version 12.1.3.3.4 which does not work with 4.x versions of NoSQL database, so I downloaded Oracle NoSQL Database, Enterprise Edition 12cR1 (12.1.3.3.5) from here (the version number was closest to the version number of the client software).

NoSQL installation

To get NoSQL Database up and running, I followed the steps described in the installation manual here. This was quite educational as it showed me the parts a NoSQL database consists of. If you want to do a quick installation, you can do the steps described here. Do mind that the {} variable references do not work in every shell. On Oracle Linux 7.2 (inside Virtualbox) I did the following. First download kv-ee-3.3.5.zip here (previous versions on the Oracle NoSQL download page).

I made sure a Java was installed. I used Oracle Java 1.8 64 bit.

I created the user oracle
mkdir -p /home/oracle/nosqldb/data
mkdir -p /home/oracle/nosqldb/root
cd /home/oracle/nosqldb
unzip kv-ee-3.3.5.zip

I updated /etc/environment
KVHOME="/home/oracle/nosqldb/kv-3.3.5"
KVDATA="/home/oracle/nosqldb/data"
KVROOT="/home/oracle/nosqldb/root"

Logout and login to make the environment active

Next I created an initial boot config

java -jar $KVHOME/lib/kvstore.jar makebootconfig -root $KVROOT -store-security none -capacity 1 -harange 5010,5030 -admin 5001 -port 5000 -memory_mb 1024 -host localhost -storagedir $KVDATA

I started the Oracle NoSQL Database Storage Agent(SNA).

nohup java -Xmx256m -Xms256m -jar $KVHOME/lib/kvstore.jar start -root $KVROOT &

And I configured it:

java -Xmx256m -Xms256m -jar $KVHOME/lib/kvstore.jar runadmin -port 5000 -host localhost <<EOF
configure -name mystore
plan deploy-zone -name "LocalZone" -rf 1 -wait
plan deploy-sn -zn zn1 -host localhost -port 5000 -wait
plan deploy-admin -sn sn1 -port 5001 -wait
pool create -name LocalPool
show topology
pool join -name LocalPool -sn sn1
topology create -name topo -pool LocalPool -partitions 10
topology preview -name topo
plan deploy-topology -name topo -wait
show plan
EOF

You can test if it is running by executing:

java -Xmx256m -Xms256m -jar $KVHOME/lib/kvstore.jar ping -port 5000 -host localhost

Oracle NoSQL Database basics

NoSQL Database has built-in load balancing, sharding and several other features related to high availability quite clearly integrated as an essential part of the software and not as some kind of add-on. Read more about it here.

Also different consistency models are available. You can sacrifice immediate consistency to gain more performance. Read more about that here.

It provides an Admin console to look at the topology and execution of plans. The console does not allow you to do actual changes to the configuration. For that you can use a CLI.

It also allows you to browse log files

And look at performance details for specific nodes.

For development, kvlite available. It is a simple single node store which can be used locally. Read more here. When using kvlite, you do not need to do the configuration as described in the installation.

Node.js application

Installation of the Node.js module is easy. Just do a npm install nosqldb-oraclejs. The NoSQL Node.js driver page gives a piece of code which you can use to test your application. The default installation of NoSQL however as described above causes a port conflict with the proxy server which is started. This port conflict is not immediately clear as it gives you the below exception.

{ NoSQLDB-ConnectionError: Error with NoSQL DB Connection: Error verifying the proxy connection
NoSQLDB-ConnectionError: Error with NoSQL DB Connection: Connection timeout
    at /home/oracle/nodejsnosql/node_modules/nosqldb-oraclejs/lib/store.js:277:25
    at Timeout._onTimeout (/home/oracle/nodejsnosql/node_modules/nosqldb-oraclejs/lib/store.js:181:7)
    at tryOnTimeout (timers.js:228:11)
    at Timer.listOnTimeout (timers.js:202:5)

I changed the port and some log settings and used this test application. When I run it with node app.js I get the following output:

[2016-08-16 19:05:02.717] [INFO]  [at Object.startProxy (/home/oracle/nodejsnosql/node_modules/nosqldb-oraclejs/lib/proxy.js:353:10)] [PROXY] Start proxy

Connected to store
Table is created.
Inserting data...
Reading data...
Writing row #0
Writing row #1
Reading row #0
{ id: 0, name: 'name #0' }
Reading row #1
{ id: 1, name: 'name #1' }
Closing connection...
[2016-08-16 19:05:09.630] [INFO]  [at Store.close (/home/oracle/nodejsnosql/node_modules/nosqldb-oraclejs/lib/store.js:299:12)] Store close

Store connection closed.
Shutting down proxy.
Proxy closed.

How does this Node.js module work?

I read that the proxy translates network activity between the Node.js module and the Oracle NoSQL Database store. The proxy can be spawned as a child process (JVM) from the Node.js module. A JavaScript Thrift client (see Thrift protocol) has been generated with the Apache Thrift compiler which is used by the module to communicate with the proxy. The proxy then uses kvclient to connect to the database. This would be something like the image below.

I wondered what the performance cost would be of having a Java proxy and two translations between the Node.js module and the NoSQL database. It would be interesting to compare the bare RMI Java client performance with the Node.js module performance and to compare performance of a query executed from within the database with a query executed from outside by the RMI kvclient to determine the performance cost of the different hops/translations. I can understand the usage of Thrift though since it provides a relatively easy way to create clients in different languages.