My first NodeJS service

Microservices implemented in JavaScript running on NodeJS are becoming quite popular lately. In order to gain some experience with this, I created a little in memory NodeJS cache service. Of course statefulness complicates scalability, but if I would also have implemented a persistent store to avoid this, the scope of this blog article would have become too large. Please mind that my experience with NodeJS is limited to a NodeJS workshop from Lucas Jellema and a day of playing with NodeJS. This indicates it is quite easy to get started. In this blog I'll highlight some of the challenges I encountered and how I solved them. Also I'm shortly describing what Oracle is doing with NodeJS. Because the JavaScript world changes rapidly, you should also take into account the period between when this blog is written and when you are reading it; it will most likely quickly become outdated. You can download the code from GitHub here.

Choosing an IDE

In the Java world there are several popular IDE's such as JDeveloper, Eclipse, Netbeans, IntelliJ. For JavaScript, the IDE's I've heard most about from JavaScript developers (as a newby it helps to talk to people with experience) are Microsoft Visual Studio Code and Jetbrain's WebStorm. Netbeans also has JavaScript support and is the IDE of choice for Oracle JET development. I have not looked into Netbeans yet. I decided on Microsoft Visual Studio Code since WebStorm requires a paid license.

NodeJS package manager

The NodeJS package manager is npm. npm can install modules globally and locally. Supporting tools like 'mocha' for testing and 'typings' for TypeScript support are good candidates to install globally. Do keep track though of your globally installed modules since if you want to reproduce your environment somewhere else, these modules could be dependencies (especially in your build process). You can configure local dependencies in a package.json file. When you do a 'npm install', modules mentioned in that file are installed locally in the node_modules folder of your project. If you want to also update the package.json, you can do 'npm install --save'. This allows you to easily update versions of modules. When your node_modules directory is corrupt because you for example interrupted a module download, you can just remove the node_modules directory and rebuild it from the package.json file.

Code completion

As a spoiled modern developer, I need code completion! This especially helps a lot when you are unfamiliar with a language and want to explore what you can do with a specific object or how to use standard libraries/modules. JavaScript is not strongly typed. You need type information to provide code completion. Microsoft has provided the open source TypeScript to help with that.

TypeScript allows you to write .ts files which can be compiled to JavaScript. These .ts files can also be used to allow Visual Studio Code to provide code completion (even when writing JavaScript and not TypeScript). There is a large library of TypeScript definitions available for JavaScript modules. There are two tools I currently know of to easily import .ts files into your project. TSD and Typings. TSD is end of life (https://github.com/DefinitelyTyped/tsd/issues/269). In order to download .ts files for your modules, you can create a typings.json file which indicates the dependencies on TypeScript files. When you do a 'typings install', it will download these dependencies into your project in the 'typings' directory. When you want to add new dependencies, you can do a command like 'typings install node --save --ambient' to save the dependency to the typings.json file. TSD uses a tsd.json file which can be converted to a typings.json file with 'typings init --upgrade'. You can read more about how to set this up here.

Next to the tsd files, you need to provide Visual Studio Code with compiler options in a jsconfig.json file.

The result looks like:

Testing

I used Mocha to perform tests. Why Mocha? I heard about this from colleagues and found a lot of references to 'mocha' when reading about JavaScript testing.

There are a lot of other modules which help with testing. Mocha is installed globally. When executed in the root of your project, it looks for files in the test directory and executes them.

Below some sample test code

Below what happens when you run Mocha.

For testing as a true outside client, you can use Ready-API or for example Postman, Postman is a Chrome plugin.

Debugging

In order to debug your project and run NodeJS from Visual Studio Code, you need to provide a launch.json file in . vscode directory.

This file indicates which JavaScript file to run and where to attach the debugger. You can read about this in more detail here.

Some general observations

Single threaded

A NodeJS instance uses a single thread. Therefore, you should strife to have all your code non-blocking. Most functions support asynchronous interaction. This allows you to perform an action and wait for an asynchronous callback. While 'waiting' for the callback, other actions can be performed and other callbacks can be received. This event driven / asynchronous way of programming is something you have to get used to. Scalability can be increased by raising more NodeJS instances. In accordance with Microservices architecture, services should be stateless. My example cache process is not stateless and does will not scale well. Instead of using a local variable I should have used a persistent store.

Few generally accepted standards

This is of course my first impression. There are few generally accepted standards in the NodeJS world. For example, for testing there are many frameworks available. This also means that direct IDE support for a testing framework is difficult or requires manual scripting. Also talking to databases has no standard module (not something like JDBC in the Java world). Several modules use similar ideas and principles however. For example, most of them use JSON configuration files as part of the project. Also since the JavaScript language has its limitations (which becomes more apparent in large projects), there are several scripts which offer useful functionality and can be compiled to JavaScript such as TypeScript.

NodeJS and Oracle

Oracle is of course also investing in NodeJS. There is the Application Container Cloud Service which allows you to run NodeJS applications. You can choose the version of NodeJS on which your application should run. This allows Oracle to (relatively easily) stay up to date with the cloud service and users to choose when to run on which version. The cloud service comes integrated with an Oracle database driver to easily access your Oracle persistent store.

Also NodeJS is used in Mobile Cloud Service under the hood.

Oracle is working on several products which go well with NodeJS such as the API Platform and most likely more is coming in this area.

If you want to know how you can access the Oracle database from NodeJS, look here. If you want to create a Docker container with NodeJS and the Oracle database driver integrated, see here.

NodeJS for orchestration

In Oracle SOA Suite and Service Bus we were used to graphical development to do integrations. Languages like BPEL which are XML based to describe flows. Out of the box functionality like security policies, management interfaces and technology adapters/transports. BPEL or ServiceBus 12.2.1 can run JavaScript in the JVM and supports untyped JSON. NodeJS provides a lot of modules but very little out of the box and not much standardization. It is fast, light and scalable though (when stateless). I can imagine that NodeJS would be especially suitable to provide services which are thin and do not require much functionality (≈microservices). An example could be a layer of data services on top of a persistent store. When however you want to validate your messages, provide traceability, advanced error handling, throttling and require features like I previously mentioned, you can do it in NodeJS but it would require quite some coding (which requires maintenance). You might be better of with something which provides these features out of the box (like SOA Suite) in such a case. If however you want a quick stateless solution to easily integrate two REST/JSON services server-side with JavaScript, NodeJS is probably currently the best solution for that.

Software lifecycle

Because of the speed of JavaScript evolution, the turnover rate of code written using specific frameworks/modules will probably be higher since they are quickly outdated. I noticed however that it is quite easy to get started with new modules; not really a steep learning curve. I can imagine in many cases it will be more easy to start over instead of trying to migrate. Large companies and governments might have to get used to such a way of working.

Some things I have not looked at

Code compliancy

I have not looked at code compliancy inspection yet. Especially for a 'sloppy' language like JavaScript, such tooling is important to make larger projects manageable (read here). I heard JSLint for JavaScript and TSLint for TypeScript are popular.

Test coverage

For test coverage reporting you also need specific tools. See here for some example modules. Istanbul seems popular when using Mocha.

Building

Grunt and/or Gulp can help you build your project. You require tools like these to help you compile for example your TypeScript to JavaScript and to perform several other build steps such as automate the release process. Also to orchestrate the tasks mentioned above such as code compliancy checking,  and test coverage checking, these tools can help.

Finally

I was surprised how easy it was to get started with NodeJS. The world of JavaScript was new to me and you can see from this blog post what I have learned in just a single day and a workshop. You need very little to get started.