Apache NiFi: Filter events and only let through the latest in a timeframe

In the IoT world, some devices generate large volumes of events that can be difficult for back-end systems to process in real time. Of course you can use NiFi to throttle messages. However, this will not be sufficient if the flow of events is consistently higher than what can be handled by the back-end system. A way to deal with this is to let Apache NiFi group and filter messages based on a specific attribute and only letting through the latest message for a specific device, in a certain timeframe. In this blog post I'll illustrate how you can do this. The trick is to merge several messages together using the MergeContent processor and then select the latest one using a Jolt transformation.

Simple IoT security system using Raspberry Pi 2B + Razberry + Fibaro Motion Sensor (FGMS-001)

In this article I'll describe how I created a simple home-brew burglar detection system to send me a mail when someone enters my house (so I can call the police). First my choice for the components is explained. Next how these components combine to achieve the functionality wanted. Based on this article you should be able to avoid certain issues I encountered and have a nice suggestion for a simple relatively cheap burglar detection system.

My purpose was to create a simple security system based on a Raspberry Pi. A Raspberry Pi is a tiny computer which can run a Debian like Linux distribution called Rasbian. I wanted to avoid going to low-level into sensor configuration and programming. That's why I decided early on to use an extension board and not directly attach the sensors to the Raspberry Pi. I decided to go for the Razberry. I also looked at the GrovePi and Arduino. Both are still too low-level for my tastes though. The Razberry is an extension board for the Raspberry which provides a Z-Wave controller chip. Z-Wave is a wireless protocol popular in the area of home automation. This was an attractive option since if in the future I would want to use additional sensors or maybe even use a commercial home automation system, I could very well get compatibility out of the box. For the sensor, I decided on the Fibaro FGMS-001 Motion Sensor. This is a multi-sensor which allows detection of motion, temperature, luminiscence and vibrations. It can even detect tampering and earthquakes (which is relevant since I live in the Dutch city of Groningen).

Z-Wave.Me (the company providing the Razberry), provides software for the Razberry called Z-Way. There are several alternatives. One of the most popular seems to be Domoticz which is provided with OpenZWave. Domoticz allows quite extensive home automation but I was having difficulty getting the sensor to work with OpenZWave so I decided to go with Z-Way. Z-Way supported the sensor out of the box. With the Z-Way server however it was difficult to automate actions based on sensor values. How I solved this is also described in this article.