Thursday, December 29, 2022

Python: Batch resize and pad files

When using AI models like Stable Diffusion, sometimes input images need to be of a specific size. In case of Stable Diffusion, multiples of 64 are required. Stable Diffusion (at least 1.5) works best with images of 512 pixels in width or height. If you for example take an image of 599 x 205 pixels and you resize it to 1496 x 512 (maintaining the aspect ratio), you end up with 1496, which is not a multiple of 64. In order to obtain a usable image, it needs to be padded to a size of 1536 x 512 to allow processing. In order to batch resize and pad images I created a Python script using OpenCV. Why create a script? Doing this for a large amount of images quickly becomes a chore and writing a script for this is fun. Why OpenCV? It is powerful, easy to use and popular.

Wednesday, December 28, 2022

Apache NiFi: Filter events and only let through the latest in a timeframe

In the IoT world, some devices generate large volumes of events that can be difficult for back-end systems to process in real time. Of course you can use NiFi to throttle messages. However, this will not be sufficient if the flow of events is consistently higher than what can be handled by the back-end system. A way to deal with this is to let Apache NiFi group and filter messages based on a specific attribute and only letting through the latest message for a specific device, in a certain timeframe. In this blog post I'll illustrate how you can do this. The trick is to merge several messages together using the MergeContent processor and then select the latest one using a Jolt transformation.