Friday, May 17, 2024

Local Model Hosting: Tips, Tools, and Trade-offs

Large Language Models (LLMs) are transforming natural language processing, enabling advanced applications like chatbots, content generation, and more. Running LLMs locally provides enhanced privacy, control, and performance. This guide will walk you through setting up LLMs on your local machine.

While hardware requirements and required technical knowledge can pose a challenge, selecting the right tools simplifies the process. I recommend starting with LM Studio for its user-friendly GUI and Ollama for a robust CLI tool. This blog post aims to help you get started quickly and provides essential background information.


Tuesday, January 3, 2023

Stable Diffusion: AI Image generation

To quote Wikipedia (here): "Stable Diffusion is a deep learning, text-to-image model released in 2022. It is primarily used to generate detailed images conditioned on text descriptions, though it can also be applied to other tasks such as inpainting, outpainting, and generating image-to-image translations guided by a text prompt.". This of course sounds nice, but what makes it special and how can you use it?

Thursday, December 29, 2022

Python: Batch resize and pad files

When using AI models like Stable Diffusion, sometimes input images need to be of a specific size. In case of Stable Diffusion, multiples of 64 are required. Stable Diffusion (at least 1.5) works best with images of 512 pixels in width or height. If you for example take an image of 599 x 205 pixels and you resize it to 1496 x 512 (maintaining the aspect ratio), you end up with 1496, which is not a multiple of 64. In order to obtain a usable image, it needs to be padded to a size of 1536 x 512 to allow processing. In order to batch resize and pad images I created a Python script using OpenCV. Why create a script? Doing this for a large amount of images quickly becomes a chore and writing a script for this is fun. Why OpenCV? It is powerful, easy to use and popular.

Wednesday, December 28, 2022

Apache NiFi: Filter events and only let through the latest in a timeframe

In the IoT world, some devices generate large volumes of events that can be difficult for back-end systems to process in real time. Of course you can use NiFi to throttle messages. However, this will not be sufficient if the flow of events is consistently higher than what can be handled by the back-end system. A way to deal with this is to let Apache NiFi group and filter messages based on a specific attribute and only letting through the latest message for a specific device, in a certain timeframe. In this blog post I'll illustrate how you can do this. The trick is to merge several messages together using the MergeContent processor and then select the latest one using a Jolt transformation.


Friday, September 30, 2022

Reduce your gas and electricity bill!

Gas and electricity costs have increased significantly in the Netherlands and perhaps throughout Europe as a result of the current difficulties on the energy market. In this blog post, I'll make some recommendations for how to lower your gas and electricity costs. Some of these are specific to the Netherlands.

My goal is to use gas and energy as economically as possible. Consequently, the goal is to lower the gas and/or electricity costs. You may also be motivated by other factors, such as a desire to stop using Russian gas or a desire to lower CO2 emissions. These motives could lead to different choices. Everyone lives in a different house and there are differences in the laws/regulations of the various countries, which can, of course, also affect the decisions you make.

N.b. I'm no gas/energy expert but the below measures are things which I have looked at and which seemed to help. 

General suggestions

Reduce gas
  • Insulate your house
    This is an obvious one and, especially in older houses, you can save a lot of money by this. You can do large operations like insulating your walls, roof, loft or floor but you can also relatively easily also do small things like 
    • use weatherstrips
    • insulate pipes. Especially if they transport hot water.
    • use thick curtains or thermo curtains to keep heat inside
    • making sure your mailbox when it is build inside your front door does not cause a draft. You could reduce this with a product like this.
    • install radiator foil when you do not have floor heating for the radiators which are against walls which are connected to the outside. Radiator foil is cheap and easy to install. See for example here. It works best if you can directly attach it to your radiator.
    • close doors when you are not inside the room. Also remember heat goes up. The rooms at the ground floor, especially if they have high ceilings, are more difficult to keep warm. If you prevent air movement by closing doors, the heat stays in the same place longer.
  • Reduce your room temperature
    You can save a lot of gas by reducing your room temperature. This probably requires you to wear more clothes. The tip here is to use thermo underwear and layers. In your bed woollen blankets can help keep you warm at night. Often also you do not need to have all rooms at a high temperature so consider only heating the rooms you use often.
  • Tweak your central heating system
    A central heating boiler (CV in Dutch) produces hot water for the tap and hot water for the radiators. The following (Dutch) site has some suggestions on this (here). If you are looking for manuals, you can check out this site (Dutch).
    • The hot water for the tap needs to be at least 60 degrees of Celsius to prevent the Legionella bacteria from causing issues. Higher is often not necessary!
    • The hot water for the radiators can usually be lowered to 60 degrees of Celsius when your house is well insulated and you have reasonably new radiators. The default setting could be as high as 95 degrees of Celsius which is not needed. Mind that a lower setting means it will take longer for your house to warm up.
    • Usually the central heating boiler has several settings related to heating. You can usually put in on 'comfort', 'eco' or 'off'. This setting determines how long it will take until you have hot water when you open the tap or start the shower. When the setting is on comfort, it will use more gas than when set to eco. This can be a waste if you don't mind waiting a bit longer for hot water.
    • The radiators can be tweaked to optimize the water flow going through and the heat it transfers. This is called hydronic balancing (see here). Look into it! It can make your radiators more efficient and help with keeping different rooms at a similar temperature. When you let someone do this, they can at the same time do maintenance on your central heating boiler.
    • A smart thermostat which turns the radiator heating lower or off when you are not there and for example at night, can also save a lot of gas. If you still have a manual thermostat, you will probably sometimes forget to turn it down when you are going to bed or when going on a holiday.
    • There are radiator ventilators which help in making radiators more efficient. See for example here.
    • Prevent air and rust build-up inside your central heating system. Bleef your radiators once in a while (see for example here). Air is bad for heat transfer and can cause rust inside your pipes which is also not good for heat transfer. Rust can also cause issues inside the central heating boiler, especially if it is a relatively new one (when the pump uses direct current). You can prevent rust/air from accumulating inside your central heating system by using dirt separators and air vents (see for example here). A dirt separator should be used for the water going into the central heating boiler and an air separator for the water leaving the central heating boiler. 
    • Replace old central heating systems
      Old central heating systems are often less efficient than new ones. New central heating systems have several features to achieve this which old ones mostly don't have yet. Do look into whether you would benefit most from a heat pump, a hybrid central heating system (which can heat on gas and electricity) or another solution. Also consider legislation. For example in the Netherlands in the future, it will not be allowed anymore to buy a central heating system which only works on gas.
  • Dry air is easier to heat
    If the air in your house is relatively dry, the air is easier to heat. Read for example here (there are many other sources). If you want to save on having to heat your house (gas or electricity), reduce humidity to below 40% in winter. This also helps reduce condensation of water on cold surfaces, which, as an additional benefit, can also prevent moulds from forming (see here). The best way to go about this is to first measure the humidity inside your house and next take measures if required. You can buy a dehumidifiers to reduce humidity and of course make sure when taking a shower or bath, that the room is well ventilated.
  • Shower
    A rain shower uses a lot of water which needs to be heated. Heating requires gas or electricity. You can save gas/electricity if you use a water saving shower head instead. Personally I prefer them anyway. You can of course also save on gas/electricity if you shower shorter. Taking a bath uses more gas/water than taking a shower if the shower is shorter than around 10 minutes (see here).
Reduce electricity
  • Reduce standby electricity usage
    When a device, for example a charger, is plugged in, it consumes electricity. Even when it is not in use. You can measure this by using relatively cheap devices like these. With the current electricity prices I noticed that my home workplace would use more than €150 euro of electricity a year when not in use! This can easily be avoided by using a power strip with an off button and turn it off when not in use.
  • Use LEDs
    LED light bulbs are the most energy efficient light bulbs you can currently buy. Should you have 'old fashioned' light bulbs, you can save as much as 10 times the electricity for the same amount of light. Of course LED lights do not help you in heating your house. If you use smart LEDs, there will be slightly more electricity consumption, even when it is turned off since it needs to be able to respond to signals for turning it on (usually using WiFi or Bluetooth). This additional electricity usage is not that much though (see here).
  • Use a hot water dispenser instead of a Quooker or water boiler
    A Quooker keeps water warm so you can have hot water when you need it. This however requires continuously keeping water warm / electricity. A regular water cooker often (depends on how you use it) heats too much water to a too high temperature. You can consider using a water dispenser (for example here. usually between 50€ and 100€). Using such a device you can configure the temperature and amount of water to use which makes more efficiĆ«nt use of electricity. Boiling water using a water boiler or a kettle has similar cost (currently).
  • Replace old devices
    • Old freezers, refrigerators, washing machines, dryers, televisions (especially plasma or LED) are often a lot less efficient with their energy than modern versions.
    • Use laptops instead of desktop PCs. Laptops use a lot less power (they are made to last long on a battery). They usually have a lower life expectancy than desktops though and they are often more expensive for similar hardware when compared to desktops.
    • If you are running a server at home, especially if it is a desktop or similar, consider using hardware for it which uses less power. For example, a server often does not need a graphics card. An SSD is a lot more energy efficient than an HDD (here). Would a Raspberry Pi or something similar not be enough for your needs?
  • Use the eco programs on your dishwasher and washing machine
    The eco program, if your machine has one, usually takes longer to complete than the fast or regular programs. They are however more energy efficient. See for example here.
  • Tweak your television
    Suppose you have a large OLED TV. It uses a lot of power when turned on and even when in standby (can be avoided with a power strip with off button as mentioned earlier). You can tweak your TV to reduce power usage. See for example here for some suggestions for if you own a recent LG TV.
  • Use timers
    You can use timers such as these to cut power to devices you do not use structurally at certain times a day. For example why power your modem/WiFi router when you are asleep?
  • Freezer
    Make sure there is no ice in your freezer. It will use less electricity without ice around the cooling units. Read here. There are stories that a filled fridge uses less electricity but there does not appear to be solid evidence for this (read here).
Are your electricity and gas meter accurate?

The gas and electricity meters might be inaccurate. Check for example here. If you suspect this might be the case, you can check for yourself first and if the meters  measurements are clearly inaccurate, you can contact the supplier. They can perform a more accurate test. Usually you have to pay for this test but if you are right and the measures were off, you will get a refund and will receive compensation for the estimated amount you have paid too much in the past.

Specific to the Netherlands
  • Solar cells good, heat pump bad
    Choose your energy/gas conservation measures carefully
    • Currently in the Netherlands solar cells have a really short return of investment. Definitely get them if you do not have them already. An air/water heat pump though has a very long return of investment if you can even save enough to make it worth the initial investment. See for example here. If you have low gas usage and no underfloor heating, buying a heat pump in the Netherlands is currently not economically interesting. An air/air heat pump, also called airco (which can also do heating) is a lot more efficient than a mobile airco. It does use electricity and it will probably be most useful to cool during summer (using the electricity from your solar cells). City heating is not necessarily cheaper or more environment friendly when compared to using your own central heating system (does it burn wood or gas? is the price of what it burns used to calculate what consumers need to pay?). A boiler and heat collectors might be interesting to look at since they can reduce gas usage but do require electricity to pump water around and require quite some space. If you totally want to get rid of using gas, your total gas/electricity bill will probably rise. If for example you want to cook using induction, even during this energy crisis, you will still have to pay more than using gas. This is the current situation. It might change in the future.
  • Check out government subsidies
    You do not have to pay for everything by yourself! The government provides several subsidies to make your home more gas/energy efficient or improve your home in general. In Groningen you can take a look at what you might get for earthquake damages (here). If you have confirmed damages, you can get the SNN subsidy to increase the value of your home. If you are lucky, you can also get an additional 10K (here). If you have used the money to improve your home (for some subsidies that is a requirement), you can also use the same measures to request a more general subsidy for which you don't have to live in Groningen (see here for some suggestions).
  • The price ceiling. Gas users benefit most
    A relatively new policy sets a price ceiling for electricity and gas starting at januari 2023 but gas/energy are allowed to implement it as early as november 2022. Until a certain amount of usage you pay at maximum a specific price. Above that usage, you pay the market price which is most likely higher. If you manage to stay below the price ceiling, gas and electricity will be relatively cheap. A challenge is mainly high electricity usage. For example you do not have (enough) solar cells to cover your electricity usage (e.g. charging electric cars and using electricity to warm your house). You could end up above the ceiling for electricity. It might be worthwhile to consider whether it will be cheaper heating your house on gas than on electricity if you have the choice.

Apache NiFi: Monitoring metrics and provenance events using Azure Log Analytics

There are several cases where you might want to use Azure Log Analytics to monitor your NiFi instances. An obvious one is when NiFi is running in Azure. Azure Log Analytics can also be used as single monitoring/alerting solution for multiple applications making operations easier by providing a single interface for this. You might want this if you want to monitor business processes which span multiple applications and you want to monitor the entire process to for example identify bottlenecks.

In this blog post I'll show you how easy it is to achieve this using the AzureLogAnalyticsReportingTask and the AzureLogAnalyticsProvenanceReportingTask from NiFi and what you need to configure in Azure Log Analytics to make this work.

Wednesday, May 18, 2022

Apache NiFi: Importing and exporting parameters

When you import a new process group or upgrade an existing one, missing parameters contexts and parameters will automatically be added. The new parameters will be filled with values from the environment where the process group was committed to the Registry (except sensitive parameter values). This is usually a development environment. NiFi 1.15 adds parameter context inheritance. If however you are on a lower version and have many similar process groups, you can have many copies of parameter contexts. If you add a large number of flows and need to add a large number of parameter contexts, it can be bothersome to have to manually update all the parameter contexts used by the new process groups. 

In most cases when you use deployment tooling the environment configuration is kept separately from the application and applied upon deployment. This is also one of the principles of the 12 factor application (read here). For example in Azure DevOps you can use variable groups, in XL Deploy environments, a Spring Boot applications usually uses property files and K8s resources can use Kustomize templates. In this blog post I created a script which allows you to use a similar method. You can export parameters from an environment (except for sensitive parameter values). This results in a CSV file. This CSV file can be imported into a different environment. This also allows you to keep a CSV parameter file per environment which can be applied on deployment. Updating parameters in a CSV is a lot easier than doing the same manually and you can easily check if everything is correct.

You can check out my code here. Specifically look at the export_parameters and import_parameters methods.