Reduce the data records of "hypertrophic" sensors on Home Assistant

3 minutes of reading
PURPOSES OF THE GUIDE:
  • Reduce the number of data written to the database Home Assistant starting from sensors that collect many readings
  • Difficulty level: basso
CONCEPTS FACED:
  • Software use and configuration
SOFTWARE COMPONENTS USED:
Prerequisites:
PHYSICAL DEVICES USED:
GUIDE more indicated for:

All environments

Notes and disclaimer
  • qualsiasi eventuale modifica agli impianti domestici dev'essere progettata ed realizzata SOLO da personale qualificato;
  • any changes implemented in probefore is a propersonal responsibility as well as a profirst risk and danger (the contents of the present page are purely educational);
  • any changes implemented in proprior to a device it voids the guarantee, quality approvals and certifications.
Driving review: 1.0

Abstract

The sensors, its Home Assistant, are among the entity more used both in automations both in the normal consultation at the frontend Lovelace UI. Temperature, humidity, luminosity, electric absorption, air quality, presence and so on: many are in fact the types of sensors deriving from the integrations Performable with this powerful HUB personal.

Sandisk microSDWhat many do not know - or rather, on which maybe they don't reason - it's the big difference in terms of impact from sensor to sensor. In fact, there are large differences in the amount of data saved by theHUB based on the readings provided by the sensors: some in fact provide very spaced readings, over time, from one another; others, instead, bombard the domotics of continuous readings, constant, sometimes even dozens in a single minute.

These are aspects that are anything but secondary. First of all, large amounts of data result in the sometimes marked deterioration in performance in reading: consultation of historians becomes slow, not very responsive. What is more serious, however, is the impact on storage: especially the users of Raspberry Pi which server to run on Home Assistant (read: almost all) should raise the antennas, aware that a large number of writes on the microSD is the cause, nine times out of ten, of its irreversible break.

To mitigate the deleterious effects of a "hypertrophied" sensor, the solution may be that to exclude it from the "Recorder", so as not to save the large volume of data from it to disk products: will always be available at the frontend the last reading received but not, of course, the historical trend.

This is a good solution, but creates a proBlema: loss, appanointed, of the historical trend. If this is not a prothen the reading of this guide is not necessary; otherwise, proOn this occasion, we offer a solution to save the proverbiali goat and cabbage.

Approccio

Home Assistant has a very important and useful component, namely "Filter Sensor“: This component allows the definition of sensor entities starting from the collectionnameof data procoming from other sensors, filtering through various statistical methodologies.

Nb This guide does not have an appromarkedly scientific. Statistical filtering methods are complex topics which we would not be able to illustrate in a suitable way, therefore we will avoid doing so. What we will do is proto set roughly functional configurations for the purpose. If any member of the community with expertise on the subject wanted proplace an approfoundation, we are willing to provide the necessary space.

What we will do then will be exclude from writing to disk i (too many!) data procoming from a given sensor, supplying them instead to a sensor “Filter Sensor"Which, based on the methodology (s) chosen prowill try to reduce the number of information while still providing a legible trend.

Definition of the sensor

We hypothesize that the "hypertrophied" sensor is a sensor that provides the instantaneous absorption in Watts of the domestic system called "sensor.hem_power". Let's assume that this sensor is sending a reading every two seconds: definitely too much, and useless.

We define a sensor in the configuration like this:

sensor:
  - platform: filter
    name: "hem power filtered"
    entity_id: sensor.hem_power
    filters:
      - filter: outlier
        window_size: 4
        radius: 2.0
      - filter: lowpass
        time_constant: 60
      - filter: time_simple_moving_average
        window_size: 00:05
        precision: 0

This sensor type "Filter Sensor”Uses three filtering modes (appone after the other, in sequence):

  • the "outlier", Or a filter"band pass“, Which cuts values ​​out of a specific medium range;
  • he "lowpass", Or a filter"low pass“, Which“ cuts ”im peaksproinformation in the data;
  • he "time simple moving average", Or the"moving average"

This "standard" configuration generates a sensor called sensor.hem_power_filtered which proit produces a number much lower than the data source, while still providing the user with the historical progression of the measurement.

Nb the configuration promail is one of the possible with respect to the sensor analyzed; careful reading is recommended of the page relative to the “Filter Sensor” in order to identify which filter (and how to configure it) is appropriate for the case proprio.
Home Assistant - pre-filtering sensor
pre filtering ...
Home Assistant - post-filtering sensor
... and post filtering.

Nb The two images above are related to the historical series of the two sensors sensor.hem_power e sensor.hem_power_filtered (with the same "Friendly name", or "HOUSE consumption“).

Exclusion from the recorder

At this point it is necessary exclude the original sensor from the recordings of the "Recorder" component, so as not to save the time series (and thus avoid the "bombardment" of data to our storage) and acquire instantaneous reading only, leaving the new filter the role of consultation.

To make this exclusion, reading is recommended of the guide explaining the functionnameof that component.


Home Assistant Official LogoATTENZIONE: remember that there is on our FORUM community an ad hoc section dedicated to Home Assistant, for any doubt, question, information on the specific merit of these components.


telegram

Stay up to date through ours Telegram channel!