Transform multiple MQTT elements into one virtual device via Node-RED

15 minutes of reading
PURPOSES OF PROJET:
  • Create a virtual device MQTT starting from the presence of several devices (physical and / or virtual) MQTT
  • Difficulty level: alta
CONCEPTS FACED:
  • Software configuration
SOFTWARE COMPONENTS USED:
Prerequisites:
  • Node-RED configured and working
  • Broker MQTT configured and working
PHYSICAL DEVICES USED:
  • MQTT actuators / components
GUIDE more indicated for:

All environments

Notes and disclaimer
  • qualsiasi modifica all'impianto elettrico dev'essere progettata ed effettuata da personale qualificato;
  • any changes implemented in probefore is a propersonal responsibility as well as a profirst risk and danger (the contents of the present page are purely educational);
  • any changes implemented in proprior to a device it voids the guarantee, quality approvals and certifications.

Abstract

TWO WORDS ON MQTT

In the realization of the propria domotica personal often happens to integration components based on proMQTT tocollo. It is usually a fortune: MQTT is a proProtocol simple, fast, reliable, which opens up a great range of features. Sensori, actuators, complex devices: a protocol, the MQTT, which allows you to integrate them with the major ones HUB personal without particular efforts.

MQTT allows you to exchange messages to and from components compatible with that protocollo by gods topical, or strings with payload (payload) that can rappresentare a command (Command) or a reply / communication (telemetry).
On inDomus we used to explain this concept the metaphor of lighthouses.

Un topicalusually contains a component that identifies its nature (whether it is a command or a telemetry), one that identifies the recipient (if a command) or the sender (if telemetry) and finally a remaining part that identifies the content of the topical. Finally, the payload (not always present) is a content in JSON notation which contains, for theapptogether, the payload of command or telemetry.

Devices that, for example, are equipped with firmware Sonoff-Tasmota (as for theappanointed him ITEAD Sonoff, or NodeMCU) usually receive commands via an MQTT message of this type:

cmnd /Sonoff/ POWER ON

in this specific case composed of two parts, the topical "cmnd /Sonoff/ POWER" and the payload "ON"

  • cmnd identifies the "command" nature of the topical;
  • Sonoff identifies the actuator addressed to the command;
  • POWER identifies the type of command, in this case "change of relay status";
  • ON identifies the payload, which associated with POWER causes the actuator relay to switch on. Had been OFF, would have commanded the shutdown.

Un topical telemetry, instead, could be the following:

stat/Sonoff/SENSOR = {“Time”:”2018-02-15T17:37:10″,”ENERGY”:{“TotalStartTime”:”2018-11-14T18:39:40″,”Total”:6.294,”Yesterday”:5.340,”Today”:0.954,”Period”:217,”Power”:2635,”ApparentPower”:2650,”ReactivePower”:282,”Factor”:0.99,”Voltage”:227,”Current”:11.661}}

where, in this specific case:

  • stat identifies the "telemetric" nature of the topical;
  • Sonoff identifies the sending actuator of the telemetry;
  • SENSOR identifies the source, internal to the component;
  • the remaining part included between { e } rappresists the JSON payload, which, in this case, contains readings of electric absorption (this example refers to the readings of a Sonoff POW equipped to sign Sonoff-Tasmota).

But it could also be something simpler, for example:

stat /Sonoff/ POWER = 0

That is a simple telemetry that indicates that the relay of an actuator Sonoff it's off.

TRADITIONAL INTEGRATION

This model, in the traditional integration phase, it greatly simplifies things compared to other scenarios. It will be enough to know the topical command and telemetry systems to integrate a given MQTT component to the profirst home automation, as such protocollo, as already mentioned, is widely supported as a transversal integration platform to HUB personal.

Take the example of a switch (MQTT compatible) to be integrated with Home Assistant: we would typically use the platform "MQTT Switch" in order to create a "switch" entity at the configuration, just fill in the fields related to topical in a timely manner; similarly, for Homebridge we would use the plugin "homebridge-mqttthing"For the same purpose and in the same manner, and so on.

COMPLEX SCENARIOS

So far, so good: a component (and therefore one "receiver commands / sender telemetrie'), a broker which acts as an MQTT message dispatcher, un personale HUB which sends commands and interprets return telemetry. Easy.

What's happening in presence of a non domotic device domotized, for example, from multiple different MQTT components?

A classic example as easy to understand is that of tappelectric arrays. Typically one or more actuators are used in order to control their functionnamento: it is the case (but it is only one of the many possible examples) of the Sonoff dual, dual channel actuator ideal for controlling the same number of channels of a tappelectric arella (one for raising and one for going down). Such a device after now traditional riprogrammation with alternative firmware, is equipped with proMQTT tocol, which allows you to control the two channels ma in a differentiated way, as it were of two disjoint physical actuators. It cannot in fact be configured directly as a single logical entity of type “tapparella ”, only as two on / off switches.

Taking another example (which we will use as a trace for the projet) let's imagine we have controlled a remote control (make sure you have read and understood that projet prima di profollow): in this scenario a single device (say of a motorized gate) is controlled by this remote control, which in turn is commanded from as many MQTT home automation actuators as there are buttons to control.


Let's say that the remote control (maybe a gate) is present just two buttons ("aperture" and "chiusura") And then two corresponding home automation actuators called respectively (in the probefore MQTT configuration) PULSANTE1 (the opening one) e PULSANTE2 (the closing one) and configured to activate (when commanded) the corresponding remote control button for a brief moment, as a human being would do in the act of using it (this is configured on the actuator side, for example with type commands (PULSETIME).

Now imagine that the MQTT commands that control the two actuators (and therefore that activated the relevant buttons / and then command the gate) are, respectively:

BUTTON relating toaperture of the gatecmnd/PULSANTE1/POWER ON
BUTTON relating to chiusura of the gatecmnd/PULSANTE2/POWER ON

Understanding which logic is simple: sending the topical command MQTT arises in the gate opening, the second in the closing.

To integrate the opening and closing of the gate in home automation so I have two roads:

  • define two simple switches that, when activated, send the topical Corresponding MQTT;
  • to define an entity that rappwith a real virtual gate e proprio, therefore accepting opening / closing commands which then arise in the dispatch of topical MQTT corrected.

Both roads are valid but, while the first one it is rough and presents proproblems related to the imprecise nature of the entity (trivially, we have two "switches" rather than a "gate" entity), the second is more refined and substantially correct, also and above all in automation use (in exquisitely logical terms it is correct to ask “open gate entities" rather than "activate 1 switch"). Let's not forget which is important, in home automation, try to rappresent what is being domotized with entities as close to him as possible in terms of features, functionality, etc.

So let's say we want to pursue the second way.
As an example conceptual (valid in broad terms also for others HUB personal), let's say we want to create a type entity "Lock”(Lock) at Home Assistant (which rappvirtually resist the gate). To do this we would use the platform "MQTT Lock", Which in broad terms so it is implemented:

#Esempio di entità "MQTT Lock"
lock:
  - platform: mqtt
    name: Cancello
    state_topic: "topic_telemetrico"
    command_topic: "topic_di_comando"
    payload_unlock: "APRI"
    payload_lock: "CHIUDI"
    value_template: '{{ value }}' 
    optimistic: false
    retain: false
    qos: 1

The configuration items highlighted they are the heart of the entity, especially the second, or "command_topic“: In this field the user must indicate il topical to be sent when the "Lock" entity is commanded associating the "payload_ *"Correct (based on whether you are opening or closing the gate).

But say topical we have it the two, Because the two are the MQTT devices we are commanding, for the two different functions related to the same object. The actuator payload, then, is always a e a only, or "ON"

cmnd / PULSANTE1 / POWER ON

cmnd / PULSANTE2 / POWER ON

And if - in other scenarios - the MQTT devices were not two, but four, ten, a hundred?

It is perhaps a proproblem without solution?
No: the solution, as almost always, there.

Black box

black-box

To solve this proBlema we will use the model black box, or the implementation of an element that is frapponga - in logical terms - between home automation and end devices, or something that "translates" of topical arbitrary proVenienti from home automation in topical intended to the correct devices, is topical telemetry proVenienti from the devices towards a single one topical telemetric verse home automation.

In doing so, we will create a sort of "virtual device”- equipped with propri topical MQTT - starting from those known and associated with the physical devices we are trying to command, as explained in the Abstract.

The role of "black box"Will be interpreted by that computer blessing that is Node-RED.


Node-RED is a simple yet versatile software that allows you to define data flows, or action sequences that usually have "input" at the input point and an "output" output point. What happens within these flows can be data processing, conditional evaluations and much, much more.

What we're going to do is basically define two flows:

  • one from home automation to devices;
  • one from devices to home automation.

In reality, home automation will be touched in a broad sense, because the flows we will define will do nothing but read and write MQTT messages forwarded to and from the broker, messages that will then be read and written by home automation.

Logic of flows

FLOW FROM HOME AUTOMATION TO DEVICES

We explained how, given the need to possess "a single one command MQTT topic"For the entity we are going to define in the configuration of theHUB contrappprecludes presence of "more command MQTT topic"(As many as there are actuators), we will necessarily have to make one up.

Returning to the example of the aforementioned gate, we will therefore define a topical Command MQTT, arbitrary (inventing it from scratch) that we will call:

cmnd / gate / share

while the two payload (since there are two possible actions) they will be, always arbitrarily:

YOU OPEN

CLOSE

What we therefore want to happen is that, by publishing a specific MQTT broker topical with a datum payload one thing happens, with another, another:

Topic MQTT command (forwarded from the virtual device defined in home automation)MQTT topic translation (then towards physical devices)
cmnd / gate / action OPENcmnd / PULSANTE1 / POWER ON
cmnd / gate / action CLOSE
cmnd / PULSANTE2 / POWER ON

The logical sequence is as follows:

  1. we say to command, through the interface of thepersonale HUB, gate opening;
  2. the gate opening at theHUB corresponds to the publication by both, at the broker MQTT, of the command “cmnd / gate / action OPEN";
  3. la black box (rappresented by Node-RED), connected to the broker, is configured to be registered at topical "cmnd / gate / share”(And therefore receives it);
  4. the black box is also configured so as to ask immediately after this reception"what is the payload of the topic?
    1. since the answer is "YOU OPEN", Publish the command on the broker"cmnd / PULSANTE1 / POWER ON"
    2. if the answer had been "CLOSE", Would have published on the broker the command"cmnd / PULSANTE2 / POWER ON"
  5. at this point:
    1. since it was published "cmnd / PULSANTE1 / POWER ON“, The actuator certified on the button of the remote control will receive it and act opens the gate, thus causing the gate to open;
    2. if it was published "cmnd / PULSANTE2 / POWER ON“, The actuator to act would have been the one attested on the remote control button that closes the gate, causing it to close.

What we need to accomplish is the black box flow, that is the 3 and 4 points of the logical sequence.

FLOW from DEVICES to home automation

As explained, physical devices generate state telemetry when commanded return after command, telemetry that must be received - univocally - from home automation, so that the accessory configured is consistent in the state.
Vivo, just saying. In essence, home automation must be certain that the command sent has been executed, and to do so it uses telemetry.

Going back to the gate example and related elements involved, whenever the PULSANTE1 or PULSANTE2 actuator is operated by command "cmnd / PULSANTE1 / POWER ON"Or"cmnd / PULSANTE2 / POWER ON“, Typically in response a telemetry is published by him at the broker MQTT.

Let's say that for these two actuators the telemetry proprepared for their activation either:

stat / PULSANT / POWER = ON

where obviously "PULSANTEx"Rappresents the sending telemetry actuator.

Since receiving telemetry "stat / PULSANTE1 / POWER = ON"Means, de facto, that the gate was opened (as well as "stat / PULSANTE2 / POWER = ON"Means that it is been closed), then this telemetry must reach the home automation system to confirm that the actuator has been commanded.

We will then define a topical Telemetric MQTT, arbitrary (also in this case inventing it from scratch) that we will call for this example:

stat / gate

while the two payload (since there are two possible actions) they will be, always arbitrarily:

YOU OPEN

CLOSE

Nb To describe the state would make more sense, being telemetry of reply, use payload type "OPEN" is "CLOSED" rather than "YOU OPEN" and "CLOSE". Remaining however in the example groove of an implementation on Home Assistant, the component "MQTT Lock"Which we are assuming to use foresees that payload to associate with topical command and telemetry are the same. Therefore, we will assume that the "opening" telemetry contains a payload "YOU OPEN", While the" closing ","CLOSE"

Node-RED

As anticipated, our "black box" will be rappresented by Node-RED, on which we will define two flows, one dedicated to communications from home automation to the actuators and another for the reverse path.

At this point let's connect to ours Node-RED previously installed (here we have one GUIDA for installation on Raspberry Pi with Raspbian operating system, otherwise it can be installed on HASSIO How add-on, or other solutions).

FLOW FROM HOME AUTOMATION TO ACTUATORS

At the top right we will press on “+" to create a new flow which we will call "Flow 1".

The left column rappit contains the list of available "node" types. We look for and find in the list (under the heading "Input") a node "mqtt in"

Node-RED - Home automation flow> devices - mqtt in

drag it in the empty space of the dashboard e let's click on it twice to open on the right the dialog box related to its configuration, called “Edit mqtt in node"

Node-RED - Home automation flow> devices - mqtt in - details

At this point it is necessary to indicate:

  • to which server (broker MQTT) to connect;
  • to which topical register, or on which topical "To listen to our node".

Let's start by defining a new server (mqtt broker).

Click on "Add new mqtt: broker"

Node-RED - Home automation flow> devices - mqtt in - edit broker

In the field "Name"We indicate any name, in the case of the example" Mosquitto LOCAL ", since the broker in use is mosquitto, installed on the same computer as Node-RED.

then we indicate, under the heading "Server", The IP of the broker (if it is running on the same computer as Node_RED, obviously we will indicate"127.0.0.1“, Otherwise the IP of our external broker a Node-RED. We will then indicate the door and any access credentials at the tab "Security"

At the end of the configuration, click on "Update"

Back to the dialog "Edit mqtt in node"It's time to indicate to the node"mqtt ip” appena added to the dashboard which topical to monitor.

In the field "topical"We will then insert the topical "Fantasy" that we will use for rapplogically reproduce our virtual device, that is:

cmnd / gate / share

We set QoS to "1"And, in the field"Name", always "cmnd / gate / share"
At the end of the configuration, click on "Done”To return to the dashboard.

If everything is correct, the node "mqtt in"Will acquire name"cmnd / gate / share”And the red triangle will disappear, a sign that the connection to the MQTT Broker is successful.

Now you need to identify and drag a "Switch"

Node-RED - Home automation flow> devices - switches

which, in essence, allows us to introduce some conditional logics.

Once the new node is placed on the dashboard, we connect it to the first node dragging the gray circle on the right side of the first (the output) in the gray circle on the left side of the node of the second (The input).

Now click twice on the node appena dragged to open the related configuration call dialog box “Edit switch node": Here we will define the node name, which proownership "Evaluate" at its entrance, finally which condition consider correct.

In the name field we indicate "ACTION"Leaving unchanged the proproperties to be evaluated at entry (ie msg.payload); in the box below "Property"Then we indicate two equalities valued respectively "YOU OPEN" and "CLOSE"

Node-RED - Home automation flow> devices - switches - equalities

Finally, click on "Done"

At this point Node-RED will have changed the switch node ("ACTION") Providing him with two exits (two gray dots on the right edge of the same node), which rappthey will return the two exit points (output) where the flow will be channeled in case the payload (coming from the previous node) is evaluated "YOU OPEN"Or"CLOSE"

In gray ball up rappresents the output in the presence of payloads "YOU OPEN"; one down"CLOSE"

At this point, among the type nodes "Function"Let's find one of a kind"Change”And drag them the two in the dashboard:

Node-RED - Home automation flow> devices - change

At this point we connect the input (input) of one of these two new nodes to a node output "ACTION", The other at the exit of the node"ACTION”Remaining.

These two type nodes "Change"Serve to transform the payload procoming from the virtual device in two payloads understandable to the physical devices that we will reach via MQTT. We will transform both payloads ("YOU OPEN"Or"CLOSE") In"ON"

Now click twice on the first node appena dragged and connected with the output "YOU OPEN"Of the node"ACTION"To open the call configuration dialog box"Edit change node"

Node-RED - Home automation flow> devices - change - details

Now let's enhance the field "Name"With"OPEN to ON"And the field"to"In"ON"

Node-RED - Home automation flow> devices - change - details-2

Finally, click on "Done"; then we repeat all the proalso with the second type node "Change"Calling it in this case"CLOSE to ON"

At this point, in the list of node types, we find and drag into dashboards the two type nodesMQTT out"

Node-RED - Home automation flow> devices - mqtt out

Then we connect the output of one of the two nodes "Change"Earlier than one of the two"MQTT out” appena dragged, then the output of the other node "Change"To the input of the"MQTT out”Remaining:

Node-RED - Home automation flow> devices - mqtt out - connection

The two nodes "MQTT out”Are the final step in our transformation flow. In practice, when the payload "ON"It will come to one of the two (depending on the path taken in the flow), it will be sent together with the correct MQTT command then to the correct physical device.

Now click twice on the first node "MQTT out"To open the call configuration dialog box"Edit mqtt out node"

Node-RED - Home automation flow> devices - mqtt out - details

Now you need to configure the MQTT server (which we have already defined previously) and the topic to send.

We have hired previously that the commands should be sent to PULSANTE1 in the event of opening and al PULSANTE2 in the event of closure.
Therefore, the configuration of the field "topical"Of the node" MQTT out "that closes the branch"YOU OPEN" Sara:

cmnd / PULSANTE1 / POWER

then configured as follows:

Node-RED - Home automation flow> devices - mqtt out - details-2

The configuration of the other "MQTT out" node will be identical, but with "PULSANTE2" instead of "PULSANTE1

At this point the flow will be complete, but we will also add a type node "Debug”To verify the correct functioningnament:

Node-RED - Home automation flow> devices - debug

Once the node is dragged into the dashboardDebug", We connect the input to the outputs of the two type nodes"Change"

Node-RED - Home automation flow> devices - Deploy

Finally, click on the "Deploy" top right.

Node-RED - Home automation flow> devices - Deploy

At this point the flow will be operational.

EXPLANATION OF OPERATION

Starting from this moment, whenever a topical "cmnd / gate / share”Will be forwarded to the MQTT broker such topical will be received by the first node of the flow ("MQTT in") Which will send it to the second node ("Switch") Which in turn, based on the proupon configuration, it will send a possible payload “YOU OPEN"Or"CLOSE"Towards node"Change”Appropriate which follows. Any different payloads will be discarded.

The knot "Change"(Which it is) will convert the payload into"ON", Finally it will send it to the node"MQTT out”Appropriate following that prohe will then forward to the broker MQTT the command, which will be eithercmnd / PULSANTE1 / POWER ON"Or"cmnd / PULSANTE2 / POWER ON”Based on the branch crossed in the flow.

The sending of these two commands will take effect, as hoped, on the physical MQTT devices to be controlled.

TESTS

Performing a test is simple: just send a command: "cmnd / gate / action OPEN"Or"cmnd / gate / action CLOSE”And check that at Node-RED the knot "Debug"First implemented downstream of the node"Change"Has received an" ON "payload, a code that occurs after the crossing.

To do this, just use any MQTT client. In our case we show the example of sending the command “cmnd / gate / action OPEN”Through MQTTBox, a common client for Mac:

MQTTBox

Coming back up Node-RED - if everything has worked correctly - by clicking on the node "Debug"And looking in the right column, it should appunequivocally give the log entry of the receipt of the payload "ON", Payload therefore also received from"MQTT out”And therefore consequently published on the broker.

Node-RED - Home automation flow> devices - debug log

FLOW FROM ACTUATORS AT HOME AUTOMATION

At this point we have to define another flow, indeed twins:

  • one that generates a topical telemetry that domotics interpret as "open gate"Against the activation of the PULSANTE1 actuator, which we will call"stat/cancello APRI";
  • a twin that generates a topical telemetry that domotics interpret as "closed gate"Against the activation of the PULSANTE2 actuator, which we will call"stat/cancello CHIUDI"
FIRST FLOW

Let's go back to our dashboard Node-RED. Let's add a node "MQTT in"

Node-RED - Devices flow> home automation - mqtt in

Double click on the node appena added to configure it via the dialog box that opens, “Edit mqtt in node"

Node-RED - Devices flow> home automation - mqtt in - details

In the field "Server"We will indicate our usual MQTT broker; in "topical”We will insert the topical telemetry provided by the actuator PULSANTE1, or "stat / PULSANTE1 / POWER". In the field "Name", We will put"PULSANTE1"

Click on "Done”To return to the dashboard.

Now let's add a node "Switch"Which we will use as a" filter "for all possible payload telemetry devices other than "ON". Let's connect it with the output of the "MQTT in" node:

Node-RED - Devices flow> home automation - switchAt this point double click on the node "Switch"To access the dialog box"Edit switch node"

Node-RED - Devices flow> home automation - switches - details

We insert in the field "Name"Check" label, and "ON"In the condition verified in the box below"Property"

Click on "Done”To return to the dashboard.

Now let's add a node "Change"Which we will use to convert the payload"ON"In"YOU OPEN". Let's connect it to the node output "Switch"And let's click on it twice to open the dialog"Edit change node"

Node-RED - Devices flow> home automation - change - details

Let's set the field "Name” a “ON to OPEN"And the field"to” a “YOU OPEN"

Click on "Done”To return to the dashboard.

Finally we add now a node "MQTT out”Which we will use to send virtual telemetry to the broker. Let's connect it to the node output "Change"And let's click on it twice to open the dialog"Edit mqtt in node"

Node-RED - Devices flow> home automation - mqtt out - details

Let's set how "Server"The usual MQTT broker, as topic"stat / gate" and how "Name"We set"Apertura

Click on "Done”To complete the flow and return to the dashboard.

Now connect the node output "Change"At the node"Debug”Added in the flow definition phase from home automation to devices, just to have a debugging tool in case of testing.

Node-RED - Devices flow> home automation - first flow completed

Now, selecting the flow with a mouse click appen defined, prowe see a CTRL + C / CTRL + V to duplicate it. Change it then replacing "PULSANTE1"With"PULSANTE2","YOU OPEN"With"CLOSE" and "Apertura"With"Closure"

Node-RED - Devices flow> home automation - both flows completed

Finally click on "Deploy"To complete the work.

TESTS

As in the case of the home automation flow> actuators, it is possible (not to say necessary) to test these two new flows by manually sending the topical Test MQTT via the usual MQTT client.

Conclusions

At this point there will be a "virtual device"Which will respond to commands

cmnd / gate / action OPEN
cmnd / gate / action CLOSE

and in response to these commands "will respond" with very precise telemeters:

stat / gate OPEN
stat / gate CLOSE

thus allowing us to configure it, as it were a single real MQTT device, in our personal domotics, whatever is ours HUB personal.


This projet is applicable (with the necessary customizations) in multiple scenarios: to domotize gates, tapparelle, anti-intrusion systems and much more: any element that is domotizzato through more actuators / elements MQTT can be "virtualized" through the techniques explained in this projet.

What is important to understand is:

  • what are the topical control for the various MQTT actuators;
  • what are the topical telemetry return;
  • what are the topical to configure within thepersonale HUB to define the virtual entity that rappwill re-establish our home automation accessory through the actuators.

It will be enough then applect the techniques explained in this projet to make it all work.


Home Assistant Official LogoATTENZIONE: remember that there is on our FORUM community an ad hoc section dedicated to Home Assistant, for any doubt, question, information on the specific merit of these components.


telegram

Stay up to date through ours Telegram channel!