Mac Home Automation

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
=====MQTT Client=====

MQTT Client Plugin for connecting to any MQTT Broker and linking XTension Units with their topic values. MQTT Topics are not usually simply numbers that can easily be imported but often complex data structures or JSON data that requires some knowledge of both MQTT and the devices you’re trying to connect to. This is an advanced topic and this wiki page does not try to be a tutorial on all things MQTT nor on the specific devices you’re trying to get info into or out of. As a tool to make the implementation easier I strongly recommend the free and most excellent [[https://apps.apple.com/us/app/mqtt-explorer/id1455214828|MQTT Explorer]] (apple app store link) This makes it MUCH easier to figure out what the data really looks like so that you can decide how best to deal with it in XTension.

MQTT has become quite popular in the home automation and IOT space. This plugin makes it possible to get most, if not all of those devices talking to XTension. 

-----

====Installing an MQTT Broker:====
If you’re already using MQTT then you can skip this step. MQTT Brokers are available for pretty much every platform. This plugin can connect to any of them but if you wish to install it on your Mac you can follow the instructions for the free broker at [[https://mosquitto.org|Mosquitto.org]] This is what I have used in testing. It requires some configuration via the command line but those instructions are pretty straight forward and are documented at their site. If you already have a broker running on any other system you can continue to use that,

For the most basic install specifically the changes that need to be made to the mosquitto.conf file are:

Under General Configuration towards the beginning and before the per listener section please add:
<code>
allow_anonymous true
</code>

Further down under the listener section you must create a listener entry with the default port number. The default line in the configuration is the word listener commented out like:
<code>
#listener
</code>

please replace that with a line like:

<code>
listener 1883
</code>

and then restart the process. If running on MacOS and installed through Brew you can do this by issuing the command:
<code>
brew services restart mosquitto
</code>

this will also take care of having it restart after the machine restarts

-----

====XTension Settings:====
{{:supported_hardware:mqtt_settings.png?600 |}}

**Address and Port:** The IP address or DNS name of the MQTT broker you wish to connect to. The default port for MQTT is 1883 but you can change this if necessary for your environment.

**User and Password:** Currently there is no support for either encrypted connections or user/password signons. The MQTT server needs to be configured to allow anonymous connections. Since most mqtt brokers are setup on your internal network and are connected to by very simple IOT devices this may not be that uncommon. Support for other connection methods is in the works. If this is something you need sooner please let me know and I’ll move it up the list.

**ESPresence:** The first of the sub plugins, meta plugins? is direct support for ESPPresence. These use a more complex data structure across several topics and JSON formatted data which can be difficult to implement with the built in “simple” devices and so is supported separately. If you wish to use these devices with XTension check the box to enable it. Initially you should leave the Automatically Create Units checkbox turned on so that it will create the right Units for all the discovered devices automatically. Later this can be turned off if you do not wish to get Units created for everyones devices that pass by the range of one of the receivers. More information on this is available at the [[supported_hardware:mqtt:espresence|ESPresence Wiki Page]] A direct link to this page is available in the link on the XTension settings window as well.

-----

====Special Device Support:====
Since some MQTT devices use complex data structures or multiple topics they can be difficult to implement with the simple unit types documented below. In those cases I will add direct support for them into the base MQTT plugin or provide separate plugins that talk to this one so that only a single connection to the server is necessary. 

The first of these is support for the excellent and popular [[https://espresense.com|ESPresence]] devices. This is incomplete and supports only the location information at the moment and not sensors though that is coming in the future. In the meantime you can use the Unit types documented below to get that information out of them. See the link in the list blow for more information on what that is and how to use it. I have plans currently to also directly support Tasmota devices and ESPHome devices. If you are interested in using those or have other ideas of what you’d like to see directly supported please get in touch and I’ll add it to the list.

Directly Supported Special Device Types:
  * [[supported_hardware:mqtt:espresence|ESPresence]]


-----

====Parsing JSON Payloads:====

Many devices send json packets instead of simple values. There are now a couple of ways to parse them. In XTension 9.5.2 I’ve added a section to all the non-special Unit configurations for simple parsing of JSON payloads. 

{{ :supported_hardware:mqtt_json_payload_settings.png?500 |}}

After entering the Topic Path, check the Parse Payload As JSON Checkbox and enter the path through the JSON to the value you would like to get to. The path through the JSON is also delineated by the same backslash character as the topic path. In the case of JSON dictionaries the section of the path must be the key into the dictionary to move to. In the case of a JSON Array it must be an integer value for the element in the array you wish to navigate to. Once the end of the JSON Path is reached whatever value was there is then passed on to the regular value parsing as handled by the specific Unit type. So Numerical values are converted to numbers, enumerated values will look into the result for the enumerations and the Description Unit will place the result in the Description field. 

This is available as an option for all non-special device units.

If you have several values to get from a single JSON packet you can create as many Units with the same Topic Path as you wish but with different JSON paths to get the specific data you need.

If this is not enough to parse a more complicated JSON packet please see the [[dictionary:more:parsejson|Parse JSON Verb]] for doing the decoding in your ON script.

For more information and examples please see the [[supported_hardware:mqtt:jsonpath|JSON Path Information Page]]

-----


====MQTT Unit Types:====
XTension provides several Unit types for getting various types of information into and out of an MQTT server. These work well for simpler data types and can be expanded via scripting to get pretty much any data into or out of the server. If there is a method for moving data around that is not yet supported that you believe would be useful please let me know. Not all that I have planned are implemented at this point either so do get in touch with your thoughts. 

These units separate the input and output processing so that it’s possible to read a simple value and generate a more complex or totally different output.

Except for special device types, no attempt is made to automatically create Units for MQTT devices. You will need to know the topics and how they work in order to get them working with XTension.

**Unit Address Note:** Unit addresses are handled differently for the below units than normally within XTension. Please do not fill in the Address field when creating new Units of these types. It will be filled in automatically with a unique id after you save the device. The addresses are not used in the traditional sense for these.

All direct units will have an **MQTT Topic Path** field. Enter the full path to the data you want here. If you enter a wild card here then the resultant data will all be processed by the Unit. This MIGHT be the thing you need to do but almost certainly isn’t so unless you know exactly why you’re doing that, don't.

Publishing of information to the broker is handled the same in each Unit type and is documented below the Units section.

===Numerical Unit:===
{{:supported_hardware:mqtt_numeric_unit_top.png?300 |}} The **Numerical Unit** is to receive values that are numeric, or mostly numeric in value. Most “numeric" MQTT payloads are not just the simple number but contain other text or suffixes that complicate turning it into a simple value. This Unit will find the first string of numbers in the payload and set the value of the Unit to that. 
  * “123” would become 123
  * “12.4F” would become 12.4
  * “Temperature: 12.4 F” would become 12.4
  * “floor 4 section 13” would become 4

Add the appropriate **MQTT Topic Path** 

For most numerical Units you would want to leave the **Dimmable** checkbox checked so that it will display the numerical value. If you turn this off then the Unit will display off while the value is 0 and on when the value is non-zero.

Remember to leave the **Address:** field empty when creating a new Unit of this type. It will be filled in automatically when you save the Unit.




===Enumerated Unit:===
{{:supported_hardware:mqtt_enumerated_unit.png?300 |}} The **Enumerated Unit** will make an effort to match all or a part of the payload with one of the enumerations you enter into the **Enumerations** field. Into that field enter a comma delimited list of the enumerations to match. These will correspond to what numerical value is set. The first value is 0 or off, so in the example the Unit will display in the off condition when the payload matches closed, and will have a value of 1 and be on when the payload matches open. If a device has more than 2 enumerations then the value will just be a number that corresponds to the location of that enumeration in the list.

Do not use spaces after the commas in the list or those will be matched as well. 

The list is not case sensitive so Closed would match as well as closed.

This is separate from the Unit Enumerations on the Display tab of the Edit Unit dialog. This way you could match a sensor that sent closed,open but display in the interfaces “The Door Is Open” or “The Door Is Closed” by using both enumeration types.

**Match Type:** There are 4 ways to compare the payload with the enumerated values. More are possible so if you need say a Grep entry let me know and I can move that up the list of things to work on.
  * **Exact Match** 
  * **Included Anywhere** the payload contains the enumeration string somewhere
  * **Starts With**
  * **Ends With**

You can also include the special value <default> which would set the value of the Unit when no other enumerations have matched. For example the enumeration string “<default>,high,low” would turn the unit off if the payload did not match high or low. You could then use the Unit Display enumerations to have the value display as something like “Unknown Speed Setting” if there is no match.


For more information on the Unit Display Enumerations please see the [[xtension_manual:editunitdisplay|Unit Display Tab]] documentation.

Once a Unit has Display enumerations you can use them to set the value. Normally you would pass a numerical value to the set value command, but once you add enumerations you can also pass those as a string. [[dictionary:unitinformation:setvalueof|set value]] of “MQTT Enumerated Unit” to “closed” to get the current value as a string of the enumeration use the [[dictionary:unitinformation:enumeratedvalue|enumerated value]] verb. remember these are separate from the MQTT enumerations field and must be entered on the Display tab of the edit unit dialog.


===Place Value In Description Unit:===

The last of the direct Units has no settings other than the **MQTT Topic Path** field. Whenever a new value is received it will be placed into the Description of the Unit. This can be displayed in the various interfaces directly or accessed via scripting to get the description. You can also create an **on descriptionChanged( newDescription)** event in the Unit script which will be run and the new value passed to your script for further processing.

Eventually we will have a specific unit for processing via script and another specifically for processing JSON or XML values but for now this is the easiest way to get to more complicated string values. If the value is JSON use the [[dictionary:more:parsejson|Parse Json]] verb to turn it into Apple Event Records or arrays so that you can get the data out you wish.

Since this does not directly affect the value of the Unit, you can set it from that event to whatever you wish. If you were to parse a temperature out of a JSON string this way you could set the value of the unit to the new value. If there are multiple values you are getting out of the passed payload you can of course send the various parts to whatever other units necessary to get all the data into XTension.

-----
====Using a Single topic to get several data points:====

The topic field in these Units do not need to be unique. You can create as many Units pointing to the same topic as you wish. For example imagine a topic where the payload was the temperature and then a string saying “OK” or “ALARM” so you might see entries something like:

Furnace Temp: 234.5°C OK \\
Furnace Temp: 423°C ALARM

Create a Numerical Value unit to the topic to pull out the temperature value, and another Enumerated Unit to match OK and ALARM to turn on if the same payload ended in Alarm.
----

====Publishing Payloads:====
{{ :supported_hardware:mqtt_payload.png?600 |}}

Each of the 3 Unit types above have the ability to publish their local value changes back to the MQTT Broker. This can be complicated to rebuild a valid string of information so several ways to do this are provided.

**Separate Publish Topic:** Some devices expect to get commands on a separate topic than where they send their state information. If this is one of those you can enter a different topic path into this field and it will be used when sending a local update to the broker. If the field is blank then the same topic that is read from will be used.

**Payload Creation Method:** 
  * **None: Unit Is Read Only** No local changes will be sent to the server. You should also mark the Unit as read only and ignore clicks in the edit unit dialog so that the user interface for the Unit reflects it’s non-controllable nature.
  * **Raw Numerical Value** Just the number, no special formatting no suffixes, just the numerical value of the Unit.
  * **Formatted Numerical Value** The Numerical value after passing it through the Format processing and adding a Suffix if one is specified. You can specify both things on the [[xtension_manual:editunitdisplay|Display tab of the Edit Unit Dialog]].
  * **Current Label** Whatever the Unit is displaying in XTension interfaces will be sent as is. If you have on/off labels set, or if you’re using enumerations for display, whatever that turns out to be after all the processing is done is the string that will be sent.
  * **Enumerated Value** This is separate from the Enumerated field in the Enumerated Value MQTT Unit. This references the Enumerations field on the Display tab of the Edit Unit Dialog. Whatever enumeration there matches the value of the Unit will be sent.
  * **Unit Description** Another way to script sending arbitrary strings. Whenever the Description is changed by a script that new value will be sent to the topic.
  * **Unit Property** Use the [[dictionary:unitinformation:getunitproperty|Unit Property]] verbs to set a value to a unit property named “MQTT Payload” and whenever that is updated by any script that string will be sent.


**QOS and Retain** The retain flag and the Quality Of Service can be set for outgoing messages with the QOS popup and the retain checkbox.

**Via script command**

lastly a scripting command is added just called “publish” where you pass the topic and a string and it will be sent to that topic no matter where it comes from or if it is associated with a Unit or not. Any value can be published to any topic this way.

If you’re running the script inside the Unit’s on or off script, or in the Interface Script no other formatting is needed. You can simply call:

<code>
publish( “message/sam/urgent”, “Your fire alarms are going off!”)
</code>

if you are running the command from any other script that is not directly associated with the unit or interface then use the tell interface form of the verb like:

<code>
tell xInterface “name of my MQTT Interface” to publish( “message/sam/urgent”, “Your fire alarms are going off!”)
</code>