Differences

This shows you the differences between two versions of the page.

--- current:beta [2021/04/25 17:04] – 9.4.40b1 James Sentman
+++ current:beta [2024/02/05 17:51] (current) – 9.5.5b1 James Sentman
@@ Line 1: / Line 1: @@
-=====Beta Versions=====
+=====Beta Version: 9.5.5 beta 1=====
-A notarized beta of XTension 9.4.40 is available as of 4/25/2021
+This is a beta release of the next XTension version 9.5.5. This release contains no new features and is concentrating on fixing potential issues and making the scripting engine more robust and reliable.
-[[https://MacHomeAutomation.com/files/xtension/xtension_9_4_40b1.dmg|XTension Beta Version 9.4.40 (build 1040)]]
+**This version makes no changes to the format of the Database so returning to a previous version can be done simply by placing the previous version of the application back into service. The default location of the XTension Support folder and possibly the Database within it are changed and it will be necessary manually move the folder back to the Documents folder if you need to roll back to a previous version. See below for more info.**
-This version includes a beta plugin for SDR radios via a wrapper for the rtl_433 application. The current support is incomplete but should support everything that the now more difficult to obtain RFX radios and most, but not all, of the speciality devices that the rtl_433 program supports. I will endeavor to complete special handling for everything that needs it before the release but if a specific device you are using does not work with this beta but is supported by the rtl_433 software please let me know and I’ll make sure I support it’s custom data output. As of this release every weather type instrument should work from Oregon Scientific like the RFX and also those from accurite and others that are supported by the rtl_433. For more info on the rtl_433 software please see [[https://github.com/merbanan/rtl_433|rtl_433 git hub page]] for more info on what it supports or doesn’t. Please note that the rtl_433 plugin is a receive only device and cannot transmit to control the devices you are receiving.
-NOTE that rtl_sdr supported radios are tested. The software is present for the soapy_sdr supported radios but as I don’t have any of those I have not tested and you may encounter problems. If you are using one of those please send me any log output if there is any problem. If you don’t have any problems please let me know that also.
+If you’re not interested in the history and why the support folder needs to be moved skip to the next section.
-In addition to the rtl_433 plugin this build also includes a fix for the database sharing where it might throw an error because of it trying to use the wrong version of the python libraries to connect via a secure connection. This should be fixed in this version as well.
+===Changes to the location of XTension’s data files===
+There are 2 ways that XTension can be installed. If the application is placed in the Applications folder then a folder called “XTension Support” was created in the Users Documents folder and the Database and other support files were placed there. You can also continue to use XTension as it was in days of old and run it from any other folder on your drive. If you do this the database and some support folders are created along side the app inside this same folder but an XTension support folder is still created in the Documents folder.
-There is not yet a wiki page devoted to the SDR radio info so I’ll place some of this info here:
+The location of the XTension Support folder has become an issue with Apple’s iCloud feature of syncing your Documents folder with iCloud. This service remains potentially a problem for many things but can and does cause corruption of the XTension database as files seem to be updated too often for it to remain in sync. It seems to restore files that were deleted as well as replacing new versions with older versions on occasion.
-====XTension Settings:====
+It is my advice to not use this iCloud feature at all as it causes the same problems with other files you may have stored in your documents folder. If you have this turned on you should turn it off immediately to avoid losing information. In the meantime moving the XTension Support folder to the Application Support folder will solve the problem for XTension **unless you are using XTension outside of the Applications folder but still somewhere under the Documents folder** in that case you must either switch to running from the Applications folder or make sure you are not using the iCloud Documents sync functions.
-Setting up the SDR should be very straightforward assuming that you have only one radio connected to the Mac in question and you only want to receive already supported devices. The current XTension configuration screen, subject to many changes before release, looks like this:
-{{:current:rtl_433_settings.png?400|}}
+===Process of moving the files===
+When first run XTension will look for an XTension Support folder in your Documents folder. If it exists in that location but not already in the Application Support folder it will move it and startup normally. Giving you a startup message window telling you exactly what it did. If there is already an XTension Support folder in your Application support folder and it does not contain a database then it will be renamed and the current XTension Support folder will be moved to the new location. If it does contain a database, even if it is an old one, then it cannot move it and will startup with that database along with a message that what appears to be your current database could not be moved and you will have to resolve the conflict manually. There should not be a live database in the Application Support folder however much earlier versions of XTension did use this location and so there may be old and outdated versions of information there that you may have to remove.
-For any radio connected to your mac and receiving the supported 433 devices you should leave this as is.
+===Switching to Running in the Applications Folder===
+If you are running XTension in the legacy way of being anywhere but the Applications folder you do not have to move anything manually yet. However at some point Apple may enforce that applications have to live in the Applications folder. If you wish to make the change now you can move the XTension application into the Applications folder. Then move all the support folders and database from the folder you used to have the App in to the Documents/XTension Support folder. This includes the “icons”, “sounds”, “views” and other folders that will be there along with the XTension Database bundle. When you first run this version of XTension from the Applications folder it will move that Support folder to the Application Support folder and run normally from then on.
-If you are using a supported SDR on the Mac itself then none of the following about the raspberry pi is needed. This is only needed if you wish to use a pi as a remote host for an SDR. If you wish to run a remote connection to a host pi then select “remote via ssh” from the popup menu labeled “connection type"
+===Returning to a previous version===
-====Raspberry Pi Remotes:====
-**Note that this section is only necessary if you are wanting to connect to a radio hosted remotely via a raspberry pi. Otherwise skip this!**
+If you find this version to cause new problems, please do not just return to the previous version without letting me know exactly what the issue was that caused you to need to do so or it will be much less likely I can get it sorted out by the release. Without XTension running just move the XTension Support folder from the Application Support folder back to your Documents folder and then start the previous version. The Application Support folder is generally hidden from the User and may not be easily visible from the Finder. If so you can use the “Go” menu in the Finder to reveal the Library folder and inside that will be the Application Support folder. If XTension is still running you can use the Reveal Database In The Finder menu item from it’s Database menu, Then Quit XTension and do the move manually before replacing it with the previous version.
-Running SDR software on the Pi takes a considerable amount of resources. I would not recommend running this on a Pi Zero or a very old device. It runs just fine on a previous generation Pi 3B+ and will run perfectly on the newer pi4’s without having to purchase the more expensive ones with more memory unless you want to host multiple radios.
+====Changelog:====
-If you wish to have many radios installed away from the XTension mac you can install the rtl_433 software on a raspberry pi and place that anywhere on the local network. You can create additional interfaces to connect to any number of raspberry pi hosts. This functionality is also working as of this beta version.
+  * At startup if XTension finds that your database is being shared to iCloud it will open a startup messages window advising you of this and suggesting that you turn that off. This info is also saved in the database so that if someone sends me a database to debug I can check easily if this is the source of some of the problems.
+  * If the database has already suffered some kinds of corruption from being shared to iCloud there are many things that it can now clean up. This includes deleted units or scripts that have been replaced by an over zealous iCloud sync service. It cannot replace active files that were replaced or reverted incorrectly but this can still be a major help in getting things sorted out.
+  * The XTension Support folder is moved from the Documents Folder to the Application Support folder. Unless there is already a Database in an older version of the folder that might exist in the Application Support folder. In which case no files are moved or deleted and a message is displayed asking you to move the most current version there manually to make sure you get the right one.
+  * Improved script error logging. Previously errors that were generated while running handlers in a different script were sometimes not logged causing instead a generic error in the script doing the calling. They will now log more useful information for debugging purposes to find out just went wrong in the other script.
+  * Quite a few major OS versions now Apple changed Applescript to be somewhat, sometimes, unpredictably threaded. This allowed the application to continue to process commands and events while another script might have still been running. This led to confusion and things no longer happening in a linear and logically consistent order. I fixed that back in the day by pausing the command queue while a script was running so that even if the app got control back no more commands would be processed until the current script being run was complete. I recently added some new RTS cameras of higher resolution to my system which is pushing my older Mini a bit and I started getting more regular strange scripting errors that rarely or never happened before. In studying the problem I realized that not all scripts are purely started by received commands or scheduled events anymore. The Idle handler and the DoLater commands use program timers to pump the script callbacks. These were still potentially happening while a script was running. This usually worked OK but occasionally there was either a race condition or a script would never return causing the timeout and an error message that the script environment was unstable and should be restarted. All timer pumped scripts will now defer until the current script is finished running. If you are having any odd and difficult to duplicate script issues and if you make use of either the idle callback or Do Later commands then this will solve those issues. There still may be one more path for things to go wrong, or there may yet be other applescript issues but 99.9% of my issues have been fixed by this change. There is a debug menu item to turn on logging of these occurrences. If you’re curious or if you want to see if it’s really catching anything you can turn that on and watch the log.
+  * Further new script debugging I’ve added the ability to log runtimes of individual scripts. You can turn this on in the Gear toolbar menu for any script be it an On/Off script or a Global Script. This may be useful if there is a script that sometimes seems to take a lot longer than it should be. Especially if that script is talking to an external app to wait for something.
+  * Errors that could be logged when deleting more than a single page of Units from a list is now suppressed. It was never a useful error message so no fix was necessary, it’s just confusing debug logging.
+  * Errors that would sometimes be logged when closing the Interface list window while doing CPU and Memory use displaying in the “show more information” mode are no longer logged. Again these were just over zealous debugging messages and did not indicate a real problem.
+  * Fixed the incorrect display of version information. The version logged and in the About window will properly show 9.5.5 beta 1 in this case rather than 0.0.0.whatever as it did in some previous versions.
+  * Made some additions and changes to the runtime debugger window to better sort items when there are more than 10k of any individual kind. I also added deltas to the memory used and object count displays to make figuring out things easier. Most of you have never opened this window from the debug menu and won’t ever need to but it is helpful while I am testing and debugging and could be useful to help gather more info if needed from any of you.
+  * Added another debug menu option to log the current state of the globals “thisUnit”, “thisScript” and “thisInterface” as well as the value of gCurrentScript internally which is used to direct handler calls of global handlers and other such things. These should all be empty with gCurrentScript being nil at any point that a script is not running. If there remain issues with scripts failing to exit or otherwise hanging up in the background these may be left set to incorrect values causing interesting results. If so this will show that it has happened making debugging easier.
-The connection to the raspberry pi is via an ssh session. There is no way to save the password into the interface so you must configure your mac and the pi to connect via ssh without a password. If you have never done this on your mac you must first create an ssh key that you can share with the pi. Start the terminal and enter this command:
+[[https://machomeautomation.com/files/xtension/xtension_9_5_5b1.dmg|XTension 9.5.5 beta 1]] 2/5/2024
-<code>
-ssh-keygen -t rsa
-</code>
-The switch for -t rsa may not be needed depending on the OS version. Use the default key name and do not set an encryption password for the key.
-now you need to send that to the raspberry pi so that you can connect from your mac to the pi without having to send the password each time. This is also done via the command line on the Mac to the pi. (make sure you have changed the default password on the pi and set it’s hostname to something other than the default of raspberrypi.local or you’ll run into trouble later)
-the command to register your ssh key with the pi is:
-<code>
-ssh-copy-id pi@thePiAddressOrName.local
-</code>
-if your user is other than the default pi make sure to change that. You can connect via the IP address of the pi if you have assigned it a static IP or via it’s hostname.local as long as you have changed it from the default of raspberrypi.local or you’ll not be able to setup more than a single pi on the network.
-You may have to answer several questions about are you sure if you have setup other pi’s as the keys will have changed and ssh will warn you that it can’t verify that you’re connecting to the right machine. If you have setup other pi’s in the past it may refuse to connect at all because it’s in your known host file with a different machines key. If thats the case you need to edit your known hosts file to remove it. If you haven’t ever done this you can do it at the command line via something like:
-<code>
-nano ~/.ssh/known_hosts
-</code>
-then use the control-w to find “raspberrypi.local” and delete that whole line. Then save by doing a control-o and then exit with a control-x. There are many many other resources for more info on doing this on the web if you need more info.
-Once that is done and you have copied the key via the above please do test that you can ssh in without a password. It may require that you do it once manually and OK the fact that it can’t verify the key if you have every connected to another one and you need to do that before the software can connect without your attention so once that has completed test it via ssh’ing in via a regular command like:
-<code>
-ssh pi@yourPiAddress.local
-</code>
-it may ask you “are you sure” because something cannot be verified. Please answer yes and if you end up at the ssh prompt for the pi then all is well. You can then move on to compiling the rtl_433 software on the pi.
-====Installing rtl_433 on the pi====
-There are 2 steps to installing the software on the pi. You need both the low level SDR radio software and the rtl_433 software. They are very easy to install.
-===Load Up The Pi:===
-If you haven’t already load up the most recent pi software and do any updates available. I wont post instructions for this here, if you can’t do this and can’t find instructions on the internet for it you probably shouldn’t be trying this ;) The program will work with the “lite” versions of the raspian images or the desktop and full installs but those are not necessary and if you’re not using the pi for desktop work you will only be wasting space by using those.
-===Make Sure GIT is installed:===
-try this command to see if git is installed on your device or not:
-<code>
-git --version
-</code>
-if you get a “command not found” error then you need to install git. That command is:
-<code>
-sudo apt install git
-</code>
-it will tell you that another 33meg of disk space or other is needed and you need to say Yes so that it will install
-Once that is finshed you can retry the git --version command. As of this writing the version installed is 2.20.1
-===Install Pre-Requisites:===
-The following packets are required to finish the rest of the install. If you already have any of them then the installer will just skip that part:
-<code>
-sudo apt install cmake libusb-1.0-0-dev build-essential autoconf libtool pkg-config
-</code>
-===Install rtl-sdr:===
-<code>
-git clone git://git.osmocom.org/rtl-sdr.git
-cd rtl-sdr
-mkdir build
-cd build
-cmake -DINSTALL_UDEV_RULES=ON ..
-make
-sudo make install
-sudo ldconfig
-</code>
-===Install rtl_433:===
-<code>
-git clone https://github.com/merbanan/rtl_433
-cd rtl_433
-mkdir build
-cd build
-cmake ..
-make
-sudo make install
-</code>
-===Setup For Remote:===
-Once you’ve done all that you can select “Remote via SSH” from the Connection Type popup as shown above and unless you changed the install locations on purpose the defaults should all work just fine and you can connect an interface to a device connected via USB to a remote raspberry pi. If you need to have more than one radio connected to the same mac or raspberry pi that is also possible, but read on for even more configuration:
-===Multiple SDR’s:===
-XTension will support multiple SDR radios on the same machine via the serial number of the device. Unfortunately all the devices I have seen come with the serial number set to “0000001” or something similar. So you have to use the rtl_eeprom tool to set the serial number before filling it into the XTension interface as shown above. This will work for both local SDR’s as well as those connected to remote Raspberry Pi hosts. I will not include instructions for how to use the rtl_eeprom tool here, please look those up on the wiki page for the rtl_433 page linked to above. You’ll find the app compiled for the Mac inside the plugin folder which is inside the XTension app. If you control click on XTension in the finder and select “show package contents” and then open the Contents folder, and then the Resources folder and then the Plugins folder and finally the “rtl_433.isf” folder you’ll find the supporting applications such as rtl_eeprom. You can take a terminal session in there and then use the app to write a specific serial number or id string to the radio which you can then use in the **SDR Serial Number** field of the XTension settings window as shown above.