Differences

This shows you the differences between two versions of the page.

--- supported_hardware:diy [2017/04/16 18:00] – external edit 127.0.0.1
+++ supported_hardware:diy [2023/02/13 14:52] (current) – external edit 127.0.0.1
@@ Line 1: / Line 1: @@
 =====Do It Yourself=====
-If you want to add some device that XTension doesn't currently support, you can roll your own using a wonderfully simple method using AppleScript and some special tools that XTension provides.
-This method lets you connect to a native serial port, or a [[:tech_notes:usb_serial_adaptor|USB-serial adaptor]], or even a ethernet based serial port, and you can switch between those connections without changing your AppleScript !!
+**This page is about the new DIY plugin which replaced the original DIY plugin in XTension 9.4.31.** The original plugin is deprecated and will be removed completely in a current version. Please try your DIY interfaces with the new plugin and let me know if there are any issues. See the section below on converting from the legacy DIY plugin, there are some minor differences in the initial setup and configuration. The legacy plugins original documentation has been moved [[diylegacy|here]],
-In general, what we do is to create an 'interface' in the XTension communication prefs, establishing the 'port' and serial preferences, and create two 'well-known' scripts that will handle the opening of the port, and respond to input received from that port.
+The DIY plugin provides a way to easily connect to a device that does not have a specific plugin yet. The DIY plugin will provide a pipe to either a TCP socket on a remote device or a serial port connected directly to the XTension machine via a USB/Serial adaptor or Bluetooth Serial Port. Use the [[dictionary:xtension:senddata|Send Data]] verb to write data to the device and receive its response in the Plugin script’s Data Available event.
-For example, I have a very old serial port multiplexer **"the PIPE"**, that I designed and built over 25 years ago that lets me switch between 16 ports, regardless of the serial port settings.  In this example, I've set up two iMacs with USB-serial adaptors, both of which then connect to two ports on **"the PIPE"**.  It's not obvious here, but **"the PIPE"** is already configured to pass any data received from one port to the other port... just a 'pass-thru' connection.
+====XTension Setup:====
+To create a new DIY plugin instance visit the **Interfaces** window and click the New Interface button. Select “DIY Interface” from the Device popup. At the moment both this new plugin and the original DIY plugin are included in the XTension default package of plugins so be sure to select the “DIY Interface” that shows a version of at least 2.0b1 and not the one entitled “DIY Scripted Interface”. Please do not create new interfaces based on the original plugin as it will be removed in a future version of XTension and will not be included in the Catalina package.
-Here I'll show how to create the port, send data from one iMac to the other, receive the data, and conditionally send a message back to the first iMac.  (note that the two iMacs are named "Wallace" and "Perelandra"...) \\
+{{:supported_hardware:diy_settings.png?550 |}}
+**Port Selection:** The DIY plugin can talk directly to a serial port or make an outgoing connection to a TCP socket. There is also the option in the Port popup to listen for incoming connections. This is not implemented in the current beta version of the new plugin. If this is something you would find useful sooner please let me know and I will move it up the priorities list. Most network to serial adaptors such as the wiznet card can be run in either server mode meaning that we would connect to it which is how you must do it at the moment, or in client mode where it would connect to us. To allow this is the purpose of the incoming connection support and will be implemented in a future version. Make sure not to set a wiznet card to “mixed” mode as after you are disconnected for a while it will stop listening for you to reconnect and instead start trying to connect to you. This means that when you do reconnect it will not accept the connection.
-Let's create the first new interface handler on "Wallace". \\
+**Serial Ports:** If you select a serial port you need to setup the necessary properties of the port such as baud and handshaking that might be necessary here. Most devices will connect at whatever baud it is built for but with the default other settings of no parity, no handshaking, 8 data bits and 1 stop bits.
-In the XTension FILE menu, choose Preferences, and the Communications Tab.
-Then choose NEW...  \\    Here, I've entered the name of the interface, "PIPE", and set the Port and the Serial settings already: \\   \\
-{{:supported_hardware:create_pipe.png|}}  \\
-Notice that you must specify the baud rate, parity, bits per character, and stop bits.  You also have the option of telling the interface to use hardware handshake, and whether to force DTR.  (CTS is by default set on, because we assume that we have a full-duplex connection.)  \\
+There is a very old specification that allowed 1.5 stop bits. This is not supported on the Mac’s hardware at all and so I did not include it in the stop bits popup. If you need this it is possible that it will work if you set the stop bits to 2. This is what the hardware would do under the hood if you tried to set it to 1.5 anyway, so it is worth a try.
- \\
-Notice also that if you choose to have a remote TCP type connection, you probably have to set up the remote IP interface using the instructions for that 'bridge'.  For example, with the Barionet, you use the web browser. \\
-Now, you will want to set up the scripts for this DIY interface.  CLICK on the Received Data Script **EDIT** button : \\
+The **Turnon RTS** and **Turnon DTR** checkboxes will keep those handshaking lines turned on for the entire time the port is open. This is not compatible with a device that uses handshaking (though most do not) but it may be necessary for some smaller boxes who use these lines to power the opti-isolators for the serial data level converters that they use. If you have a device that will not communicate when all the other settings are correct try turning on these lines and it may wake up. The state of these lines can also be controlled with the scripting commands to **setrts** and **setdtr** see the scripting commands section below for more info on using those.
-{{:supported_hardware:edit_pipe_scripts.png|}} \\
-Notice that there are two 'handlers' here.  The first one will be invoked every time you enable this DIY port, or whenever it is automatically restarted after some failure.  In this handler, you will want to specify any handling preferences that are different from the defaults. \\
-XTension provides a verb for setting these :  \\
+The Baud popup contains the standard list of baud rates that most or all USB/Serial adaptors support. It is possible to open the port with a non-standard baud rate if your USB/Serial adaptor supports this. At the moment there is no interface in XTension to setting a non-standard baud rate but if this is something you need please let me know and it can be added.
-{{:supported_hardware:diy_verbs.png|}} \\
-You can use these verbs anytime, but it's the most logical to use them here in the **on init()** handler.  \\
+If you need to send a “Break” to the serial port see the **SendBreak** command below in the scripting commands section.
-The **on DataAvailable()** handler is the workhorse of the DIY interface system.  \\
+It is possible to read the state of the RI and CD line from the serial adaptor. At the moment there is no support for this in the plugin. If this is something you need to do please drop me an email and I will add supporting it to the list.
-Here you put all the code that knows what to do when data is received from your device. \\
+If the underlying hardware supports it it may be possible to properly talk to and configure an RS485 connection. As of this release there is no support for this in the plugin. If it is something that would be useful to you I can move the supporting of it up the priorities list. Please let me know. If you simply need an RS485 connection for something most USB/485 adaptors will handle most of the settings by default that may work just fine. The interface in XTension would only be necessary if these defaults did not work and you needed to change the drive level of the lines, the delay before tx/rx or the loopback settings.
-In this example, I simply write whatever I receive to the XTension Log, but this is a bit different on the 'other side' of this little show-and-tell...  \\
+**TCP Connections:** If you select an outgoing TCP connection you’ll see an additional section of the interface to enter the remote address and port of the device you wish to connect to. The Connection will be opened when the interface is enabled in XTension and held open until the interface is disabled or until XTension is quit. The Serial Port Settings are ignored when connected via a TCP socket. If you’re using a network to serial adaptor such as a Wiznet card or other the serial port settings are usually part of the configuration of that device. There is a protocol for sending these settings through the data connection so that I could implement this for them if needed. I am not aware of any devices that actually support this at the moment, but if it is something you need please let me know and I can look into implementing it.
-Now, on the iMac named Perelandra, which is also connected to **the PIPE** by a USB-Serial adaptor, I have created a similar DIY port, and set up the **on DataAvailable** handler like this :  \\
+If you are connecting to a TCP/Serial device that supports [[https://tools.ietf.org/html/rfc2217.html|RFC2217]] then it may be possible to set the port settings over the TCP connection. As of the current plugin release there is no support for this protocol but if it is something that would be useful to you please let me know and I will bump it up the priorities list. I’d also like to know what device you are connecting to as I’m not currently aware of very many that support this.
-{{:supported_hardware:hello_perelandra.png|}} \\
-This handler will also write the received data to the Log, but will additionally test the input to see whether it contains the word :  "Hello".   If so, it will send a message back to the PIPE, which will be passed thru the PIPE to the port that "Wallace" is connected, and thus it will be received by the DataAvailable handler which will write it to the Log on "Wallace".  \\
+====Parsing Options====
+Parsing a data stream can be difficult and fraught with danger, especially if you are attempting to do it in AppleScript. The plugin has some rudimentary settings that can help in packetizing incoming data so that your **Data Available** or **Binary Data Available** events as documented below are more likely to contain the full packet and only one packet at a time.
-**Update** there are now 2 different onDataAvailable events you can use depending on the type of data that you’re receiving. The default handler now has 2 parameters, theDataAsString, and theDataAsBytes:
+If you are connecting to a true serial port the port will be flushed after opening so you should not receive a potentially large chunk of truncated garbage from the port buffers. This cannot be done for a TCP connection that will be up to the device you’re connecting to.
-<code>
+**NOTE** in the original DIY plugin these settings were not available in the initial plugin configuration and required that you use the [[dictionary:more:setparsingparams|Set Parsing Params]] verb in the **init** handler of the interface script to set them as the port was opened. You can still use the verb to change the options later if need be, but the state during the initial command
- on dataAvailable( theDataAsString, theDataAsBytes)
-</code>
-you can process the data in the original way via “theDataAsString” parameter. Or if you’re doing something lower level and need easier access to the individual bytes in the stream you can use “theDataAsBytes” that is an applescript list with each element being the value of an individual byte that was received.
+===Send the data as soon as it is received:===
+This would be the default selection. At any seemingly random time while receiving data the system may choose to tell the plugin that there is data available to read on the port. The data is then read and the **Data Available** events will be called with whatever is there. It may be a single byte, a truncated packet or many packets stuck together. You can manage this in the scripting if you wish. This is the fastest way to get data into the script.
-When talking to something that uses a binary protocol and not a text or human readable protocol it is possible that AppleScript will have trouble converting those non-printable characters into a string and may give you an error when data is received instead of passing it through to your script. If that happens then you need to use the alternate event called “binaryDataAvailable” which does not try to pass the data to you as text, but only as the binary list:
+===Fixed length packets:===
+If your device always sends the same length of packet you can use the fixed length packet parser. The port is opened and it will read until it receives the requested number of bytes and then pass those off as a single **Data Available** event to your script.
+===Wait for more data until:===
+The port is opened and the plugin will wait until some data is available. As soon as at least a single byte is available it will begin buffering any other data received until the number of milliseconds you specified has run out and then that buffer will be sent to your **Data Available** event.
+===Split data on terminator===
+This is arguably the most useful of the parsing options. Many simple ascii protocols will terminate each packet with a carriage return or newline character or something similar. At this moment you can enter any simple character string into the field and include this short list of special characters:
+|\n|Carriage Return|
+|\r|New Line|
+|\t|Tab|
+|\0|Null Character|
+So to split on a carriage return line feed pair you would enter “\r\n” into the terminator field.
+If you need other special characters or can imagine other ways that pre-parsing could be improved to be more helpful please let me know and I can add those.
+At any point you can call the [[dictionary:more:setparsingparams]] verb to change the parsing options. Though this is not necessary anymore as it was with the original DIY plugin.
+====Receiving Data:====
+To create the script that will receive the data received and parsed by the plugin use the Edit script button on the Edit Interface window. The initial script will appear with templates for all the available events that the plugin will call into your script.
+There are 2 different formats of the **DataAvailable** event depending on the kind of data that is being sent. For simple ASCII or text protocols you can use the simpler version.
+In AppleScript the handler would look like:
 <code>
- on binaryDataAvailable( theDataAsBytes)
+on dataAvailable( theDataAsString, theDataAsBytes)
+  write log “DIY Interface Received: “ & theDataAsString
+end dataAvailable
 </code>
-any data including nulls or binary 0’s can be used in applescript via this handler. Both handler are available in the template script that is loaded when you first create a new DIY interface along with more discussion in the comments that surround it. You can only have one handler defined in the script at a time. If you switch to the binary one just comment out the original and the new one will be called instead.
+This handler will pass you the data as a simple string, and also as an AppleScript list of the bytes received as numbers. This simplifies many parsing jobs as it is difficult in AppleScript to convert between characters and bytes.
+If you are dealing with binary data then it may not be possible for AppleScript to send you the data as a string in the above example. If that is the case then you’ll see an error in the XTension log such as “unable to make data into specified type” or something similar whenever the plugin is trying to execute the Data Available event. If this happens then you need to use instead the **binaryDataAvailable** which does not attempt to turn the binary data into a text string but will instead only pass you an AppleScript list of bytes as individual numbers.
+<code>
+on binaryDataAvailable( theDataAsBytes)
-Now, to start this all off, we're going to use the XTension verb **"send data"** and the ad hoc "Command Line" feature of XTension, to send a string out the PIPE DIY interface : \\
+  set s to “received “ & length of theDataAsBytes & “ bytes: "
-{{:supported_hardware:hello_pipe.png|}} \\
+  repeat with thisNumber in theDataAsBytes
+    set s to s & (thisNumber as text) & “ "
+  end repeat
+  write log s
+end binaryDataAvailable
+</code>
- which will be passed thru **the PIPE**, to the iMac "Perelandra", which will receive it via its DIY interface.  There, the incoming data will be passed to the **DataAvailable** handler, which, as above, will write the message to the Log : \\
-{{:supported_hardware:perelandra_log.png|}} \\
+====Sending Data====
+Sending data to the port is accomplished with the [[dictionary:xtension:senddata|Send Data]] verb. The documentation on that page is extensive of how to send everything from simple text to individual bytes to specifically formatted multi-byte numerical values of the various C standards.
-The handler will then see that there is the keyword "Hello" in the message, and will send back the "Hello Wallace" message... \\
+If you are sending the data from inside the Interface script that also contains your Data Available event then you can leave off the **interface** parameter of the Send Data verb. You can send data to the port from any other script or any other unit but in that case you must specify the interface parameter.
-Which will be passed to the **DataAvailable** handler on "Wallace", which will write it to the Log : \\
-{{:supported_hardware:hello_wallace.png|}} \\
-Oh yes, don't be confused by that first line in the above snippet...that is just what XTension displays in response to the **send data** command ... it's a text representation of the HexaDecimal value sent to the DIY interface. If you don't understand 'hex', or the difference between binary, hexadecimal, and 'displayed binary/hex', then you are probably in for some interesting times trying to debug your new DIY interface :)\\
-Now I've probably made this example too complex by showing two copies of XTension, but this is something that you can set up yourself, without **the PIPE** in between. You just connect the two [[:tech_notes:usb_serial_adaptor|USB-serial adaptors]] together using a simple **null modem** (you do know what this if you're even thinking of doing a DIY interface ?)... \\
+====Working with Binary Data====
+Working with binary data in AppleScript is difficult. XTension provides many helper functions that can make it easier or just possible to do so. See the
+[[dictionary:bitwise|]] portion of the AppleScript dictionary for more information.
-The [[dictionary:xtension:senddata|Send Data]] verb has grown significantly since the first version as well. You can now send strings as you always could, but you can also send a list of individual bytes or numbers in specific C like syntax if you need to do so. See the wiki entry for the [[dictionary:xtension:senddata|Send Data]] verb for more info on using those features.
-Update:  Notice that in the above example I am working with an interface that is TEXT only.  IF you have an interface that uses ASCII HEX codes, you do not want to put spaces between the HEXITS. Also, for Binary protocols, note that James has implemented the entire AppleScript class for Hex, Octal and Binary codes.  See the XTension Dictionary using Script Editor.\\
 ====Sending commands from units====
@@ Line 108: / Line 133: @@
 The value of the unit in the database is not updated until after these handlers return.
+====Additional Scripting Commands====
+In addition to the [[dictionary:xtension:senddata|Send Data]] and [[dictionary:more:setparsingparams|Set Parsing Parameters verbs]] verbs the new plugin supports some additional scripting commands that are used differently inside a 'tell xInterface “name of interface” to doSomething()”
+===Send Break===
+For some serial devices it may be necessary to send a “break” A break on the serial port was used by some devices as a synchronization method or a way to reset a conversation. It is still part of the DMX protocol. A break is really just a longer than appropriate holding of the data line at the mark voltage, the interface interprets this as a breaking of the standard data flow and it can be read by the controller and passed on to the software. You use the **SendBreak** command along with a floating point number of the number or fraction of seconds you wish to hold the port in this break state. The plugin will pause while the port is in the break state and not attempt to send any more data until it has expired. You do not have to add delay or pause statements to account for it.
+To send a 500ms break you would use something like:
+<code>
+tell xInterface “name of the DIY interface” to sendBreak( 0.5)
+</code>
+Or if inside the Interface’s script you could use the (thisInterface) value like:
+<code>
+tell xInterface (thisInterface) to sendBreak( 0.5)
+</code>
+===SetRTS===
+Pass a boolean to the command to set or clear the RTS pin on the fly. The initial state of this should be set in the Edit Interface dialog but you can change change it after the fact using this command. If the device requires handshaking and you have that turned on and are not trying to manage it yourself you may break that by calling this, or depending on the serial interface it may refuse to actually make any changes at all. This is probably most useful when handshaking is set to none or xonxoff.
+<code>
+tell xInterface “name of interface” to setRTS( true)
+tell xInterface “name of interface” to setRTS( false)
+</code>
+===SetDTR===
+Identical to the above SetRTS command and with the same warnings about hardware flow control except that it controls the state of the DTR line.
+<code>
+tell xInterface “name of interface” to setDTR( true)
+tell xInterface “name of interface” to setDTR( false)
+</code>
+====Converting from the Original DIY Plugin====
+In general the new plugin should work and handle your data identically to the old version. Please test this and let me know if there are any problems. Of all the plugins this may be the most difficult to test since there are so many wildly different use cases.
+The conversion is not destructive and if you have problems you can immediately go back to the original plugin.
+  * Disable the interface in the Interfaces list window
+  * Edit the original interface.
+  * Change the selection in the popup from “DIY Scripted Interface” to “DIY Interface” which will also show a version of at least 2.0b1 in the popup.
+  * Add the proper parsing settings to the Edit Interface dialog as these can now be set at startup without needing the call to that in the init method. You can comment that call out with the new interface if you set the settings in the edit dialog. The verb is still supported however and will work as before.
+If you need to revert simply change the Device popup back to the original plugin and re-enable it.
+====History:====
+  * The [[supported_hardware:diylegacy|original DIY plugin documentation]] is preserved and contains a much more detailed use case and demo code than on this page. Please have a look there for more information.
+  * The initial DIY plugin was part of XTension since the early 2000s.
+  * The new API2.0 DIY plugin replaced the original in XTension version 9.4.31 in June of 2020.