Developing for Edge - What I've Learned So Far

Hi - I’ve been writing some LAN-based edge drivers for the new Edge platform (started pre-beta) and wanted to share some personal experiences in hopes it may help others who are just getting started. This is targeted to developers, as opposed to users.

This is not meant to replace all the great documentation and posts from the SmartThings team, but rather just to give more of a community member perspective. I apologize up front if any my thoughts or recommendations are erroneous. This is a rapidly evolving platform, so I’ll make updates as needed to keep this useful and accurate. And I hope that others can contribute to this topic as well.

I don’t profess to be an expert by any means, but am happy to answer any questions that I can.


Development environment

Mine has been super-simple & cheap, but thoroughly sufficient for LAN-based device development: a Raspberry Pi 4b with 4GB of RAM running Raspberry Pi OS (Debian Linux)! I’d definitely recommend Linux over Windows.

I don’t use an IDE for Lua, so can’t recommend anything in particular there. Unfortunately there is no debugger available for Lua so plan on liberal use of log and print messages. (See comments regarding Logging below)

Lua

It’s a simple enough language to learn. Key concepts to fully pay attention to because you’ll use them a lot: require, tables (big topic in Lua), nil, forward referencing. Also understand the coroutine concept because it’s central to how your Lua code will be executed, however much of this will be managed by Edge (see below).

Install version 5.3. There are more recent versions available, but there have been problems with those during alpha.

Unfortunately, the Lua website charges money for the latest programming guide, but you can read the free one available on their website from the original release. And the language reference is also available online for free. (I’ve suggested to the SmartThings team that they should be able to get us a discount code with all the developers they are driving to Lua now!)

Also install Luarocks, which lets you get additionally available Lua libraries you may want to use, e.g. sockets, dkjson, xml2lua, etc. Stick with the libraries you know are supported on the hub per the documentation. Keep in mind that all Lua libraries your driver will use are already loaded and available on the hub. Installing them on your development machine is only needed for off-hub testing.

LUA_PATH

This will need to be defined to ensure you are picking up the right libraries as you test on your development machine. The syntax and way it’s used to search is not typical Linux, so be careful here.

The Edge platform

The key concept you’ll have to wrap your head around is that this is NOT a multi-threaded processing environment as far as your Driver is concerned. (Lua was designed as an embedded scripting language most typically called by C programs.) The Lua coroutine concept is one where code is synchronously executed in chunks with a yield and resume mechanism. On the hub, the Edge platform manages that mechanism. The result is that your driver will be primarily composed of lifecycle handlers, channel handlers, and callback routines that are invoked by Edge. It forces you to re-think how you would handle communications tasks with the devices that you might ordinarily implement in multiple threads.

Development and Testing

I’d highly recommend writing as much of your driver code as you can ‘off-hub’ and testing it that way - especially as you are just getting up to speed with the Lua syntax. You’ll have to temporarily restructure your code differently from how it will be in your final on-hub Edge driver, but it will be MUCH easier to rapidly find errors, fix, and retest. Once you start moving your Driver to the hub, error messages can be quite obtuse and if you are using multiple layers of require modules, even pinpointing the line number of the error gets very difficult if not impossible (hopefully this will be improved in the future). See my tip below.

Testing / Errors Tip

EDIT: the following is for when you are testing your driver ON the hub:
As mentioned above, if you are getting some kind of runtime or fatal error as seen in your log messages while you are testing your driver, sometimes the messages aren’t very helpful - they can be difficult to decipher, or the exact line number within the offending module is not shown. In those cases I will temporarily comment out the Edge-specific require statements in the module where you think the error is coming from, and then in a Lua interactive session on your development machine, require the module in, and then you can get a more helpful and pinpointed error message. I’ve found this tip can save an enormous amount of time hunting for simple Lua syntax errors.

Logging

Once you are running your code on the hub, this will be the only way to see your driver’s activity. So you will make liberal use of log messages within your Driver to track what’s going on. And because of the various handlers and callbacks that are being invoked in a non-sequential manner, you’ll find the need to include sufficient information in each of your log messages that identifies it to a specific function and device. Otherwise, you won’t be able to logically link your log messages together for a particular device or routine because they will appear as almost a random mishmash of messages!

Logging output can get rather noisey because there is currently no way to suppress log messages coming from Edge itself. There is no one message type used, so filtering won’t help.

There is an irritating problem with running the cli logger on Windows (which is one of the reasons I prefer Linux). Scrolling back through prior messages while the logger is still running is a problem because it keeps wanting to snap back to the latest one as soon as you let go of the mouse button.

Loading Drivers to the Hub

EDIT: I’ve removed my comments in this section; please see Patrick’s posts on this topic below.
The more I learn, the more I realize I have much learn :grin:

LAN-related

All the work I’ve done is with LAN devices, so there are a couple things to note on this topic:

Your driver cannot currently request a specific IP port number on the hub. Ports are randomly assigned. Hopefully this limitation will be addressed in the future, but unfortunately for now, this will preclude drivers for devices like Shelly’s wifi motion sensor that allow you to configure to send motion updates to a specific IP and port.

When initially coding and testing before moving to the hub, you can use the Lua socket library for all your LAN communications needs. This gets replaced by the cosock library when running on the hub, with no change needed to your code except for the require statements.

Device discovery is an important function of any LAN-based driver. Examples are forthcoming (see my UPnP comment below). Once you discover the device, you can implement any protocol you need (TCP, HTTP / RESTful API, SOAP, etc) to interact with your device.

You’ll need to figure out how you are going to address maintaining the online/offline status of your device, either through polling or through periodic device ‘I’m alive’ messages. How often this is needed will obviously be based on various factors.

Finally, a shameless plug - I have a full-function UPnP control point Edge driver I will soon be making available to the community with the intention of making it easy to create drivers using the complete UPnP architecture standard. (This also addresses the device online/offline monitoring)

17 Likes

This must be 4 gb of ram

2 Likes

Fixed! Thank you.

2 Likes

<3 I really owe you a beer (or beverage of choice) if we ever cross paths in real life now.

This is all super great. Thanks!

AFAIK we haven’t fixed this yet, it’s wasn’t deemed a blocker for beta release (as was any bug that had a known workaround).

I’ll also add to this, I suggest using luacheck, which can be installed from luarocks. It’s a code linter. I find it a big over aggressive in the warnings it puts out, but even if you only pay attention to the errors it’s a much quicker way to find errors without even having to run the code. (Great for mesh device code that you can’t run on a PC and don’t have unit tests for.)

Sadly no one else will have access to this. That hacked together web UI was a stopgap until we had the CLI. It has some pretty big problems at the moment.

We think these should be fixed now. If anyone sees this again, please leave both drivers as is if possible and report it to us. Having to delete every device between updates would definitely be painful and you shouldn’t have to do that.

These haven’t been published publicly quite yet. These and other examples are going to be trickling out now as we have time to polish them up to make them less confusing (and especially in the case of the wemo driver less wrong/out of date). The Lan drivers are also a bit of a special case, since there’s some platform issues that will prevent us from fully testing all the error paths in them they can’t be “officially” released and we haven’t decided if there some other way we should publish them even if we can’t fully test them yet.

1 Like

I couldn’t agree more with Patrick, this is such a cool and useful Tips and Tricks-like thread, and I believe that it will help a lot of developers that are getting started with SmartThings Edge.

Thanks for the contribution and enthusiasm, @TAustin :raised_hands:t3: !!!

2 Likes

I made the necessary changes; thanks for your comments!

One question for clarity: when you say:

Do you literally mean that that’s not required anymore, or you agree that it needs to be addressed?

Thanks Eric :+1:t3:

It should never have been required. To be clear, deleting devices before deleting the driver is definitely required. But having to delete the driver before updating it (outside of removing profiles and other breaking changes) should not be required.

1 Like

Then I think you meant to say “Having to uninstall a driver between updates is something you don’t have to do” Got that.

But having to delete all devices for a driver between driver updates is a minor hassle. I can certainly understand why it’s required, but I’d still vote for a CLI command to be added to do that, just for convenience so you didn’t have to do it through the mobile app.

I did mean delete, I thought that’s what you mean, but the same is true for uninstall. You shouldn’t have to.

You shouldn’t have to do this either. I was assuming you were doing that as a prerequisite for deleting (or uninstalling) the driver.

When you’re doing development you should be able to just package assign and install and the new version of the driver should get pushed to the hub fairly instantly and after about 5 seconds the older driver will be killed and the new driver will start.

I just run this all as one big command when I’m doing development:

st edge:drivers:package -p personal . ;and st edge:channels:assign -p personal -C <chan-id> <driver-id> ;and st edge:drivers:install -p personal -H <hub-id> -C <chan-id> <driver-id>

I’m hoping we can simplify that in time, but we’re still discussing the right simplification.

Thanks Patrick. Since I was doing a driver uninstall each time, I would get an error message saying I couldn’t do that because there were devices. If I follow your suggestion, are the devices getting auto-deleted when the new driver is installed, or are they persistent from driver to driver? This is a key point because if I need to start with a clean slate because I’m testing the discovery process, I want the devices deleted. This is where I was coming from.

Ah, they persist when upgrading a driver to a new version. I didn’t realize you meant for deleting devices to retest discovery.

Also for deleting all your devices from a certain driver, here’s a little pipeline that will do that for for you:

smartthings devices --json | jq -r '.[] | select(.lan.driverId | . and contains("c3b0087f-9789-4316-b3c4-fbeac29dff7b")) | .deviceId' | xargs -L 1 smartthings devices:delete 

Just change the UUID to your driver id. And that will only work for lan devices (but that’s all the supports discovery ATM).

jq is a super helpful tool for automating things that can output json. Its selection syntax can be a bit obtuse at times, but I think it’s worth figuring out for situations like this.

5 Likes

NICE! Thank you!

1 Like

Thank you.

1 Like

Also, as of v0.29 of the CLI you can package, assign and install in one step:

st edge:drivers:package -p personal --install

Or, you can specify the channel and hub to avoid the questions:

st edge:drivers:package -p personal --hub <hubId> --channel <channelId>

5 Likes

I have been trying to get Lua 5.3 to run with vscode on macos, but cannot get it to find any of the system modules. How did you setup the paths? Guessing I have something messed up with runtime or plugin paths.

1 Like

I got it working with the vscode-lua plugin. I also set it up to work on my laptop in windows, but keep the code remotely on my Raspberry Pi, where I keep all my Edge development stuff. I think it’s called the remote extension host or something like that.

I installed 5.3 via Homebrew and then LuaRocks as well. I used that to install LuaSocket and pointed it at 5.3 during the install. When I go to run the code via command line, it cannot find the modules related to LuaSocket.

Sure which that worked with the latest 5.4

Your LUA_PATH environment variable probably isn’t set right. Make sure it includes the LuaRocks path. For example:

export LUA_PATH="/usr/local/share/lua/5.3/?.lua;/usr/local/share/lua/5.3/?/init.lua;/usr/local/lib/lua/5.3/?.lua;/usr/local/lib/lua/5.3/?/init.lua;/usr/share/lua/5.3/?.lua;/usr/share/lua/5.3/?/init.lua;./?.lua;./?/init.lua;/home/pi/hubalpha/lua_libs-api_v0/?.lua;/home/pi/hubalpha/lua_libs-api_v0/?/?.lua;/home/pi/.luarocks/share/lua/5.3/?/?.lua;/home/pi/.luarocks/share/lua/5.3/?.lua;/home/pi/.luarocks/share/lua/5.3/?/init.lua"
1 Like