[OBSOLETE] MQTT Bridge [device + app]

Looks like that fork @ncherry made is pretty comprehensive (sorry, didn’t realize that was you!).
@stjohn you might wanna take a peek at that one

Wow, @ncherry please give a PR back with all of those device changes, that’s crazy comprehensive.

By the way, I made it work easier without Docker. You can even install it from NPM:

$ npm install -g smartthings-mqtt-bridge
$ smartthings-mqtt-bridge
info: Starting SmartThings MQTT Bridge - v1.1.3
info: Loading configuration
info: Loading previous state
info: Perfoming configuration migration
info: Saving current state
info: Connecting to MQTT at mqtt://localhost
info: Configuring autosave
info: Configuring API
info: Listening at http://localhost:8080

And it now has the configurable port like you were looking for!

3 Likes

Hehe, yes went a bit OCD on the code last night. I’m still a bit lost on why 2 out of 3 of my virtual devices are not working but I’ll look at that later.

I’ll make the PR (pull request, I’m learning :wink: ) in a day or two. I’ve added a bunch of log code that needs to come out and I’ve added even more devices (the stuff in the comments. I don’t have all these things so I won’t be able to test them.

Forgive my ignorance, but I’m pretty new at this. I’m trying to apply this to a ST/8266 project I’m working on, and want to understand all the steps.

I’ve created (In the ST IDE) and privately published StJohn’s MQTT bridge Smart App and Device Handler. I’ll customize and load the .ino file into my ESP8266. I think what’s missing is a ST new device handler for the 8266 switch. I assume that new device would be selected in the MQTT Smartapp.

Are you willing to share your device handler groovy code?

With that, I assume I can discover the device and specify it in the MQTT Smartapp under settings “Switches” and “Notify this Bridge.”

Any other pieces I’m missing that are required to make things work?

Also, please confirm my understanding that I don’t need a separate MQTT server running, and that it’s all on the 8266.

Thanks for your help!

I’m just using the “On/Off Button Tile” device type. It can be anything with a switch attribute because the MQTT bridge will send on/off commands from the ESP to the device specified in the code, in this case it was hard coded as “Virtual Test Switch”. So create a new switch device, give it a name, then put that name in the ESP8266 code.

You do still need an MQTT broker for this to work - the ESP8266 is a client, and this bridge is also a client. Mosquitto is a popular open-source option that works on several platforms.

@looie53, you can also play with the simulated devices. I’m trying to get some virtual device handler code working but for the life of me I can’t get that working. At the moment I have a Fubarino talking to MQTT. I had this working with one simulated device (temperature).

Just to clear things up, simulated is a built int device handler, the virtual is device handler code I’ve downloaded. I think half my problem is that I haven’t fully read the docs and I’m trying to do more than I know. Eventually I’ll share all my code. I’m pretty sure someone else may have better examples.

Thanks everyone. I got Mosquitto running on a spare Raspberry Pi last night, and plan to load Home Assistant on it today. This should be a nice solution – better than leaving one of my desktops awake 24/7.

Big thanks from me too !

For those nervous re Linux I am using the ThingBox Raspberry Pi image v( v1.8.6) based on Rasbian Wheezy and that already includes the MQTT Mosquitto server, Node.js, Node-RED. Really easy - no Docker - all I did was NPM install the web bridge service as shown in post #46 and the ST app and device handler and away you go. There were a few dependency warnings for older versions but it still seems to be working fine.

http://thethingbox.io/#home

The really useful aspect for me here is converging my existing HA system with SmartThings. I can include my existing HA devices in SmartThings and also expose ST devices back to my existing HA system with near instant updates and bi-directional control. This of course assumes you can connect your external devices to MQTT which fortunately is really easy for me and most HA software now seems to offer an MQTT plugin.

This opens ST up to almost any device and works really well for supporting locally connected IP devices like my C-Bus lighting system plus heating, telephony and security. MQTT is the middleware connecting everything allowing you to get around the local processing and IP comms side (even UDP in my case with xAP). Of course it’s still cloud dependent but loads of things are now possible. There’s Node-RED on the Pi too - to interact with your ST things.

Thanks once again…

K

PS One thing I have to think through is how to map an existing device topic on MQTT to the smartthings/thing name/attribute topic hierarchy or vica versa such that the linkage works and can be maintained at either end … e.g. Creating a virtual device in ST that watches a specific arbitrary topic on MQTT representing the real device. What would be useful it appears is a sort of topic ‘alias’ feature where the MQTT server treated the two topics as if they were one and the same. I’m thinking this probably is a frequent need and so some standard solution might exist.

1 Like

In my home I’ve been using an event-management platform called EventGhost for well over a decade, going back to the X10 days. While it has it’s problems, it’s a fast, reliable, and simple-to-use platform for automation and I still rely on it for a lot of my home use cases. A while back I wrote a SmartApp to publish ST events to my EventGhost server which has been working reliably.

Since getting this MQTT bridge setup I’ve gone and tied EventGhost to my new MQTT broker using the EventGhost MQTT client. There I ran into the same problem as I did here - the plugin wouldn’t work without a leading slash (seems to be a common misunderstanding!) The author has modified the plugin to support standard topic names and now I am able to pass events back and forth between EG and ST (and any other MQTT-compatible platform).

Having a universal event message bus at home is turning into a huge benefit for me. Instead of hacking together REST commands between n-different platforms, I can now throw messages onto a single universal bus and react to them from any device or platform. I’m just beginning to scratch the surface of what is possible and the potential use cases keep piling up. Thanks to everyone that has contributed to this bridge!

1 Like

@xAPPO, yes the Pi makes a great computer for this. Plenty of HP for HA. I’ve setup everything without docker. I simply ran the node server like this:

  • npm install
  • vi config.json # I needed the server to run a port other than 8080
  • node server.js

I already have a dozen or so topics for the rest of my HA projects. One thing on the topics:

smartthings/label/attributes

The label comes from the label in the create devices.

1 Like

Hi Neil,

Are you integrating any existing MQTT linked devices from your other HA projects in SmartThings, say via a virtual device ? If so how do you get it to be linked to ST as those devices will not have the smarthings/label/attribute topic hierarchy ? I’m still thinking about how best to structure my topics as I have a lot of other projects.

At the moment I have kludged a scripted MQTT client in node-RED that simply subscribes to say topic a/b/c/d and publishes it as smarthings/xxx/yyy and the reverse so it mirrors the topics but that seems rather punitive on the MQTT broker. I may even use a pair of Pi MQTT brokers for this if necessary.

The MQTT bridge at the moment is more suited to exposing internal ST devices to others rather than the reverse.

P.S. Hmm - I wonder if the ST label might allow for an ‘a/b/c’ style of entry - effectively creating more levels in the topic hierarchy for the ST devices - and then I could effectively just strip or ignore (*) the top smarthings level. I’ll investigate… (it will likely break the existing code mapping back from MQTT to ST though)

1 Like

@xAPPO, yes I’m currently using a Fubarino with a DHT22 on it that reports to MQTT. I had to use Node-Red to go from the topic

home/network/devices/fubar/txt

to the topics:

smartthings/CR Temperature/temperature
smartthings/CR Humidity/humidity

Though I don’t actual have the devicehandler part working yet. I see the information going though the smartapp but I never see anything in the logs about the device handler getting them. I’m currently reading the documentation to get an understanding of how smartthings works.

On the subject of topics, the smarttthings topics are in the format of

smartthings/ label / attribute

I don’t think that format will change to anything else. Of course I haven’t given it much thought.

Understand I currently have Misterhouse, Node-Red, scripts (Perl, Python & Bash) and some other things banging away at MQTT. Other than MQTT locking up from time to time (grrr, need to see if the latest fixes that) it works perfect. Though I’d better start documenting this mess. :grin:

This… changes… everything!
I love you guys!

I feel like this approach (or something like it) is the best solution for translating between different topic/message formats. While it might be possible to shoehorn in some setting into the MQTT bridge here, there is very little chance that you’ll arrive at a general-purpose approach that will fit all use cases. For example, Domoticz sends all events to a single topic as JSON objects which makes a straight translation difficult without resorting to something like Node-RED anyway.

I feel like it’s best to use the right tool for the right job, and tools like Node-RED are custom-made for handling this kind of translation.

We could probably move the server.js into a node-red flow (not a bad idea really). So that those of us already running node-red can just add it there rather than start a new server. It seems that the server.js is already written like a node-red flow.

1 Like

Yes … I think Node-RED is the most flexible approach. The JSON payload is another aspect that I have used Node-RED to get around before and so as it’s available and capable it’ll do nicely.

Interestingly the MQTT plugin for another HA app (HomeSeer) creates it’s own fixed topic hierarchy for devices it ‘owns’ but allows you to predefine any topic construct for virtual devices, as it assumes their topic hierarchy will be already defined. Such an approach would be difficult to implement on ST probably and as mentioned above virtual devices with several attributes get handled by multiple topics so there’s always going to be awkward cases as well as payloads with multiple values that may or may not be JSON objects.

Dang, gotta throw the MQTT onto HomeSeer too … never enough time in the day. :wink: Oh woah is me, that’s HS2, I have HS3 and Linux.

We might get cute with ST but I’m leaning towards not doing that. ST is cloud based and there are limitations we’ll need to be careful of (like the per minute transaction limit). But adding a node-red module is still a good idea.

Yes, my 3rd edit, I found the HS3 version of the MQTT plugin. I’ll see what I get later this weekend.

I now have virtual dimmer working, got bit by a missing toInteger().

I have a portion of the node-red code working. Still trying to figure out how to respond with the proper status okay. Then I need to figure out what to do with the /subscription information as I’ve decided to cheat and subscribe node-red to ‘smartthings/#’

I’ll be getting a new set of fire alarms. Which I really needed, very cool. I’m going to pickup from of the First Alert ZSmoke and ZCombo alarms. This will make my wife very happy as she can now get alerts when the alarms go off (and I’ll be able to monitor the dang batteries).

Good stuff Neil - I’m about to investigate virtual dimmers too for my C-Bus lighting. Any tips re that toInteger() ?

Keep me posted on the Node-RED implementation as having the code running there would sit well with my implementation. I altered the original js code a little to avoid posting duplicate updates from ST> MQTT f (there’s already a check on the MQTT>ST side) - to further help with potential race conditions. I haven’t checked the validity of this yet though. It is quite easy to get such race conditions if the device is externally controlled and monitoring MQTT on the same topic. I’ve kinda kludged a way around that - to know if ST is just reporting a status change or requesting one i.e. a command from a UI interaction button press.

One heads up for anyone using a plug module incorporating current sensing for the reported on/off status. When the switched device goes into it’s low power state or turns off, for example once a kettle has boiled then ST reports it’s state as off. This then gets updated through MQTT and the effect is that the device gets completely turned off !

I have a few Wemo Insight sockets - powering an electric heater, an electric blanket and a kettle. All these report as off when not heating - and they switch states many times during their heat cycles based on their thermostats. Well they all just turned themselves off after their first heat cycle once I enabled MQTT - so I have removed them from being MQTT linked.

@xAPPO, the vDimmer.groovy is here:


Nothing too fancy, just added the tVar and the var.toInteger(). I was really tired so it is not good code. I’ll look at it tomorrow or Wednesday with fresh eyes. It really was coding with no thought (I hate that). The on/off portion is untouched and the extra log statements are not needed. I was getting a vague java.lang.ClassCastException message in the logs that I couldn’t pin down to a line. When I did a search I found another ST user had found the same error and mentioned adding the toInteger(). Apparently Groovy doesn’t like doing math on ints and strings. :wink:

ST. John gets around the resending of the mqtt topic by keeping a copy of the latest topic value and if it doesn’t change, he doesn’t send it on to the hub. I haven’t touched that yet in my node-red code but I am aware of it. I may also add an option to avoid sending the mqtt topic until after the first change at start up. I really need to think about that.This actually might be one of those things best handled with the retention flag and QoS. I’m not aware of any other race conditions yet but once I get MisterHouse/HS3/SmartThings really running I expect I may see some. This is going to be a really good presention next month at TCF in NJ. :slightly_smiling:

Ah current sense! We discovered that in the early X10 days (I’m a dusty old fart, I’ve been doing HA since 1978). The solution with X10 was a 7W light bulb in parallel with the device that goes on and off. Or another was to clip a resistor in the module so it could no longer measure the current. I’ll need to get my hands on one of those devices and see how many watts are needed. I prefer to use something meant for measuring power and an on/off circuit for appliance checking (2 separate things I toss into an Arduino type of board). Also watch out for any switching power supplies (SMPS) devices on current checking modules. With some SMPS the current sent builds up enough power to turn on the device for a moment. Then the immediate rush of power (lacking actually) discharges the cap causing the device to turn off and this will happen in an infinite loop.

I’m very leery of anything with a heating element. There were a lot of problems with electric blankets and X10’s. A few members of comp.home.automation (a newsgroup on Usenet) reported fires. I can’t recall if it was the X10 module, the blanket or both. You have to be careful of the wattage ratings of the module. A module rated for light bulbs up to 600W (example) can’t handle 600W of an inductive load (a motor or heating element). Take those rating with a grain of salt and mark then down to 80% just to be safe. Remember it’s not really paranoia if they are really out to get you! :wink:

Always limit the use of dimmers to just light bulbs and only bulbs those that say you can dim them (LEDs & CFL especially). Worst case, you can have a fire if you put a TV on a dimmer. Best case, you’ll shorten the life of the TV. Finally never put a dimmer switch on an outlet (I think that’s against code also, at least in the US).