Current state of MQTT on Smartthings- Hoping to be taught a bit on the subject (How to)

Definitely, MQTT has many uses outside of zigbee/zwave

Here’s a non-zigbee/zwave mqtt example I use. I can start/lock/unlock my car with MQTT using this onstar2mqtt program

There’s a ton more. MQTT can be kind of like a “Rosetta stone” to bring many different kinds of devices together.

1 Like

After just an initial look at zigbee2mqtt, I think the biggest inhibitor to implementing something like that for SmartThings is that in SmartThings Edge, there’s no concept of dynamically creating devices. All devices have to have a pre-defined profile, which defines its capabilities. I don’t know HA, but it looks like this feature can dynamically create devices based on the capabilities defined for the zigbee device.

In a SmartThings Edge driver, you’d have to define a canned set of device types with certain capabilities and try to best-fit match them to the discovered zigbee device. You might be able to support some simpler zigbee devices, but definitely not the expansive list that zigbee2mqtt supports. At least not without a lot of pre-building the associated SmartThings device profiles.

One might have more flexibility using a SmartApp, but then you’re dependent on a cloud-based solution.

That’s just my initial take, anyway…

Cool example! Yes, the sky is the limit. Any SmartThings Edge driver can be developed to use MQTT so all of this is possible. Some solutions will require custom drivers to be developed. Others might be able to take advantage of the generic MQTT Discovery driver, but I suspect those will be more customized on the device end of things.

3 Likes

I’m not sure if you’ve already reviewed the ST MQTT bridge project…it’s super old and hasn’t been touched in years so idk how relevant it would be but there may be some helpful information to be gleaned from it. I still use it to this day, and have been dreading the demise of Groovy because there hasn’t been anything that I’ve seen that could be a replacement for it. There have been a few attempts to get the MQTT bridge modernized but those were still Groovy based, so this is the first time I actually have some hope that I’ll still be able to use MQTT with the next phase of ST.

That project had something like 50+ device types that you could select from to send/receive status changes to. It was a little clunky, but what I did was choose which (real or in some cases virtual) devices in ST to expose to the MQTT bridge so I could get the data out of ST. A perfect example of this was when I was able to expose the status of all of the ST/ADT motion, door/window, flood, smoke, etc sensors that everyone said couldn’t be used outside of ST. Well, the MQTT bridge made that a piece of cake. Until Samsung pulled the rug out from the ADT users, but I digress…

Once the devices are publishing to MQTT, the sky is the limit. Plus, it’s two way communication so I could then use MQTT to send a command to toggle a switch or set a dimmer from node-red or home assistant in complex automations instead of using webcore. I also used it as a way to get some unsupported devices into ST…just create a virtual device in ST, expose it to MQTT and then allow the real (unsupported) device to feed data to that virtual device via MQTT.

There is a native ST integration for HA that would make doing this via MQTT somewhat redundant but I would much prefer to keep the device communication at the edge instead of having the HA cloud talk to the ST cloud every time I want to toggle a device. Plus that integration is part of their paid service which I support but realize not everyone is signed up for.

As you mentioned earlier, if you follow the HA native MQTT discovery structure, those exposed devices would then automagically become available in home assistant and that would be totally awesome. Honestly that seems like a cherry on top pipe dream at this point so needless to say I’m super excited to give your work a spin. I could have sworn Samsung said they were going to natively support MQTT at the dev conference a few years back, but kudos to you for moving that ball forward. It is very much appreciated. Now I just need to figure out how to get my Aeotec hub into edge mode!

1 Like

They did, but like the “local processing” they promised for the V2 hub, what they delivered turned out to be very different than the expectations raised by the early discussions.

Last year a community member was hoping to use mqtt to create an integration with a green energy device and I was the one who posted the link to the dev conference presentation. Which turned out to be useless for the purpose. :disappointed_relieved:

1 Like

@TAustin, great work with mqtt. Thank you for spending the time and coming up with solutions.

If you plan to share your MQTT solution, I would love to build on it for specific device running on Zigbee2MQTT. The device discovery is nice to have but not critical. As an example, I run mix of Zigbee platform (ST, Z2M and Hubitat, each of them are Zigbee capable). It would be nice to be able to bring a some of the device that just happen to be in Z2M into ST. I personally would not mind manually configure broker ip address, topic and etc for each device that I am interested on. There are cases where I just want to bring one or two devices from Z2M. I don’t think in this case device discovery is important.

Just to give a context why I am running some devices in Z2M. Z2M has some backup capability where one can replace a broken Zigbee stick with a new one without re-pairing all the devices . I exploits this capability to set up multiple identical Z2M servers at different location where a device can join to a Z2M server that is in range without needing to be re-pair at each location. This device that I am talking about is my Arrival Sensor. When I have Zigbee mesh at Home and Office, the Arrival Sensor can be detected when they arrived at home or office. Here is a configuration that I did on my Hubitat. I hope it explain it better. It is not my intention to open a discussion hubitat vs ST. But, I have already some information there. I just do not want to duplicate it here. With your MQTT solution, I can probably do this in ST as well.

1 Like

Thanks for your post! A couple thoughts I had:

  • There is an important point here about supporting TWO directions: one which is to enable a device or application to send attribute updates TO a MQTT-purposed SmartThings device. (The MQTT Discovery driver allows that). With the second being able to allow an existing SmartThings device to publish its states out to an MQTT broker.
  • I think the latter scenario above is always going to require a SmartApp, which is a bummer since in the new world, that means you have to host it either on AWS or your own internet server. I don’t think most SmartThings users are going to want to do that. I’ll have to think more about this. There IS an alternative, and that is to run a SmartApp locally on a Raspberry Pi, but again, I don’t think most people are going to be willing to get into all that.
  • Edge allows us to forgo a ‘bridge’ and directly subscribe to or publish topics/messages. So that hopefully will be a simplification.
1 Like

Hi - I’d encourage you to try out the MQTT Discovery driver I posted earlier. The ‘discovery’ part of it just allows an easy way to get a device created in SmartThings to represent your actual (this case, zigbee) device. After that, you just need to configure your device to publish to a certain topic. My driver already supports a presence type device, so you might be able to get that working fairly quickly.

I’m still in the stage of understand requirements, so any other feedback or ideas you have is welcome!

1 Like

@TAustin, I am happy to help.

I apologize if I may sound confused. I am more excited to see the capability to subscribing to a topic in mqtt. My arrival sensor is paired in z2m. The data is in z2m. It would be nice to be able to synchronize the sensor attribute from z2m to ST. Z2M is purely ZigBee manager. We could use ST as the automation and gui for the sensor. Combining the best of both world.

For those of us running home assistant and node red and all of the other smart home docker containers, one more smart app wouldn’t be any issue at all. I’ll hopefully have some time tonight to test things out and will report back with any feedback.

Again, I cannot thank you enough for doing what you’re doing!!!

2 Likes

Let me see if I understand correctly:

In your case, you don’t already have a SmartThings device, but you’d like to have one that gets its state updates from your arrival sensor via MQTT. Correct? You already have that arrival sensor set up to send message to MQTT. Are you able to configure additional topics for that device to send status to? This is the part I’m still pretty fuzzy on, and that is what flexibility you have to configure an MQTT-enabled device. Does my Edge driver/device have to recognize whatever topic structure your device sends out? Or if I defined a specific topic structure I needed, would your device be able to also send out messages to it?

Not @iharyadi but I’ve been following his amazing arrival sensor on Reddit for a while (and want a couple of them for myself tbh) and I think what he’s saying is that the arrival sensor publishes to /z2m/sensor/arrival/presence currently (for example) and it would be very helpful to have a device in ST that is able to subscribe to /z2m/sensor/arrival/presence so that the virtual ST device would be sync’d with what the physical sensor is doing.

Worst case, if the structure has to be /smartthings/sensor/arrival/whatever we could always just use something like node red or home assistant to set up an automation that republishes whatever comes in to /z2m/sensor/arrival/presence out to /smartthings/sensor/arrival/whatever.

2 Likes

That’s helpful thanks. I guess what I need is a giant mapping of these devices, what their “capabilities” are (referring to the use of that term in SmartThings), and the topic structures and message payloads that might be sent. There doesn’t seem to necessarily be a standard, so we just need to settle on one and go from there.

@fooktheta, thank you for the clarification. Your understanding is what I am excited to see.

@TAustin, I personally not looking for something fancy. If there is specific edge driver that subscribe to /z2m/sensor/arrivalxxx/xxxx that specific for arrival sensor, it is already a tremendous start. Someone else with a different device in z2m may need a different driver and so on. At least for starting point, It would make a lot of people happy.

I understand that in the long run having something that is more generic is really cool.

Try the very first driver I had posted, which was just meant to be a proof of concept. But it does allow you to subscribe to any topic and simply spits out whatever messages it receives in a text field on the device Controls screen.

All we have to do is turn that into a specific device type - like a presence sensor - or turn this into another virtual device creator that adds the ability to subscribe to an MQTT topic for each supported device type.

1 Like

This would be great for what I am doing. When you are ready to test this, I can help testing it.

1 Like

@TAustin
I spent some time getting everything set up tonight with the MQTT test driver and MQTT discovery mechanism. And I must say, for it being so early in the process, I am very, very excited. I think we will be able to accomplish the goals of a lot of the folks in this thread and really open up some great new potential for the ST platform.

Here are a few observations that immediately jumped out at home and could be helpful for others that are testing these things out:

  • As noted very clearly, we are at an early stage and authentication isn’t implemented. I had to spin up a secondary mosquitto container and turn off authentication to be able to test this stuff out. As always it’s good to read and follow the directions! In the future, having configuration options for username, password, client id, port#, retain, etc will be important.
  • I first started with the MQTT test driver 1 and am very happy to report that I was able to get the communication going subscribing and publishing MQTT messages. One thing I noticed (I’m on iOS) was the app was doing funny things with automatically capitalizing the first letter of the topics even after I intentionally made it lower case. Nonetheless the proof of concept worked! You love to see it!
  • Then I tested out the MQTT handler - once again I was easily able to create a test switch device and toggle it on and off via MQTT. One thing that I noticed did not happen was if I went in and toggled the switch via the app, the switch did not publish the state change to MQTT. Is there a way to bring the functionality of sending the MQTT message that the test driver has to state changes here? That would be really helpful. Even now, for something like what @iharyadi is trying to do (a physical, outside device providing status updates to a virtual device in ST) it seems like the MQTT handler will easily do the trick.
  • Home Assistant only allows 1 MQTT broker integration at a time so I’ll need to spin up another VM to test the auto discovery but that’s not really a big concern seeing as I know I can manipulate the device via MQTT. Getting it to work in HA will be trivial at this point. Still very awesome though!
  • I also added the vEdge Creator, then used that to make a virtual light and was thinking that it would be amazing to incorporate MQTT status reporting and control to the ease and simplicity of creating the devices in app. I know this won’t necessarily happen right away but that would be a nice feature to put on the roadmap.

I’m going to keep testing these out, but those were my first impressions. Again, thank you!

2 Likes

Terrific feedback. Thank you!!

Yes, unfortunately this is a known quirk of SmartThings and I have no control over it from the driver. The first character is capitalized in any Settings item, but it’s only for display. If you edit the field, you’ll see it is not really capitalized.

Yes, this can be added. I wasn’t sure what would be the norm for the topic you’d send state changes BACK to. Would it be the same subscribed topic, or might it be a different one to get it back to the device? As you probably saw, I included in the ‘test driver 1’ an option to echo back the received message to a different topic, for example

Yes, I’ve been toying with the idea of basically taking that driver and just adding an MQTT messaging option to all the devices. However I probably will keep it separate because a lot of those devices were configured for specific use cases (Alexa integration, auto-reversion, auto-synching, and other things), that wouldn’t apply to MQTT devices. What I need is to start building a priority list of device types (or device configurations; in other words which SmartThings capabilities would they contain) that would be desired for MQTT integration. As I had mentioned in an earlier post, I think SmartThings is a bit different from HA in this regard, as you can’t just dynamically create/configure devices based on discovered physical device capabilities. Rather, all possible device types/configs have to be pre-defined in profiles that are built along with the driver itself. So basically you end up with a list of supported device configs, similar to how the vEdge driver was implemented.

1 Like

I did notice that…got it!

Typically yes, from what I’ve seen the state changes are typically sent back to the same topic. That keeps everything neat and tidy. I did see that separate topic functionality, and while I haven’t seen it implemented before, honestly it’s cool to have as an additional option or that a user could add/configure.

Interesting. I might suggest that there may be use cases where an event being sent to the MQTT broker might be beneficial to a user. That’s part of why I really went all in on MQTT - it’s so versatile. It can be used for controlling things as well as notifications and automations and everything else. As long as you can get the data into the broker, you can then do a whole lot with it.

Couldn’t agree more. Focusing on the most common devices and use cases and prioritizing that will absolutely be the best route. I will be more than happy to help in any way that I can. This is awesome.

1 Like

After tinkering some more with the edge drivers this weekend, I got to wondering if there could be a way that we could expand the reporting and control of the devices to any device that is connected to ST. Is there a possibility that the edge driver could see the activity of devices that it did not create (and then ideally report that activity out via MQTT/control the devices via MQTT)?

For example if I have a Samsung TV that I have connected to my ST, is there a way that the edge driver would be able to see the capabilities of the TV and then expose what they’re doing out via MQTT? Obviously I wouldn’t want to create a virtual TV just to be able to send commands back and forth, but I think it could be very powerful to be able to just see what the actual TV is already doing.

Separating the reporting/control from the device creation might actually simplify the process because then we wouldn’t have to worry so much about creating all of the device types…the user could just pick from a list of the ones that already exist and/or just your other driver to make virtual ones if needed.