[RELEASE] New MQTT Bridge with Tasmota Sonoff Wifi and RF Integrations and other custom MQTT

@Tech_GFam
I have mosquitto installed… I have a few different tasmota devices with MQTT setup. Using MQTT i can see everything is fine.
I’ve even put in the prefix ‘smartthings’ in the tasmota device console to give me
“smartthings/%prefix%/%topic%”
The config and the devices YML files have been configured…
config.YML set to talk to mosquitto
devices.YML to assume a single switch using the simple switch option with the name the same as the device in smartthings.

The smartapp and the devices have been setup… using the tasmota switch and the MQTT bridge and the name.

BUT there is no talking at this point.
If I could see what is being passed too (see this in the debug log in ST) and what is then recieved in the bridge, this would help. but currently I havent a clue where the problem might be.

Any hints welcome.

I would start by looking at the log in the events%date%.log in the log directory.
That should show you exactly what is happening.

Did you configure the Bridge option - IP, port, MAC Address - that is very important.
In the smart app when you click save - that should resubscribe everything.

If there is formatting error in any of the config files the server will basically not read them - but I would start with the events.log and if you could post here that would be great.

Any exceptions would be in the error%date%.log file in the same directory

and in ST also there should be debug log for the bridge, smartapp and the device which will show what is being passed

so extensive debug and logs both on server and in ST for now

I should have read the code before I asked! but you did help.
So… switched the log level to debug, but didnt get any extra info, despite debug logging in the code. (did put in an winston.info to check the log level in config was being picked up and it was (which I guessed because the IP of mosquitto is being identified!)
gets to line 590 of the code and then does nothing! No further info (or debug) logging. Just stops afer 'Connecting to MQTT at 192.168.1.248:1883"
I’ll have a play after reading through the code, but my problem seems to be that I am not connecting to mosquitto from the bridge side, even though the Tasmota side is happily connecting with the right details.
I’m 80% sure it is something I’ve done wrong in the config, I just cant see what!

Most likely because Im running on a VM and the network is being funny, though I have checked (and tripple checked) the settings.

Rich,

Sorry for not getting back earlier - not getting notified from the messages here so didnt know you had replied. Maybe better if we can troubleshoot this on github issues since that notification always works and maybe others can benefit too.

Actually I take it back (ignore for now the section below the =====)
your log file should read something like

2019-11-05 04:17:10 AM info: Connecting to MQTT at mqtt://192.168.1.248
2019-11-05 04:17:11 AM info: Subscribing to - [bunch of topics if you have any]
2019-11-05 04:17:11 AM info: Configuring autosave
2019-11-05 04:17:11 AM info: Configuring API
2019-11-05 04:17:11 AM info: Listening at http://localhost:8080

if you are not getting to subscribing or the configuring autosave lines it implies the mbs-server is not able to connect to you MQTT broker. Try troubleshooting as follows
Check the error%date%.log - if mtt.connect is throwing an exception it should be captured there (any other exception too).
that will point you in the right direction - it definitely appears to be a problem with connecting to the MQTT broker

  1. Are you running the broker and the mbs-server on the same IP address in the same VM?
  2. is the node js mqtt library installed and in node path? did you install the package with NPM or jsut copied the files over?
  3. Can you ping/telnet to this IP port?

=============================
If you get to Listening at http://localhost:8080 . That last commands means it is waiting and listening.

That would mean your mbs-bridge device is not connecting to the server and neither is the mbs-smartapp.

Please confirm

  1. the bridge has been configured with the IP of 192.168.1.248 port 1883 (where the broker is running) and the MAC address of the machine / VM running the mbs-server (not the mqtt broker if they are running on different machines)
  2. In the smartapp besides checking the devices make sure you have also selected the bridge that you created. That configuration is much lower after the devices so you could miss it

also make sure your firewall is not blocking the mbs-server app from receiving requests on port 1883. At least in windows when I run the server firewall dialogue box comes up request access.

are you seeing any debug messages from the bridge in smartthhings IDE logging?

thanks for this… going through your suggestions now.
I’ve added some extra logging and noticed a few things relating to the indentation of the device.yml, but that’s a separate conversation!

to your point

if you are not getting to subscribing or the configuring autosave lines it implies the mbs-server is not able to connect to you MQTT broker.

yep that’s the problem! just got to work out why!

I’ll also be upfront now on something that shouldnt make any difference, but I am running the server and mosquitto on ubuntu in a virtual machine I have spare! I am obviously going to be fully documenting the process once it is working, but this will also lead to people maybe using a RPi as the MQTT bridge and broker. Have to admit, most instructions are the same as you provided! And as I said Mosquitto is working fine, and the smartthings MQTT bridge installed fine and does run! just appears a connection and setup thing… just cant see what it is though!

Im now moving over to Github to continue the conversation.
BTW… I’ll also be able to test on shelly 1, shelly 4pro, touch EU, touch 3, ifan03 and a few other devices which might be easier with MQTT rather than specific drivers.

Running on pi should not be an issue - i also ran it on an old pi but was not using VM. just running it as a pi service.

if both the server and broker are running in the same VM then not sure why they are connecting. did you check the error log ? maybe it is not finding the nodejs MQTT package, maybe the nodejs path cannot reach MQTT.

did you do npm -g install for the mbs package or just copied the files. don’t think i see a pull from NPM. if you just copied the files make sure mqtt and any other dependencies have the right vesion and are in the node path.

Just in case people were reading the chain and getting confused… we have got the bridge working on Ubuntu which means people can use it with a RPi or a windows box (or like me in a VM running unbuntu)
Just testing with a bunch of different devices including sonoff switches with tasmota, shelly switches with tasmota, shelly pro 4 (shelly code) - this means it would then be possible to use any shelly device without flashing!, and the sonoff iFan03. Will update when we have more news over next few days.

I’m currently running the old MQTT bridge and it has been working well for me. What would I gain from switching over to your bridge? I love that someone (you) is still active with the MQTT stuff and that alone might be reason enough to switch.

Hi - if the old bridge is working for you I am assuming you are integrating HAssIO with ST. If it’s meeting all your needs then no need to change.

If your setup remains the same benefits will be mostly in better logging and using updated node js packs so you have support going forward.

For someone like me who is heavily vested in ST and had no ability to use MQTT devices this provides a direct interface to pretty much any MQTT device - you can add pretty much any MQTT device to ST without waiting for someone to build custom device handler.

For DIY people like myself I can flash tasmota on any esp8266 project and create custom IOT devices to be included in ST without a shield, custom device handlers, different firmwares - like tasmota, st_anything - now I can focus on tasmota and MQTT alone and be done with it. Basically most of the benefits of HassIO but not as developed in discovery in rules engine.

It has already been tested on sonoff, Shellys , wemos d1, RF devices, and several others. So for everyone wondering should I switch from ST this should be a more viable option than the old bridge.

Selfishly, I would ask you to see if you can try out this bridge. I have tried to maintain all of the legacy functionality but have had no way to test it. For you it should work out of the box without any changes save specifying some new option in confit.yml. The environment set up is exactly the same unless you used a docker or any other pre-configured image.

It will be interesting to know if the legacy functionality still works

1 Like

Just as backup to the point, I’ve been playing with it for a little over a week. We have tested it on both WIndows and on a Ubunto machine and also know it works on a RPi.
I’ve tested on a range of devices and have it successfully working with a Shelly 4 Pro (better than any other attempt to support shelly 4 pro) and am just in process of writing the handler for the energy monitor on the shelly4.
Between us we have tested on a plethora of MQTT devices that are common and I’ve been giving a little feedback on some areas of confusion.
I am about to write some updated guides to help get started so GFam can focus on getting some updates. Watch this space over next few weeks.
Oh also for those with Sonoff and any other Tasmota device… this is much richer than the brett solution BUT the handlers will need to be written if they dont exist already… the main advantage is the respose to manual control of lights and access to all functionality in Tasmota.
Personally, I am loving this new implimentation.

Really appreciate the response. Like I said, I really like the fact that there is continued development and active community participation. That alone will probably be enough for me to switch over. I’m watching this thread and will report back once I configure it.

Process is simple - but I am writing a little more detail.

  1. install mosquitto… this is cross platform and seems stable - im running in ubuntu
  2. install a mqtt monitor to help with debugging… im using mqtt-Spy - im running on a windows machine
  3. enable MQTT on a test switch add ‘smartthings’ at the front of the topic as a prefix
  4. check you can see the messages coming through on mqtt-spy
  5. install MQTT Bridge on same machine as mosquitto (I assume it can work across machines, but this is easier!)
  6. setup the config.yml
  7. setup a simple switch initially in devices.yml
  8. run the mqtt-bridge-smartthings
  9. add msb-smartapp and tasmotaswitch device handlers using the IDE
  10. add the mqtt bridge app using the IDE
  11. create a switch device and bridge device using the handlers
  12. run the app on the phone and select the switch and bridge.
  13. watch you have the messages where they should be!

You need to be careful with the settings in the devices.yml and config.yml… if you got to step 4… it will be about settings for MQTT bridge, firstly the bridge should be seeing the MQTT messages flying around. If that works, and you see the messages. the problem will be how you set up the device handles.
If you are getting the MQTT bridge to crash with an error… it means the config or devices yml files are not correct.

This is by no means a compete explination, but if you have some knowhow, this is a quick starter.

1 Like

I’m using docker to run the MQTT ST Bridge. Do you have any plans to ship builds to dockerhub?

I have not done a docker image before.
Just read about it and seems like I should really do it.

So plan is to do it - hopefully I do not screw things up.
One change I do notice I will have to make is change the config directory because of the VOLUME requirements for docker.

So i will spend some time - doesn’t look like too onerous but what do I know yet about docker?

1 Like

This seems very interesting, looking forward to hearing more and hopefully a instruction to be able to set it up.

I just released version 1.0.3 on NPM and alpine and rpi docker images on docker hub.
Please check the read me page for the latest and complete instructions.

This a major upgrade to vesion 1.0.2

Changes in version 1.0.3
Significant upgrades

  1. Overlapping MQTT wildcard subscription enabled
  2. Completely removed and refactored the joi packages and jcsrc - every node js package is now at their latest versions
  3. Changed location of config files for docker implementation
  4. Providing docker images for Alpine and RPI platform
1 Like

I’ll be on vacation for the next week, but will be keen to test this out once I get back. In the meantime, keep up the GREAT work. It is much appreciated!!!

Getting your build installed via docker tonight on an Odroid XU4. Is there any particular reason why the rpi build is over 700mb? Only asking because the build that I was previously using (albeit on an x86 machine) was only like 36 mb (https://hub.docker.com/r/stjohnjohnson/smartthings-mqtt-bridge/tags).

I’ve got the space on the drive, so it’s no big deal, but perhaps is there some optimization that could be done? Anyway, once I’m up and running, I’ll be sure to provide feedback and the like.

Thanks again!

Frankly I am not sure. This is my first time building the docker images and i followed all the steps. I guess it would likely be based on the base image used and I used the most widely used images for the two platforms. Maybe the rpi-base image I am using is bloated too.

What I can say is that even though it is your first time building those images, the RPI image built successfully on my Odroid XU4. So we can work on slimming the image down, but at least we know it is working!

Looks like they were using the alpine build which is much much slimmer on their build for x86 - what if we rebased the ARM version to 10-alpine?
https://hub.docker.com/layers/arm32v7/node/10-alpine/images/sha256-88e17cc13b38382cb94870980d34b5b5cdfcbe76f3aba36d202ac59e154bc53d