@TAustin : Are you aware if there is any embedded MQTT Broker (like a Edge Driver) doing the same thing then MQTT Mosquitto ? My Mosquitto MQTT Docker works very well, but if I can get rid of it and using any Smartthing Hub integration instead I would give a try
I can definitely do this. Hopefully by sometime next week.
There is no Edge MQTT broker that I am aware of. The problem right now is that no Edge driver can request to use a specific port number. So not only couldn’t you use the standard MQTT port, but whatever port you did get assigned would keep changing every time the hub got rebooted or the driver got updated. And how would the clients even know what ‘latest’ the port is?!
I personally don’t think a SmartThings hub would be a good place to run something like an MQTT broker. It’s just not reliable enough. You really need something that never has to be rebooted and is rock-solid reliable. That’s why Raspberry Pis are so great for this - they consume next to no power and run forever.
I guess today was the day, at least for me. Thanks to TAustin for saving ST integration for my expensive security system (MQTT Creator), whole home audio (MQTT, Web Requestor & vEdge) and soon my Google Sheets logging. I never thought there would be a solution for those issues. Thanks for the great work! Now on to a solution for Ecobee Suites.
1 Like
I lost all mqtt messages from both a v2 and v3 hub. The broker is working fine with the habitat messages. I have rebooted the hubs, restarted the broker, removed and re-added the drivers. I get a ping send and response from luamqqt of the broker log, but nothing else. I open the device creator, see the status as connected, press the gear icon for a button. I get a device created, then go to discovery, refresh, but still nothing.
What next?
Thanks Todd for to clarify this, Your explanations make sense. I run my MQTT in a docker on my Synology NAS I will continue to keep it there following your information.
I just discovered that the MQTT Device Creator V1.1 doesn’t reconnect automatically to the broker.
For testing purposes I shut down the MQTT Broker for a few minutes and the state changed to “Reconnecting…”. After starting up the Broker the Device Creator didn’t reconnect for 30 Minutes until I hit the Refresh-Button. Also the subscribed topics didn’t update until I refreshed a random MQTT-Device.
Is there a way to refresh MQTT-Devices using a routine, e.g. every 10 Minutes, just for the case the Broker goes down?
@TAustin
Everything worked great during three weeks MQTT Devices driver’s testing.
Today I modified my MQTT setup.
I moved MQTT broker from Raspberry Pi to Raspberry Pi Zero.
I have temperature measurements connected to Raspberry Pi.
The MQTT publisher client on Raspberry Pi publish temperature measurements to the MQTT broker on Raspberry Pi Zero.
The MQTT broker on Raspberry Pi Zero publish temperature measurements to SmartThings.
Also Zigbee temperature and humidity measurements are published from SmartThings to MQTT broker on Raspberry Pi Zero.
Everything worked about 3 hours.
Then publishing temperature measurements from MQTT broker to SmartThings stopped.
Publishing temperature and humidity measurements from SmartThings to the MQTT broker continued to work.
In driver log
2023-01-13T13:20:55.514790272+00:00 WARN MQTT Devices V1.1 MQTT run_sync returned: closed
2023-01-13T13:20:55.523758605+00:00 INFO MQTT Devices V1.1 <Device: xxxxxxxx-4f82-49b7-8a85-xxxxxxxxxxxx (MQTT Device Creator)> emitting event: {"attribute_id":"status","capability_id":"partyvoice23922.status","component_id":"main","state":{"value":"Reconnecting..."}}
2023-01-13T13:20:55.552718938+00:00 INFO MQTT Devices V1.1 <Device: xxxxxxxx-4f82-49b7-8a85-xxxxxxxxxxxx (MQTT Device Creator)> emitting event: {"attribute_id":"topiclist","capability_id":"partyvoice23922.topiclist","component_id":"main","state":{"value":" "},"visibility":{"displayed":false}}
2023-01-13T13:21:10.591496965+00:00 INFO MQTT Devices V1.1 Attempting to reconnect to broker...
2023-01-13T13:21:10.700014197+00:00 INFO MQTT Devices V1.1 Connected to MQTT broker: CONNACK{type=2, rc=0, sp=false}
2023-01-13T13:21:10.716647843+00:00 INFO MQTT Devices V1.1 <Device: xxxxxxxx-4f82-49b7-8a85-xxxxxxxxxxxx (MQTT Device Creator)> emitting event: {"attribute_id":"status","capability_id":"partyvoice23922.status","component_id":"main","state":{"value":"Connected to Broker"}}
2023-01-13T13:21:10.758434059+00:00 DEBUG MQTT Devices V1.1 driver device thread event handled
2023-01-13T13:21:10.771385865+00:00 DEBUG MQTT Devices V1.1 driver device thread event handled
2023-01-13T13:21:13.822066444+00:00 DEBUG MQTT Devices V1.1 Already subscribed to topic koti/th/lt2
2023-01-13T13:21:13.827863007+00:00 INFO MQTT Devices V1.1 <Device: xxxxxxxx-b908-49fe-a30f-xxxxxxxxxxxx (MQTT TH LT 2)> emitting event: {"attribute_id":"status","capability_id":"partyvoice23922.status","component_id":"main","state":{"value":"Subscribed"}}
In Mosquitto MQTT broker log
1673615981: Client luamqtt-v3-4-2-4e60e6f has exceeded timeout, disconnecting.
1673616070: New connection from 192.168.0.106:40346 on port 1883.
1673616070: New client connected from 192.168.0.106:40346 as luamqtt-v3-4-2-ea4b996 (p2, c1, k60).
1673616070: No will message specified.
1673616070: Sending CONNACK to luamqtt-v3-4-2-ea4b996 (0, 0)
I can mail to you more log data if you think that it’s useful.
Edit
After pushing device’s (MQTT TH LT 2) Detail View Refresh button communication from MQTT broker to SmartThings continued to work.
Thanks for the information. I think this line has provided a clue:
2023-01-13T13:21:13.822066444+00:00 DEBUG MQTT Devices V1.1 Already subscribed to topic koti/th/lt2
What could have happened is that after reconnecting to the broker, the devices didn’t get resubscribed. I’ll have a look at the code and see if there’s something I can do.
Yes no resubscribing in Broker’s log
Next item in Broker’s log
1673616100: Received PINGREQ from auto-293F1C1B-B65E-0B12
1673616100: Sending PINGRESP to auto-293F1C1B-B65E-0B12
@TAustin - thanks very much for your local web request multi app - this works really well for triggering a script on a Raspberry Pi. I ended up using python3-flask as the listener.
Then using this helpful guide - Python WebServer With Flask and Raspberry Pi | by Marcelo Rovai | Towards Data Science but augmenting the hello world example with:
os.system("/home/pi/runscript &")
import os
from flask import Flask
app = Flask(__name__)
@app.route('/')
def index():
os.system("/home/pi/runscript &")
return 'command executed'
if __name__ == '__main__':
app.run(port=8080, host='0.0.0.0')
Two quick questions:
- In your instructions you mention you can create a ST button - is this a Scene or do you recommend another way? I can use a virtual switch - on IoS you have to add the virtual switch via the web environment.
- If I don’t wish to have a webhook and just want a one way export of ST sensor values can I use any of your drivers to push values into Home Assistant or MQTT.
Thanks a lot - your work is excellent!
Best wishes
Ian
I think I was referring to creating a virtual device as you did. You can use the switch virtual device, but momentary button might be more suitable for this purpose
Each driver has a specific purpose so it depends on what you want to do. Some like webrequestor or HTTP Devices drivers use HTTP requests, others like my MQTT Devices driver uses MQTT. The tricky part is that there’s no easy way to take ANY device and “export” its value - for that you really need a SmartApp. There’s no way to “mirror” device properties onto another device. It takes automations which limits what you can do. Not sure I’ve answered your question?!..
To all who have been reporting problems with their SmartThings MQTT devices “losing” connection to the broker and no longer getting updated:
I’ve done some work to try and mitigate these issues and improve recovery when the connection with the broker is lost - or it is not found at driver startup.
I’ve pushed the updated driver out with these changes and it will be automatically updated on your hubs. You can verify you have the update by checking the driver version:
Name: MQTT Devices V1.3
Version: 2023-01-13T23:24:37.711287782
Please let me know if this cures the issues you have been experiencing.
Note also that although this driver update should help you recover from disconnects with the broker - you really should try to find out why it is happening in the first place if it seems to be a frequent occurrence. It could be a networking issue: weak wifi signal, or something else.
1 Like
Momentary button is the way to go yes good point!
No thats perfect thank you very much - I think if I want to keep ST as my main Zwave/Zigbee hub I may play with Nabu Casa as while I could host a webhook it sounds like a hassle creating the SSL and opening up a port. Thanks again - I will certainly make use of the HTTP GET/PUT stuff while I figure out what is best for the sensors. All the best!
Again, let me thank you very much for your fantastic work.
Unfortunately the new version doesn’t work for me. After shutting down the MQTT-Broker the Handler-Device won’t reconnect, hitting Refresh doesn’t lead to a reconnect.
The only way to get the Handler-Device connected is to reboot the hub.
What kind of hub do you have? This is really perplexing…
I have a V3 hub, the MQTT-Broker (Eclipse) is running within a Docker Container on my NAS. The Broker is configured for TCP, should I change to websockets?
Before the new driver version the Refresh-Button of the Handler-Device was working :-)
I found today one similar connection problem from driver log.
WARN MQTT Devices V1.3 MQTT run_sync returned: closed
Now devices get resubscribed
INFO MQTT Devices V1.3 Device <MQTT TH LT 2> subscribed to koti/th/lt2: SUBACK{type=9, packet_id=2, rc={1}}
No, TCP is the right configuration.
What was the reason you needed to use the Refresh button in the first place? Had your device stopped getting updates?
Make sure you are not having port blocking problems running the broker inside a container. I wonder if there is something specific to the Eclipse broker that is causing issues…
I used the Refresh button only for testing purposes to check how to reconnect when the broker reboots.
There shouldn’t be any port blocking problems (it reconnects after the hub reboots, static IP via DHCP), but just to be sure I will move the broker to a Raspberry Pi (no docker) on Monday. I also could test if downgrading the broker from 2.0.15 to 1.6.15 makes a difference.