Integration Solutions using MQTT

Devices & Integrations
, , , ,
406

I have small problem with MQTT devices. It seems like my hub is killing the driver after couple of days, as there’s nothing when I’m checking logs. I suspect this has something to do with sheer amount of events the driver is processing.

I have 5 RuuviTag sensors and I collect data from those with rpi4 and publish to Mosquitto running on same rpi every 5 seconds. As the tags have multiple sensors, I have separate humidity and temperature mqtt device for each tag. The sensors are ble beacons and sends data every second. To throttle it a bit, there was mode for ready-made collector which reduces it to once per 5 second. So, 10 events every 5 seconds.

I have couple of other mqtt devices as well (like switch, which gets info from Sleep As Android if I’m sleeping or not, which is used as a state info during the night (like when visiting bathroom, PIR sensor turns on one light with minimum brightness for a short time instead of normal dark time light scene for a significantly longer time)) and those won’t get updated either. Mosquitto and collector works fine as rebooting my hub (v3) fixes the problem for a couple of days.

I’d like to use the other mode of the collector, which pushes each event from sensors to MQTT and route it to grafana as well, as one sensor especially is in a place from which I’d like to get temperature and humidity to ST with one minute resolution (or 5), and acceleration data (IMU, axes are separated) to grafana with one second resolution.

So, would it be possible to throttle the data processing somehow in the driver? Either global or device level, something like “process only one event per device per x amount of time and ignore/drop the rest”, or do I need to figure out how to do that in the collector by using multiple topics for same data with different update cycles? I think I should be able to pull that off as the collector is written in python, but I thought this might be beneficial feature for others as well.

Also as a side note, device, which could receive all data (temperature, humidity, air pressure, acceleration with x, y and z separated, and battery voltage) from RuuviTags as JSON dict would be awesome! I want to thank you otherwise as well, your work is enabling me to do more with automations.

407

This is an interesting one. Would you be willing to run a long-term log so we can see exactly what is happening when the driver fails? You could open a terminal window on your Pi and use this command:

smartthings edge:drivers:logcat <driverID> --hub-address <hubIPAddress> | tee -a mqttedge.log

This will send the log output to a file as well as the console. When you notice the driver has stopped responding, Ctrl-c out of the CLI and inspect the file to see what happened.

You may be causing a significant load with the frequency of MQTT messages, however it is curious that it would run fine for so long before failing. I would have also expected you to perhaps run into a SmartThings rate limit, but that must not be the case.

Often when things run fine for so long and then fail, it turns out to be a network issue - something interrupting the link between the hub and broker. However the driver should eventually recover by itself from any disconnect from the broker.

I don’t think doing anything in the driver itself to throttle the processing could help. Most of the processing is in the receiving and parsing of the messages themselves- beyond that there’s not much beyond emitting updates to the SmartThings capability attributes.

Let’s see if we can confirm what is actually causing the driver failure. If we conclude it really is load-related, then we’ll see if there is something I could do on the driver side, OR you may need to implement a small script on your Pi to publish updates less frequently on a different topic.

408

Left that running on a screen on my linux vm as I’m bit concerned of my sd card on my pi :smiley: It’s behind ipsec at work but it should be fine, I have proper network gear (Juniper SRX on both ends, I’m IT admin in ops).

Is there any other information I could pull? I will post any findings from the gathered log once it decides to die again but as I rebooted my hub today, it’ll take anything between 2-10 days. I usually notice it when lights doesn’t turn off when I start sleep tracking or doesn’t turn on when I wake up and stop the tracking x)

Edit: And to add - the pi and hub are both connected via ethernet, 2 managed and one dumb switch in between so network connectivity shouldn’t be problem. Or if it is, I need to implement some finer monitoring.

409

Pull a sampling from the log and send to me in a direct message. Even before it fails, I’d like to get a feel for the rate of activity the driver is experiencing.

Prefix and postfix the log text with 3 back-ticks (```), which will help formatting for easier reading.

Thanks!

410

MQTT DEVICES DRIVER UPDATE NOTICE!

Driver Name: MQTT Devices V1.5
Driver Version: 2023-03-13T01:37:04.670198805

Change Summary:

  • New custom MQTT publish command for sending any on-the-fly message via a Rule (specify topic, message, and QoS). (@montyfert) Reference the example Rule provided in the updated README file.
  • Energy device enhancements:
    • Energy value can now be negative number
    • Added power field (@ragoss)
    • Can now set both power and energy field from within the app or by automation, and have the updated values published to a configured MQTT topic (@raisingskell)
  • New CO2 device (@thorsten1)
  • Temperature device: value range can be -120 to +200 degrees (@S.F.B, @montyfert)
  • Text device: text value now shown on dashboard (@jdbadams, @disgusdad)
  • New Numeric device for generic analog values (-1,000,000 to +1,000,000): separate value and units field; each can be updated in the app or by automation and updates published to a configured MQTT topic (@jdbadams, @simonbove)
  • Reconnection retry frequency is now configurable in device creator settings (@TapioX)
  • Default Switch values now all caps: ‘ON’ and ‘OFF’ (@BartschLabs)

WARNING:
Because this driver update adds new options to the list displayed by the device creator’s ‘Select & Create MQTT Device’ button, these type of driver updates can cause that button to (a) not get updated with the new options, and/or (b) be greyed-out and inoperable. This is particularly true on iOS devices.

The platform can take some time to straighten itself out - sometimes up to 6 hours or more.

These are the options you can try:

  • Be patient and wait to see if it eventually works again; be sure to reload your mobile app
  • If the ‘create new device’ button is greyed out, tap the Refresh button on the creator device to see if that makes it available again
  • If it’s still not working, you can delete and recreate the creator device via Add device/Scan for nearby devices; this won’t affect your existing MQTT devices, but you will have to re-enter your broker settings. You may notice anomalies with the newly-created creator device, such as the dashboard state saying “Offline”, or the Status on the Controls screen seems stuck on ‘Connecting…’. The former should clear up with time; the latter may actually be erroneous, but you can confirm by tapping the Refresh button, which will force a re-connect. You may then have to exit the Controls screen and return before you see the proper ‘Connected to Broker’ status being displayed. All this weirdness should clear up over time.
  • Last resort: reboot your hub
5 Likes
411

Thank you very much for the CO2 device, it is working perfectly.

1 Like
412

thank you works great
martin

1 Like
413

This is again great add-on to driver.

I have a completely new use for this function.
I have used it for Rules API rules execution monitoring.
I have added writing the status text to rule as a second command.

Here is an example of Rules API rules execution monitoring:

mosquitto_sub -h localhost -t “event/GU10_lamp”
On Level 60%
Level 80%
Level 100%
Off
On Level 60%

In this way it is possible to monitor the rules execution. and
you can get a list of the rules execution to Raspberry Pi’s or PC’s display.

414

That’s great! It’s a creative way to implement your own history tracking.

415

Todd,

I modified by esp8266 to send energy data in a JSON format as:

{
“KW”: 66,
“KWx”: “66.76”,
“KWHx”: “454.00”,
“KWH”: 454
}

However the MQTT edge driver does not seem to read this. I have it in Node Red and it works as expected.

I also found that when I publish from MQTT explorer this JSON as an example work fine:
{
“KW”: 10,
“KWx”: “0.00”,
“KWHx”: “0.00”,
“KWH”: 5430
}
The edge driver reads it just fine, this verifies the topic is ok, just not for the data from the ESP8266.

I am not an expert on MQTT but there does seem to be some reason I cant get the data into the Edge driver.

416

Can you show me a screenshot of your SmartThings device Settings page? It could just be a Setting is not quite right. Not sure which driver/device you are using?

417

image
image555×929 54.7 KB

Attached is the screen shot, as I said, if I publish the json from MQTT explorer it works, but does not work with the ESP8266 published items even though I can see the item in the MQTT explorer.

418

Todd,

I had taken my code the ESP and modified it to send both mqtt text and JSON, in MQTT explorer this seems to look ok, but I guess there was something not right about the code. I removed the MQTT text so there was only the JSON with power and energy. This seemed to fix the MQTT device in Smartthings. I am now getting readings as expected, almost.

I am sending usage in watt / hours, but the devices still seems to want to show KWH, not WH.

This is the JSON
{
“KW”: 996,
“KWH”: 492
}
and a screen shot of the results

image
image555×929 59.3 KB

419

Richard - so sorry - I completely missed responding to your post almost a month ago(!)

Don’t know if you already figured things out, but from what I can see from your screen shots it looks like you don’t have the units set correctly in device settings. So for example, you have a power field key of “KW”, so I would assume that means the value is in kilowatts. But you have the Power Units field still set to Watts. Similar issue for the energy field.

1 Like
420

DRIVER UPDATE NOTICE

I’ve pushed out a driver update for the MQTT Devices driver: Version 2023-04-21T03:13:12.777920586.

This update includes the following enhancement:

  • Added Window shade device type at request of @montyfert

You will have to delete and recreate the MQTT Creator device in order to be able to create any new devices, including the new window shade type device. You can do this without disrupting your existing MQTT devices: delete the current creator device, do an Add device / Scan for nearby devices, then configure the broker credentials and IP address in the new device.

If you have no need to create any new devices for now, you need not take any action at this time. Your current devices should continue to operate ok.

Let me know if you run into any problems.

3 Likes
421

works fantastic thank you so much
your a star
martin

2 Likes
422

Todd,

Thanks for getting back to me. I am still having issues with energy/power units on the driver. I would have thought that by changing this i could have KW or just Wats sent from MQTT it would be scaled accordingly. The only setting that seems to work is to sent energy in kWH and power in watts I may be all wrong on the point of this setting and would appreciate if you could clarify how this should work.

Thanks for all great work.

1 Like
423

I’m glad you pointed this out, and I was able to reproduce the issue and make a fix. I’ve pushed out an updated driver: Version 2023-04-23T18:18:01.353213457. The problem should be corrected now.

1 Like
424

Todd,

Thanks for the fix, it works as expected now.

1 Like
425

hi @TAustin
the window shade is working great thanks,
would it be posible to add the ability to read the battery status in it as well if its posible .
thanks and keep up the good work
its much apreciated.
martin