Any idea why my Smartthings hub is doing DNS lookups for domains like:
1.0.0.37660496
1.0.0.37663752
1.0.0.37660560
And others similar to those. It started doing this around 1:20am EDT.
Quite a few more continue to come through:
I’ve got a pi-hole DNS server set up. It’s designed to catch DNS queries and filter them. It has a logging feature to show you the queries. I’m noticing an odd number of lookups from the ST hub. They appear to be programmatically incrementing. Like there’s a software bug mistakenly looking for a domain instead of some other function.
Anyone else seeing DNS queries like that?
2018-01-30 22:20:29 1.0.0.33454272
2018-01-30 22:20:58 1.0.0.33454464
2018-01-30 22:20:59 1.0.0.33454464
2018-01-30 22:21:28 1.0.0.33458856
2018-01-30 22:21:29 1.0.0.33458856
2018-01-30 22:30:44 fw-update2.smartthings.com
2018-01-30 22:30:48 1.0.0.33573488
2018-01-30 22:30:49 1.0.0.33573488
2018-01-30 22:31:20 1.0.0.33578688
2018-01-30 22:31:21 1.0.0.33578688
2018-01-30 22:31:50 1.0.0.33578896
2018-01-30 22:31:51 1.0.0.33578896
There are lot of them like that until around 4pm when they changed to:
2018-02-01 16:17:31 1.0.0.38940040
2018-02-01 16:17:32 1.0.0.38940040
2018-02-01 16:18:02 1.0.0.38938704
2018-02-01 16:18:03 1.0.0.38938704
2018-02-01 16:18:38 dc.connect.smartthings.com
2018-02-01 16:18:46 124.2129103564.0.32968448
2018-02-01 16:18:47 124.2129103564.0.32968448
2018-02-01 16:18:50 fw-update2.smartthings.com
2018-02-01 16:18:55 dc.connect.smartthings.com
2018-02-01 16:19:16 124.2129103564.0.33029736
2018-02-01 16:19:17 124.2129103564.0.33029736
2018-02-01 16:19:46 124.2129103564.0.33029736
2018-02-01 16:19:47 124.2129103564.0.33029736
2018-02-01 16:25:28 10587268.2129103564.0.33020496
2018-02-01 16:25:29 10587268.2129103564.0.33020496
2018-02-01 16:26:00 10587268.2129103564.0.33020496
2018-02-01 16:26:01 10587268.2129103564.0.33020496
...and more like...
2018-02-01 22:10:22 124.2126506700.0.15779744
2018-02-01 22:10:53 10587268.2126506700.0.15780320
2018-02-01 22:10:54 10587268.2126506700.0.15780320
Anyone else seeing junk lookups like this?
I sent a support request to ST, but have no idea what to expect.
following with interest. Im not sure how to even look mine up. I did have a malware attack recently change the DNS addresses on one of our computers. I resolved it doing regular maintenance. I am one of the few freaks that tags the IP address on everything in my house and specifies DNS addresses at my router. Good thing I did. I will have to see how I can monitor my ST hub.
1 Like
I’ve looked into it some more and have determined that something happened around 5:19pm this past Tuesday afternoon (1/31/18) that has the ST hub going bonkers. It’s made TENS OF THOUSANDS of these requests.
Power cycling it has not stopped it. I pulled the power, the lookups stopped, I plugged it back in and the DNS lookups have resumed.
If you want to follow this sort of thing yourself the easiest way is to setup a pi-hole server. This can be run on anything with linux. All you need do is run this from a root shell on your linux box:
curl -sSL https://install.pi-hole.net | bash
What it does it act as a DNS server for your network that can filter lookups. The primary benefit is to be able to block tracker or advertising domains, effectively stopping your internal computers from even being able to find the remote service network names.
What you have to do is run the install script and then change the DNS records on your internal machines to point to where ever you’ve installed pi-hole. This can be done on each individual machine or in your DHCP server records. Anyway, that’s probably discussion better served elsewhere. But if you’re curious, get a linux box and head on over to https://pi-hole.net/ to learn more.
The only thing I did was to fire up the ST app on my phone. I’ve really not made much use of the ST setup for quite a while. It’s been running along doing some light duties babysitting some sensors and scheduling some wall outlet switches (to avoid running battery chargers for too long). But for the most part I’ve had no interaction with the hub. I didn’t do anything specific to the hub other than fire up the app and look around. I didn’t setup anything new on it.
FOUND IT!
I went through my list of devices and smart apps, removing many of them, one-by-one while watching the dnsmasq log. None of the smart apps made a difference. Nor did taking out a number of unused devices.
Not until I remove the Hue hub. Once I removed that all of the bogus DNS queries STOPPED. I’ve added the hub back again and the bogus DNS queries have not returned.
I also had this conversation, with more details, over in the smartthings subreddit over here:
So, why would the ST hub be using DNS to find a missing Hue hub?
3 Likes
It really shouldn’t be. That code doesn’t even do any dns lookups AFAIK. Does your Hue stuff actually work? I checked out your hub logs and I see errors saying it can’t poll state from the bridge. Actually, if you flip some bulbs on or off in the hue app, does that ever end up getting reflected in the ST app?
Right, it seems odd. I’m guessing the lookups where being made incorrectly. It would appear my ST configuration had an old round version of the Hue hub. I’ve not made a lot of use of m ST gear, so this didn’t get updated when Philips forced and update to the new square version of the Hue hub. I just never updated it in ST. I added the new square one back into the ST config and see that it shows an ID with a 22xxxxx hex address. This is the same address as is on the bottom of the Hue hub itself. I didn’t make specific note of it, but when I removed the old hub from the config the ID was 16xxxxx.
As I looked over the DNS logs it appears the leading portion of the DNS queries were consistent. It was only the last number that was changing (and not actually incrementally). I’m guessing these might be somehow related to the hardware addresses of the various Hue devices that were previously configured in the ST hub as being connected to the Hue hub. But the numbers in the DNS logs look to be decimal and aren’t the same as what’s printed on the labels attached to the Hue devices.
My current guess at what is going on is that when we format the IP address for the URL for our HTTP library, we’re somehow getting garbage instead of a real IP.
I’m not yet sure how that’s happening, but I’m going to keep digging through to see if I can find anything.
Would you mind if I rebooted your hub to try to get a look at something earlier?
1 Like
You’re more than welcome to do whatever you need if it helps track down bugs. While I may have eliminated the bug for my install but I’d be glad to know it could be fixed for others.
1 Like
Thanks.
I rebooted it and got what I wanted, but it does look like you’ve already fixed it. I think we can at least fix it so it can’t make obviously invalid IP like that.
Post back here if you see any more odd dns lookups like that in the mean time, though.
This took some wrangling with text imports and sqlite, but I boiled down the 93k records to what look like just six unique addresses:
"10587268.2130000588.0"
"10587268.2130000588.19981392"
"10587268.2130000588.19982808"
"10587268.2130000588.19982944"
"124.2130000588.0"
“124.2130000588.19931216”
These being the three leading parts of the request. The fourth part seemed like some kind of timestamp or something. It changed every few minutes.
I have 5 Hue devices, two 71992 light strips, two 72997 bloom lights and one 71460 Go (the portable one). I know the two light strips and the two blooms were added to the ST hub. I don’t know if I had the Go added into it or not. The devices have ID numbers on their labels but they don’t look like the query requests in the log.
Yes, sorry about fixing it before you had a chance to look. I wanted to snuff out what ever was causing all that traffic.
Does the use of a dotted address like that look at all like what might have come from a zigbee lookup?
| “124.2130000588.0” | “7C.7EF53ACC.0” |
|---|---|
| “124.2130000588.19931216” | “7C.7EF53ACC.1302050” |
| “10587268.2130000588.0” | “A18C84.7EF53ACC.0” |
| “10587268.2130000588.19981392” | “A18C84.7EF53ACC.130E450” |
| “10587268.2130000588.19982808” | “A18C84.7EF53ACC.130E9D8” |
| “10587268.2130000588.19982944” | “A18C84.7EF53ACC.130EA60” |
If there’s a way to roll back to a previous version or something I’d be glad to let you give it a try.
Otherwise my steps to repeat it would have been
- set up an old circular Hue hub
- add some Hue devices to it
- add that old Hue hub to an ST hub (probably using software from around 2/2016, if I had to guess)
- disconnect the Hue hub
In this case it’s just from trying to build an IP address string. Although the ones that are only 3 segments (2 dots) are a bit odd. Not sure what to make of those yet.
I don’t think there is, but thanks for offering. I assume this issue still exists in the current fw but is just really hard to trigger or possibly impossible to trigger now due to other changes in the hub or in the cloud.
I’ll track down an old bridge and play around with adding/removing/disconnecting it when I get a chance to see if I can trigger anything like this. And I’ll at least get this issue tracked in case anyone runs into it again.
The queries aren’t IP addresses, at least not the fourth item. There were thousands of requests being made, but they all boiled down to those six leading addresses. The last, fourth element changed. But not in an obvious or consistent way. That and the large numbers (well in excess of IPv4 limit of 255) made me think it wasn’t an IP address.
I’d have to dig further to see if there’s a pattern to the fourth element, but I was inclined to guess it was a timestamp or something. When I look for distinct data it returns 482 of them, as opposed to just 6 using the leading 3 elements. However, no two sets of queries used the same last element.
I didn’t play around with any date/time formatting because those numbers weren’t incrementing time-wise. That is, a query at one time was using a fourth number lower than one made before it, but not predictably and not coinciding with the turn of the hour or day. Perhaps the 4th one was some sort of checksum or something constructed on the ST hub’s end of it?
I can send you a dump of the dnsmasq log if you’d like.
Patrick, did you ever find out anything more regarding this issue?
We never did find a true root cause. We know that is was something that was trying to format IP addresses to be used in a URL, but was getting corrupt data from a cache on the hub. Because the data was corrupt some assumptions that code was making were being violated. That meant that the URL we had just built had the not-actually-an-ip format discussed above for the host. Then, very “helpfully” the library that URL was sent to would notice that wasn’t an IP, assume it was a hostname, and would try to do a lookup for it.
We have a start on a change to put in some checks to make sure it is at least plausible that what we got represented a valid IP. However, I don’t think it made it into the most recent release. It seems this issue kinda fell by the wayside and didn’t get picked up again since it didn’t seem to have noticeable issues besides extra errors in some power users’ logs.
And actually, on that note, thank for reminding me of this. It may be related to another issue we have just started seeing. [edit: Probably not related, darn.]
@collinjames are you seeing this issue now?
Yes, I did the same thing as OP and fired up a pi-hole this week to monitor my traffic. My hub is hitting these addresses about 3 times per minute.
Are you noticing any other issues with your system, specifically with control of your Hue devices?