Hi, new user here. I’ve noticed that if I go into my IDE, and I look at the events list, all of the devices that are executed in the cloud seem to be going offline in the middle of the night. I’ll see activity that if offline or ping for those devices, like cree bulbs, or the Samsung multipurpose sensors. If I check devices that are locally executed around the same time, I see they never go offline. If I look at the events list for my hub during that time, I don’t see it as being offline.
So what is going on when my cloud executed devices are going offline? If a multipurpose sensor was tripped, like a window being opened, would I be notified of it?
I don’t have any automations that would run during this time. And I have noticed this seems to be happening for the last few nights in a row.
It happened again last night. I’ve done some more digging, and determined the following:
- Only my zigbee devices are going offline, the Zwave devices are staying up.
- Of my zigbee devices, the multipurpose sensors and cree bulbs are the most severely affected. I have 3 ST smart outlets also on zigbee, and one of them has never gone down, and the other two do not go down nearly as much as the cree bulbs and multipurpose sensors.
- This only happens between 2 am and 6 am. Nothing is going on at my house during that time, I don’t have anything that would trigger then, unless one of the multipurpose sensors are opened.
- The multipurpose sensors and cree bulbs go up and down independent of each other between 2 am and 6 am…
- I did not have my hub setup to update the firmware on zigbee devices, and the multipurpose sensors were running an older version of firmware, 0x00000019. Current version is 0x0000001B. I have changed that so the hub will now update the firmware on the multipurpose sensors.
- In the last few days that I can see the logs for, 12/1 thru the 12/7, there is one night where this did not happen, 12/2. Early that morning at 2:30, I was still up playing with the system, coding some pistions in webcore and using the multipurpose sensors to test the pistons. Somehow that kept everything online that night, even after I stopped testing.
Any thoughts or suggestions on what else I can do to determine the root cause and fix this? I guess I will know tomorrow morning if the firmware update for the multipurpose sensors fixed this.
Do you have a laptop or desktop connected to 2.4ghz? The only night you didn’t have an issue was Saturday. You might have a McAfee scan (running Sun thru Fri) or something going on creating a lot of network traffic and causing interference with your Zigbee between those hours.
Login to IDE and see what channel Zigbee is on. Check what channel your 2.4ghz band is on. Not in the settings (you probably have it set to Auto, but on the actual consumed wireless it should show you like Auto (11) or something so you can see the current channel it chose.
I do have several laptops that might be on the 2.4 band, I’ll check and see what they are doing around that time in the morning. Thanks!
That tells me though there is most likely interference because of the channels that both of those are sitting on.
Let me know what the Zigbee channel is and what your 2.4ghz channel is set to on your router. I will tell you if they are sitting on top of each other. If they are, this will continue to cause you issues down the road.
My zigbee is on channel 20, I won’t know what channel the router is using till later when I get home.
Just post it when you get a chance. The lower the channel that your wifi is on, the better. There’s no easy way of changing the Zigbee channel. 20 is kind of no man’s land that wifi channels 6 thru 11 would sit very close to.
I will post an article later referring to all of this.
is it worth it to try and rebuild the zigbee network? I can do that by powering down the ST hub for 10 minutes, right?
Ya. There are several things you can do, but the way I troubleshoot I do in thing at a time, so I know the exact cause of an issue. The fact that you are having problems with Zigbee only and not Z-Wave, and the fact that your Zigbee devices work fine with exception of between 2am and 6am only and not on Saturday, that leads me in the direction of interference and not a mesh problem.
I see the following things to perform and not all together:
- verify the channels - this will be a huge indicator
- make a channel change and let things be for 24 hours. See if issue occurs again.
- if it does, how far is your ST hub from your router. Sitting right next to it? Mine sits literally 18 inches from my Nighthawk and I have never had an issue with any of my Zigbee. But this does cause issues for some people and moving it further away would be the next step depending on where it is now.
- rebuild mesh (or I should say Zigbee heal) - I don’t think this is your issue though. Just my gut.
The reason being is that I have some LIFX bulbs outside (they are on 2.4ghz band and I have a scan running on my laptop and those bulbs will go offline, and I can open my app, pause the scan and the bulbs immediately return online in ST so I know that network traffic can play havoc on devices, but Zigbee is even more at risk.
So the 2.4ghz was on channel 11. I set it to auto.
After you set it to Auto, what channel is it now on?
The closer you are to Wifi Channel 1, the better - That’s furthest away from Zigbee channel 20.
Based on Zigbee channel 20, I would set your Wifi to either 1, 2 or 3. When you were on 11, you can see how close to the Zigbee channel you were.
I would then wait and see if you have the same issue between 2am and 6am tomorrow morning.
So… No dice. Same zigbee devices still went off last night. So the:
- Firmware updated on the multipurpose sensors
- Changed channel on wifi access point to auto(picked channel 1)
- Laptop was shut down overnight
My ST hub is about 18" directly below my wifi access point. I guess the next step is to think about moving it? But I would think if that were the issue, I would see this connection problem 24/7, not just between 2 and 6.
It’s pretty fascinating stuff. So was this right in the timeframe again?
So you mentioned Access Point and you got that set on channel 1, but what about the router itself? Ultimately, everything is going through there, so I would set that on Channel 1 as well.
One thing I wanted to ask. Does your Router have a 5ghz band?
If so, you said you shutdown the laptop. How about your phones? Are they connected to the 2.4ghz band or 5ghz? I would either shut all the phones down as well or have them connect to 5ghz only (as part of testing for now).
Something is definitely happening in that timeframe in your environment.
So you have the following things you can do:
- Move the hub further from the access point. (10ft or further)
- Skip the access point for now and plug hub directly into router (10ft or further)
Then (I would only do 3 and 5 alongside the above for now). The heal would be done later after these tests are performed, because that can just compound any issues as the heal can take up to 24 hours to complete it’s process)
- Shutdown all electronics on 2.4ghz or change them to only connect to 5ghz
- Perform a Zigbee Heal (power down / batteries out / wait 15 minutes) - Still don’t think this has anything to do with anything given the small window of time this is happening in.
- Turn on logging in IDE at 1:30 and leave it on. Whether you do this on a phone or your laptop, make sure you only connect to 5ghz with that device.
These are fun to track down. I know it’s frustrating but there is a culprit causing it and when you find it, you are going to laugh.
Oh btw: Do all the devices come back online all by themselves after 6am or close to it?
Additionally I wanted to add with electronics this could also include a tv and that is connected via WiFi 2.4ghz and downloads movies or some other streaming like device. I have a feeling one piece of equipment is causing this issue or something in the ST environment heavily inundating things (SmartApps on schedule going crazy, a ton of Routines - logging will show that), but I think it is local to whatever you have on.
I would also bring up your Router on the laptop (connected to 5ghz) and look at all the devices that are connected to 2.4ghz and watch the bandwidth and see if something sticks out. I have a screen on my Nighthawk that I can sort all devices by band and see the bandwidth. That would be a value add to perform as well
I have a Verizon fios modem, one that has the wifi built in. Also have a dual band wifi router. I disabled the wifi on the fios modem, and turned my wifi router into an access point. So the fios modem is strictly acting as my gateway to the internet. My ST hub is plugged directly into the wifi AP.
Its going to be difficult to move the ST hub 10 ft away from the wifi AP, because there is no other place to plug it into. And my wife doesn’t like seeing long cords everywhere, and its probably not the best idea with young kids in the house to have long cords everywhere, cause I get upset when things get broken.
My phones do connect at both 2.4 and 5, depending on where they are in the house. This may or may not be important, but I use the same SSID for both my 2.4 and 5ghz networks. Is this a bad idea? I set it up that way so if a device did not have a good 5G connection, it would easily be able to switch to 2.4ghz.
I’m thinking for tonight I will make no changes, and see if everything stays up tonight. If it does, I’ll have to figure out what is unique about Friday night/Saturday morning. if the problem comes back the following night, I think I will just try disabling the 2.4ghz band and see if that makes a difference.
I did write a few webcore pistons, and use the smart lighting app and SHM. Could this be causing this issue somehow? But if it were, I would think all zigbee devices would go down.
Great… I thought the smart things outlets were doing well, but last night they went down too. Here is what the logs look like for a device when this goes on…
2017-12-08 6:01:35.867 AM EST
2 hours ago DEVICE switch on Christmas Tree switch is on
2017-12-08 4:52:21.707 AM EST
3 hours ago COMMAND ping ping command was sent to Christmas Tree
2017-12-08 2:50:13.249 AM EST
5 hours ago DEVICE activity online Your SmartThings device Christmas Tree is now online
2017-12-08 2:48:19.265 AM EST
5 hours ago DEVICE activity offline Your SmartThings device Christmas Tree is now offline
2017-12-08 2:47:18.132 AM EST
6 hours ago COMMAND ping ping command was sent to Christmas Tree
2017-12-08 2:30:13.149 AM EST
6 hours ago DEVICE activity online Your SmartThings device Christmas Tree is now online
2017-12-08 2:28:15.556 AM EST
6 hours ago DEVICE activity offline Your SmartThings device Christmas Tree is now offline
2017-12-08 2:27:14.085 AM EST
6 hours ago COMMAND ping ping command was sent to Christmas Tree
2017-12-07 10:00:06.553 PM EST
10 hours ago DEVICE power 0.0 Christmas Tree power is 0.0
2017-12-07 10:00:05.450 PM EST
10 hours ago DEVICE power 44.8 Christmas Tree power is 44.8
2017-12-07 10:00:04.338 PM EST
10 hours ago DEVICE power 89.6 Christmas Tree power is 89.6
2017-12-07 10:00:03.584 PM EST
10 hours ago DEVICE switch off Christmas Tree switch is off
This was from one of the cree bulbs during the same time period:
2017-12-08 6:40:33.391 AM EST
2 hours ago APP_COMMAND setLevel Turn front door light up to 100% when motion detected on ring sent setLevel command to Door-front-light
2017-12-08 5:10:48.807 AM EST
3 hours ago COMMAND ping ping command was sent to Door-front-light
2017-12-08 4:38:32.839 AM EST
4 hours ago DEVICE activity online Your SmartThings device Door-front-light is now online
2017-12-08 4:36:43.512 AM EST
4 hours ago DEVICE activity offline Your SmartThings device Door-front-light is now offline
2017-12-08 4:35:41.292 AM EST
4 hours ago COMMAND ping ping command was sent to Door-front-light
2017-12-08 4:23:32.888 AM EST
4 hours ago DEVICE activity online Your SmartThings device Door-front-light is now online
2017-12-08 4:16:36.160 AM EST
4 hours ago DEVICE activity offline Your SmartThings device Door-front-light is now offline
2017-12-08 4:15:35.098 AM EST
4 hours ago COMMAND ping ping command was sent to Door-front-light
2017-12-08 4:03:32.930 AM EST
4 hours ago DEVICE activity online Your SmartThings device Door-front-light is now online
2017-12-08 3:56:39.948 AM EST
4 hours ago DEVICE activity offline Your SmartThings device Door-front-light is now offline
2017-12-08 3:55:38.276 AM EST
4 hours ago COMMAND ping ping command was sent to Door-front-light
2017-12-08 3:33:32.830 AM EST
5 hours ago DEVICE activity online Your SmartThings device Door-front-light is now online
2017-12-08 3:21:54.967 AM EST
5 hours ago DEVICE activity offline Your SmartThings device Door-front-light is now offline
2017-12-08 3:20:53.704 AM EST
5 hours ago COMMAND ping ping command was sent to Door-front-light
2017-12-08 3:08:33.942 AM EST
5 hours ago DEVICE activity online Your SmartThings device Door-front-light is now online
2017-12-08 3:01:45.810 AM EST
5 hours ago DEVICE activity offline Your SmartThings device Door-front-light is now offline
2017-12-08 3:00:44.428 AM EST
5 hours ago COMMAND ping ping command was sent to Door-front-light
2017-12-08 2:38:32.970 AM EST
6 hours ago DEVICE activity online Your SmartThings device Door-front-light is now online
2017-12-08 2:21:50.108 AM EST
6 hours ago DEVICE activity offline Your SmartThings device Door-front-light is now offline
2017-12-08 2:20:48.501 AM EST
6 hours ago COMMAND ping ping command was sent to Door-front-light
2017-12-07 8:08:08.794 PM EST
12 hours ago DEVICE level 40 Door-front-light level is 40
So let’s do this if you are going to do another wait and see tonight. Go ahead and perform the Zigbee heal now and get that out of the way so if the problem persists, that can be crossed off the list. Leave it powered off for 20 minutes.
Also - Do you have Device Health Check turned on or off? This could still be causing issues in ST with specific devices so turn it off for now.
Do those two things today and then turn on your logging in IDE prior to 2am. I would be more interested to see what’s going on in the Main Log and if their is any heavy activity going in between those hours, not just the bulb logs, but if you have a ton of stuff firing from WebCoRE, Smart Lighting, Routines that are just continuously running during those hours (or maybe you know if you have stuff scheduled to run between those hours).
Also as for the SSID being the same, I don’t think that poses any issue other than all your devices wanting to goto the 2.4ghz channel on their own because the signal is stronger and the further your devices are away from the network, the better chance they are going to automatically connect to the 2.4ghz band and with ST and Zigbee, less traffic on 2.4ghz when you can use 5ghz is better (if signal strength warrants it)
So on the IDE under my home, I click on list events, and I can see where this morning and yesterday morning ping commands started for things going offline, and they started within a second of each other on both mornings.
Device health check was turned on, I turned it off.
How do I turn on the logging in IDE? Do you mean just leave the web browser window open with live logging going?
I won’t be able to power cycle the ST hub till later this evening when I get home.
-
Yes
-
Ok, if it’s going to be that late, then you will need to allow for a full 24 hours for the heal to be completed, so test tomorrow morning and Saturday (lol the day we know there aren’t issues), so test thru Monday morning.