Every Zigbee device is offline (Jan 2020)

Couple things I ask people to check themselves that are usually the source of problems:

  1. Did you install ANY new device in your home around the time that devices started to go offline? USB 3.0 devices have commonly been shown to be an issue, but there are many devices that can cause interference issues. If you did, power down/remove the new device and see if the Zigbee devices come back online after adequate time.
  2. Look at the devices offline in the IDE and look at their Next Hop field. See if there is a common device that all of the offline devices have listed as their Next Hop, and if so, power down that device and see if the offline devices come back online. If they do come back this would be a good indication that you have a device that is a bad router and Iā€™d recommend replacing with a new device of different make/model. Example: Screen Shot 2020-02-28 at 10.26.01 AM

These two cases have so far covered every single ā€œAll Zigbee devices are offlineā€ issue Iā€™ve looked into in this thread and from users directly reaching out :slight_smile:

6 Likes

@BroderickCarlin, all my offline/unavailable devices are zigbee and the Next Hop field all point to my hub. I removed a Ring camera that appears to bring down my zigbee devices since the last 2-3 days but still no luck. I rebooted my hub from IDE and battery/cables removed with still no luck. How can I rebuild my zigbee mesh?

Have you powered down the hub for at least 30 minutes to force a zigbee mesh rebuild?

What other zigbee repeater devices do you have in your mesh, and are they placed well throughout your home?

How many zigbee devices do you have, and are they spread out?

Update: after I removed the recently added Ring camera from ST, slowly over time almost all of my zigbee devices became available and working. I didnā€™t add back the Ring camera. This morning, however, a few of the same zigbee sensors are acting up again.

@johnconstantelo, I have about a dozen end-point zigbee sensors throughout my house and no zigbee repeaters. Everything has been working fine until I bought couple of Ring cameras.

You may have removed it from ST, but is it still powered up and working through the Ring app? Iā€™m assuming so. Itā€™s a wifi connected services device, not something thatā€™s part of your zigbee mesh, BUT it could be introducing enough interference in the 2.4g space that they are now/still causing you grief for zigbee devices.

I suspect you may be having interference and distance issues, so I recommend trying a zigbee repeater device like an IKEA, Innr, Peanut, or GE plug (there are also others).

2 Likes

Thank you. Things started working but I donā€™t know why. I didnā€™t change anything on my end but I had been bothering smartthings support for over a month. I donā€™t know if a firmware broke things for that period.

1 Like

I think I have found a reason why sometime things donā€™t work and then mysteriously start to work again.

For my case, first it was the Ring camera I had recently installed which was interfering with the zigbee frequency.

Second, I believe the mysterious working/not working situation is due to the router choosing or switching channels. When it picks a channel that is busy in the neighborhood, things start to act up. When router switches to a more available and free channel, thatā€™s when zigbee devices begin to work again without any intervention. Routers are by default on auto channel select but not all routers are quick to switch to a more free channel.

Ok - brand new external HD bay, USB 3.0, sitting right underneath my hub, which is up on a shelf. Turned it off and all my Cree bulbs are back online. So now the question is: how far apart do I need to move them?

@BroderickCarlin, could you give me a little help please?

I have experienced some sort of zigbee issue. A bunch of Xiaomi devices fell off the network together, what wouldnā€™t be really surprising as they are Xiaomi, but they some were stable without any issue for more than a year.
I can see in the event log a lot of ZBJoin messages about 4 hours ago, when this mass event happened, and the weirdest thing, all AC powered Osram plugs are non-operational. I have a total 3 pieces, they were working this morning, but they are not anymore responding to commands.
Nothing new has been plugged in or changed in the house at that time, so I am quite clueless. Has anything changed on the cloud side maybe?
Any idea how to find the source of the issue?

Update:
It is quite awkward, but all those Osram plugs stopped working together. They cannot be controlled and if a device was using them as a repeater, then it stopped working. If a device was using an IKEA device as a repeater then it is working. Any idea what is going on?

Update 2:
It makes things more awkward. Xiaomi devices pair back to the Hub without any issue. Osram plugs meanwhile not working, but when I re-pair then by holding the power button for 10+ seconds, they pair to the Hub without any issue and react to commands.
What could cause that devices are completely dropped off the network and unable the join by themselves?

What do you have Unsecure Rejoin set to?

Failure of part of your mesh due to a failing device, or a change in your routerā€™s wifi channel that will cause interference (some newer routers will change on their own but I havenā€™t experienced that).

Do your Osram plugs have the latest firmware?

These are what I can think off offhand right now.

1 Like

It is unsecured rejoin as before. And the router hasnā€™t changed channel. Zigbee 14 vs Wifi 11.

I donā€™t see any reason why those Osram plugs are unable to rejoin by themselves.
The firmware hasnā€™t changed on those since I got them, so that is not playing neither.

I thought of all of those.

1 Like

Some router capable devices are known to have bugs that for one reason or another can cause them to get in a bad state and stop properly routing messages - this is sadly a more common issue than weā€™d like. I canā€™t say for sure that you are hitting a device bug, but it is odd that the only devices you are having issues with are all the same make/model.

One thing to take a look at is the metrics field on the deviceā€™s page of the IDE. This is a relatively new addition that we are still iterating on, but it should give you a lot more insight into what is going on with these devices and their connection quality. Here is an example of one of the deviceā€™s in my home that I would consider to have a reliable connection to the hub. Youā€™ll want to verify that you have minimal duplicates and failures (zero in my case!).

2 Likes

You must be running a different firmware or my network behaves completely differently, but I have last hop LQI almost everywhere 255, but not lower than 240.
None of my router devices have zeros for duplicates or failures, all show some. Most surprisingly some new IKEA RGB bulbs does have failures and duplicates as well.
This event must have been centralized, as some devices dropped connecting directly to the Hub, some dropped connected to well known good IKEA repeaters. The Osram plugs hasnā€™t recovered by themselves, I had to re-pair all of them. And all were connected directly to the Hub.
I started to think, that it must be some sort of interference, but I havenā€™t installed anything new during the past few days, but added a few new IKEA bulbs a few days before and all works well.
It has definitely happened in a time window of 1 to 3 hours long, because I saw some missing reports during that period. The Xiaomi devices which hasnā€™t failed, must have had a schedule to log in which was just out of this time window. The only worrying thing is the issue of the Osram plugs. Why havenā€™t they rejoined? All IKEA or SmartThings plug joined back without any issue, the same applies to the SmartThings sensors too.
The presence sensors havenā€™t showed any sign of being away, what makes things more awkward.

@BroderickCarlin Iā€™ve got a similar mystery for your next-level smartthings knowledge to crack.

I just came across this thread and have checked the items you mentioned are normally the problem, but support canā€™t figure out why my Zigbee devices all failed a week ago, when I wasnā€™t even at home for 6 or so days before and a day after.

Iā€™ve moved my hub, did a soft reset, a hard reset for over 30 minutes off power, not one particular device my zigbee route goes, I moved my hub, and I donā€™t remember adding anything new. Iā€™m assuming an update caused a problem?

Appreciate any help as I have around 90 devices and really donā€™t want to have to reset every single light and automation.

I just got off the phone with my third person from SmartThings saying ā€œwe donā€™t keep backups,ā€ so itā€™s impossible to ā€œrestore a zigbee table from a backupā€ as @BroderickCarlin did. I was told Iā€™d have to go back to the property and unplug everything and plug it back in (though Iā€™ve done this multiple times this past week) and if it doesnā€™t work, I have to start from scratch, which will take days of programming.

I know youā€™ve got a better solution than this, Samsung! I would so appreciate someoneā€™s help

this has now happened to me twice. once last february and now just a week ago. this was right after firmware updates. both times i had to delete all devices in the IDE (faster) and hard reset the hub.

This is done by unplugging it, taking a paper clip and holding in the red recessed button the back while plugging it back in, release the button when the front led flickers yellow/red. the hub will re-download the latest firmware and re-install it. The front LED will show blue, magenta, and finally settle on solid green.

during this time the zigbee radio channel is set and cannot be changed unless you do another hard reset. so, before you do a hard reset go into the IDE and take note of what channel the zigbee radio is on. after you do a reset you want to first check to make sure the zigbee radio is on a different channel then when you previously had trouble.

if they canā€™t keep the zigbee radio channel the same as before a firmware update then they need to come up with a way to backup the devices for an much easier add back else more and more people are going to get upset and jump ship.

1 Like

Iā€™ll have to keep an eye on that because I donā€™t recall my zigbee channel changing after a hub update, but thatā€™s a good thing to note for next time.

1 Like

this last time mine went form something (20) to 19 and when i did a hard reset it went to 20. then everything was working again (zigbee devices paired and stayed paired).

2 Likes

Success! @Brad_ST escalated the issue to an engineering manager, and they fixed it within the day of posting. It seems as though @BroderickCarlinā€™s rollback method may have been employed as the newer devices I added after the failure are now offline (which is fine, as there are only two to fix now.)

Thanks so much for your help, everyone! Not sure why it happened, but hopefully it wonā€™t happen again. =)

1 Like

Sorry to HiJack this thread, but I am also having issues with all of my Zigbee devices staying offline. I see mention of a rollback, is this something the average person can do? Tried rebooting the Gen 2 Hub, unplugging the hub and removing the batteries for an hour or so, but nothing seems to help.
Thanks in advance @Brad_ST