Most ZWAVE devices offline. Ones online not responding

I have had a solid Zwave & zigbee network for 3+ years running on a 2gen hub. Yesterday, while trying to repair a Schlage lock which was showing online but constantly giving me an network error (a network or server error occurred. Try again later.) Later in the day i noticed some basic automations were not running, and when i looked in the IDE, nearly all the ZWave devices local & cloud based were showing offline (a couple zigbee devices as well.) These were mainly switches so not battery related. Even the switches that are showing online, are not responding to controls from the APP giving me the network error.

I have rebooted the hub, power cycled it, performed ZWave Repair which completes without error. Any thoughts appreciated on what to do next

When you power cycled it… did you remove the batteries?

But… login to IDE at https://account.smartthings.com and check the routes for your z-wave devices and look for any unknown routes which could indicate a ghost device. If there are, do a search on ghost in the forum to find instructions on how to remove. Then run z-wave repair again.

If you do need to power cycle… remove the batteries and power off for several minutes.

Thanks @jkp I removed the batteries years ago, so it was a true power cycle.

Double checked and a lot of the local offline devices are going straight to the hub. Anyway, I am not seeing any ghost devices

Any ideas on how to see if the ZWave controller on the hub might have died?

In IDE, look under the hub in the z-wave section

I looked there it says ZWave Functional, but every single device still showing online gives me the Network or Server error occurred. So that was leading me to wonder if it died.

Experienced any power events lately?

You may want to try going to the breaker and turning it off for different areas that contain the z-wave devices, turn it back and and see if that has any effect Give it a few minutes. Maybe do one area at a time. Run z-wave repair each time. Not sure on the exact steps to try… maybe others can give better answer on steps to take.

I would mention possible hardware issue on one of the z-wave devices but you mentioned a few of your zigbee devices were also off-line.

Also, contact ST support and let them investigate. If you are in the US, the best method may be to call and speak with someone directly to get a quicker response.

Last note: does IDE show the devices are off-line just to rule out it is not an app issue.

Thanks for the suggestions
I did another power cycle and waited to get the notification from the server that it was offline.

No power events, the switches work manually thankfully!
The IDE does show them offline so it’s not the app.
Interesting that going to live logging and looking at the MasterBR Dimmer which is online, but doesnt respond to the app, shows it is recvg the OFF command.

At this point, it does seem like a call to support is the next step.

As of addition and removal of some devices yesterday, I am seeing VERY similar behavior. I do have a single ghost device but a ling list of OFFLINE devices – only one device has this ghost device in its route. Almost all other routes of OFFLINE devices are direct to the Hub. I also see no errors in a zwave repair.

If you got any useful help from SmartThings support let me know!

Support hasnt been real useful, but then again considering what we pay for the support, cant complain too much (i did have to laugh at the first suggestion to “reboot my router” :rofl:, that seems to be the stock response these days for cloud based services).

I did have great success getting the devices back online, by going to each switch and cutting power via the tab on the bottom. (could have used the breaker, but that would impact more than i wanted or needed). So one device did show a ghost connection, that repairing the ZWave mesh as well as a reboot of the hub solved nicely.

I sill however have no control over the devices from the App, ActionTiles or Alexa. Strangely, automations from WebCore and within the app are able to control the switches, although super lagged

Thanks. I will try that (tonight) as MOST (but not all) devices seem to be GE hard-wired switches. I do have SINGLE ghost device but cant seem to get rid of it (I see no issues in a zwave repair). I’ll report back here!

unfortunately it sounds like you will have to reset and start over to get everything working once again. trying to nit-pick at things will never get everything working. I speak from experience. yes, its very frustrating, but know that it will fix all of your problems. though right now i would not start over until they finish killing off everything like the IDE. then you will have a good clean re-start and not have residual shit mixed in with new.

Sadly, I have come to the same conclusion. Even after getting all the switches back online, within an hour, they were off again. Now i have to decide if I redo everything in ST or just move to something else.

what i have done and recommend at this point is to remove all the batteries form your battery devices and hard reset the hub. for now just add a few basic devices using the IDE that will get you along until they kill off the IDE and fully engage the edge device drivers. then you can hard reset the hub again and just have the edge device drivers.

Any idea on the timeline for the edge drivers? As soon as i heard about that in the works, i stopped my other POC. But now I have nothing except ZigBee devices working :slight_smile:

How long did you power down the hub?

Note: I am not a fan of resetting hubs - my opinion

When contacting support - request that it get escalated to a higher level of support. Calling is generally the best method when contacting them if available in your region.

Yeah, disconnecting the GE switches didn’t help at all. I WAS considering hubitat. Maybe now is the time. Resetting from scratch is a LOT of work for me. Tonnes of devices, and a lot of WebCore (and actiontiles) to update … And honestly, I suspect I’d eventually just hit the same issues. Of course, hubitat is not perfect either from what I see.

I may try a few things but from advice here, doesn’t look promising … And now (GASP) we will need to turn on lights on and off manually for a bit :slight_smile:

So, oddly, all devices are back online and all seems operational. I removed the one device (eventually) that had a ghost device in it route, unplugged two new motions I was adding when this all happened (2 zooz outdoor motions). And did a repair and reboot for kicks. Still looked in bad shape. Overnight most of the offline devices came back online. The others I basically took the batteries out and put them back (they were all aeotec multisensors) in to wake them and they came back online. The other thing I did was disable webcore, apptiles, and a lock smart app to reduce load basically. I have no idea what happened (does not give me a warm fuzzy). I’ll report here on progress (if any). Right now all seem to be working, onl;ine, and with apptile/webcore operational. Odd.

So a few days later update. All is humming away still. Actually better than I have seen for some time. Maybe there was some bad networking that got fixed in my tinkering?

Same here! I escalated with support on Friday and recvd a note saying give us a couple days to figure this out. I unplugged the hub for a day and half. Plugged it back in this AM and started noticing ALL the devices were online and accepting commands from the API as well as the APP. Dont have any idea if they did something or what

Ha, maybe that changed helped me too!