Device Offline Issue

Starting yesterday, about 15 devices starting showing/acting offline. This was about 3 locks, 2 contact sensors and 10 dimmers or switches. I have tried to reboot the hub both manual and ide, repair z wave, change batteries in those that can. I have even pulled the air gap in the switches and reset the breaker for one area. Nothing works.

I finally decided that I would try one device removal and re-add just to see, and even that has gone haywire. I had to force delete it, and nothing seemed to happen when I manually excluded it, but i cannot add it back either. So now that switch isnt even connected, but still works as a physical switch.

I have not done anything recently that should have affected ST.

are all the offline devices z-wave? brands/models for the devices? which hub do you have? if v2, did you try powering off and remove the batteries?

have you opened a ticket with ST support?

it possibly sounds like one of your powered repeater devices went offline and took everything else that was routed thru it offline.

have you looked at the IDE for your acct?

Yes, they are all z wave. Its everything from GE/Jasco switches and dimmers (some z wave some z wave plus) to ecolink sensor to a remotec outlet. There may be a honeywell switch in there, not sure.
I have the v2 hub, and tried it without the batteries.

I have a ticket open, and every response tells me to remove and re-add, which has caused a bigger issue atm.

Yes, I have. There is no one common device that they run through, and at least one or two of them connect directly to the hub.

I think @jkp and @awestun are running down the right path. Iā€™d bet money you have a pretty serious ZWave routing issue going on. Why? This behavior sounds EXACTLY like what I had a couple months ago. and it was the result of a few ZWave Ghost devices clogging up the routing on my network. It took me 3 full days to fix it.

Have you run a ZWave repair recently? What were the results?

I have, and several devices seem to fail updating its neighbours, even one thats connected right to the hub. Why did yours start? I made no changes to mine recently.

1 Like

If you have that Iā€™m almost POSITIVE thatā€™s the issue then.

No specific time - it was something that just happened somewhere around the 32.x firmware release.

Ghosts can happen. Iā€™ve had one or two a year ever since the beginning. I try to go in once a month and look for stuff like that and clear it. (anyone who says you donā€™t need to do maintenance on your system is a flat out liar) What has been different - SOMETHING (we donā€™t know exactly what) changed in the ZWave stack around firmware v.32 - we know they made a bunch of changes to make them more ā€˜standards compliant.ā€™ (We know this because they broke the living hell out of child devices and such when they did and had to put out 2 emergency firmware updates to fix it.) My COMPLETELY UNSCIENTIFIC and not backed by data opinion - so my ā€˜feelingā€™ is that the ZWave stack ā€˜feelsā€™ less forgiving to ā€˜general weirdnessā€™ than it used to be since then. It seems to generally behave itself better when adding / deleting / repairing devices and seems to do a much better job on average then it used to in keeping the mesh healthy - BUT at the same time when thigs go wacky - they go completely out much faster. Before that time I never had issues that completely took out my mesh.

Now, when fixing this - youā€™ll find LOTS of articles on how to recover from ZWave mesh issues (search for Zwave Ghost and ZWave Repair) thereā€™s a FAQ that should help you figure out what needs to be fixed. Just remember:

Youā€™re working on a freight train - not a sportscar. ZWave changes are slow. You do not do yourself any favors by fast repeated repairs, add/remove/rejoin etc. In fact, repeated consecutive repairs will probably prolong your issue because the mesh never ā€˜settlesā€™

Helpful hints:

  • Do your repairs when you have plenty of time to let it run AND SIT afterwards - like HOURS - not minutes. (This will not be fast, sorry)
  • Realize that the Zwave Routing display in the IDE is just one of many possible routes, may be delayed for up to 24 hours and just the one that happened to be reported at the last update - not THE route. SO if a device is working but appears to show a wacky route - LEAVE IT ALONE.
  • If you need help - get the list from the repair (you can follow the repair log and copy/paste from it in the Hub log on the IDE)

Thanks. I will look into this as this seems like a good bet.

By Ghost device, do you mean something like a device with an ID but no name? I noticed in the repair it tried to fix device (14) rather than say Master Light (15), and there is no 14 in my device list.

If thats the case, how do I remove that ghost? Also Ill search lol.

1 Like

Yep - if you see a device ID for a nonexistant device - thatā€™s a Ghost. It usually happens when a device is removed from your system improperly. (Almost always, you removed it from the IDE but didnā€™t exclude it from the hub but other weirdness can cause it too, thereā€™s a lot about the interactions between the new app infrastructure and how the hub behaves that we the general users still donā€™t understandā€¦) So your hub is still telling all of your other devices that the ā€˜missingā€™ device is a valid repeater. They try to repeat through it andā€¦ Poof. In some cases they create an island of devices that cant communicate or drop a strategic repeater and cut off a whole chunk of your network.

The general repair for it is to manually create a new device (I use the ā€˜Zwave Deviceā€™ DTH) in the IDE, assign it the network ID of the ghost, then (VERY IMPORTANT) Delete it in the APP - not the IDE.

If teh app detects youā€™r eattempting to remove a ZWave Device - it actually will kick the hub into exclusion mode and try to do it gracefully, including deleting the ID from the routing table. (Thatā€™s why you need your temp device to use a DTH for a ZWave device)

Dropping it from the IDE - would be more like amputation by machete rather than amputation by scalpel. Sure, the machete gets the job done for now and with some stitches, youā€™ll liveā€¦ We get you off the battlefield to fight another day. But itā€™s not pretty and thereā€™s a mess to clean up later.

Thanks. I removed the one ghost that I found as non existent, Ill have to look if there are more. I only removed one device so hopefully this starts letting things come back online. The lack of a replace in the new app is brutal.

1 Like

Yeah I feel ya - I KNOW about the replaced ID trick for device replacement- and I donā€™t recommend it because of the same note above about ā€œLots o stuff changed after 32.x that we really donā€™t understand yetā€ so I will not explain how it works here.

Once you delete the ghost - give it a sit (about 30 minutes, have some scotch, watch an episode of Wandavision, whatever) then kick a repair. If you have a LOT of ZWave devices (some say 20, some 40) you may want to wait longer. Your ZWPlus devices will automatically pickup changes pretty quickly but your non plus ones (If youā€™re me, a TON of 12 and 14xxx series GE/Jasco/Honeywell switches) wont pickup anything until you run a repair or the network does normal maintenance. Then go back and see how it is (test a few devices, donā€™t worry about looking at the routes again - 24 hrs, remember?) after it sits again. Wash Rinse Repeat until you get everything works again OR one device that just REFUSES to fix itself in your repair. At that point, delete the device from the app - force delete if necessary. Then wash rinse repeat on wait, repair, test until you get to the same checkpoint above. If you get to everything works, NOW you rejoin any devices you had to force out.

Well, apparently my z wave mesh was quite haunted, but everything seems to be coming back now.

1 Like

i think i finally have my problems fixed. the ge/jasco/honeywell switches were failing. probably due to bad capacitor. ive replaced them with homeseer switches and devices dropping off line have stopped.

yeah, the homeseer swiches are probably made by jasco and will probably not lastā€¦

perhaps i will start recapping them.

Jasco extended the warranty for those to 5 years. Donā€™t recap them, send them for replacementā€¦ :sunglasses:

mine are GE, which i believe are made by jasco.
so are they covered?

Yes - all of the Jascos. If they are less than 5 years old and died due to the capacitor failing (blink /click of death, the usual) itā€™s covered. Call Jasco support, itā€™s a 15-20 minute phonecall, have the device in hand so you can read some codes and if it qualifies theyā€™ll set you up with a shipping label.