Suddenly, nothing is working and everything is laggy

@Rogersmj I’ll follow up on your ticket.

I also wanted to comment that we had no outages or slowdowns yesterday. The last issue we encountered was on 08/07.

Our tech team uses monitoring suites and PagerDuty to make everyone aware of an issue - then the status page at http://status.smartthings.com is updated within minutes of the incident.

That @Tyler ain’t so bad… :stuck_out_tongue:

Yes, support has been extremely helpful. I just post here as well in the interest of openly discussing these experiences in the community, so we can all learn.

They found what appeared to be a “zombie” node on my network – perhaps a device I had once included but then attempted to remove, but it didn’t really get removed. That has been removed and things mostly seem to be behaving better now. I still have some periodic lag but I still have to follow their suggestions about unpairing and repairing some switches, I just haven’t gotten around to that yet.

An update…not going so great.

Things are just constantly not working. Having ongoing conversations with Support. I decided to dismantle most of my network – I removed every single switch and plug device, leaving me just with motion sensors and contact sensors.

Then I added back just one set of switches, the ones in the basement closest to the hub. The problems persist – things aren’t firing when they’re supposed to from motion sensors, and I often can’t control stuff from the app. Perhaps of note, motion is always accurately reported – it’s just the switches aren’t responsive.

I also noticed something extremely weird after re-adding the 6 closest switches – one of the switches didn’t actually get added, but instead one in a different area was. I had named one switch “Basement Game Main Switch”, but a little while later when I tapped on that tile in SmartThings, my bar lights turned on! I was the only one home when re-adding the switches, and that bar switch was never touched. Here’s the other thing – this happened before. A few weeks ago, when troubleshooting, I noticed a few of my switches in the basement area were all mixed up. The names in SmartThings didn’t correspond to what I expected on the physical switches. I figured I had somehow mixed things up back when I was first adding them. But now since it happened again, I think there’s something else going on.

So anyway, I again removed all but two of the switches – actually a pair of switches in a virtual three-way – and the plug unit for my landscape lights (and the Aeon remote, that’s still included because I needed it to make the three-way). All of these z-wave units are within 15 feet of the hub. And still everything is not responding well. Z-wave network repair consistently reports failures to update mesh info on all 3 remaining z-wave devices.

I really don’t know what to do now. I have hundreds and hundreds of dollars invested in these switches, and I feel like I’m almost at a dead end here. Right now half of them don’t even work since the their three-way association got deleted (apparently) when I removed them from SmartThings, even though I had used the Aeon Minimote to set that up.

Playing around with the pairing process over the last couple days shows me some big flaws in how devices are fingerprinted and discovered.

Sounds like you are willing to separate out your network and add the switches back in one by one?

I think there is some behind the scenes caching of devicetype (especially custom devicetypes) for the fingerprint / discovery process. I don’t know how to “purge” this cache.

Anyway, I hope you can get this working, Z-wave can be tough to troubleshoot, but it appears you are going about it right…

I assume you run the z-wave repair on each add of device?

My only thought is to bring each switch on, one at a time, with no power to the other switches and make sure the IDE in devices sees only that switch you brought on.

Good luck, I know your frustration… Keep working with support. Keep trying. Don’t give up.

Well, I decided to continue deleting z-wave devices until there were basically none left. I removed all the z-wave motion sensors and door sensors I had too. Even once all the z-wave devices were removed, I got an error on z-wave network repair that mesh info failed to update for device [2B]. Support says not to worry about this, that the system will retain records for old devices for a period of time.

So with zero z-wave devices still included (except for the Aeon Minimote as a secondary controller), I proceed this evening to try and add back a switch. I can’t. Not a single one. In add mode, the hub never picks up the switch. I tried with three different switches, all within 10 feet of the hub’s location.

Then just for kicks I try to include them with the Aeon Minimote (even though I know secondary controllers can’t include devices). It responds promptly, and correctly, by flashing its red light to indicate it can’t perform the inclusion. So I know the switches are sending out signals. And when I do three-way associations with the Minimote, I never have any problem. It’s just with the hub.

So why the hell isn’t my hub working anymore? I used to have trouble including devices, but if I tried a couple times it would pick it up. Now it seemingly never does. If I think about the behavior I’ve been experiencing, it’s like the hub has slowly and consistently been degrading in z-wave performance over time. I started out having pretty much no problems, then a few glitches, then consistent misfires, then almost never working, to (now) pretty much not working with z-wave devices at all. I’ve asked support at least twice now if my hub could be defective but they haven’t directly addressed the possibility.

I’m going on two months now of screwing with this stuff and having half of my house not work properly.

Note that I have no other 900 MHz devices (that I’m aware of) that could be interfering with the z-wave network. I even unplugged my baby monitor just to be sure (reportedly the #1 killer of z-wave networks), even though it’s supposed to be 2.4 GHz. None of my immediate neighbors have babies (and thus no monitors). All my wifi access points in the house are locked to 5 GHz. There’s no security system or other wireless devices I can think of that would be interfering.

I think it’s time to demand a hub replacement, because I don’t know what else to try.

This is probably a stupid question, but you say you “removed” and you say “deleted” your devices, but are you sure they actually excluded? Is it possible the device still thinks its included in the network?

I did the proper exclusion for all my switches, and received the appropriate confirmation from the hub in the app that the device was excluded (and I no longer see it in the hub’s list of Things). Then more recently, when trying to troubleshoot, I put the hub in general exclusion mode and went around tapping switches…none triggered any other “device removed” messages as if it was still included. I’m not sure what else to do to make sure the switches are really removed.

I’m having this problem today.

@tslagle13

That’s what BGP is for.

Update on my saga: I have a very long ongoing thread with Support, and they finally agreed to replace my hub. We had pretty much gotten to the point where it HAD to be the hub, because we had eliminated every other possibility. I even excluded every single z-wave device from SmartThings, then created a whole separate Z-Wave network with every z-wave device in my house using my Aeon Minimote as the primary controller, and it all behaved perfectly.

I’ve had the new hub in for a couple days, and I’m SLOWLY adding devices to it, starting with the ones closest to the hub. So far, everything’s working OK, although I continually have issues where the hub detects things as the wrong device type (it insisted an Ecotech motion sensor was a door sensor, and that an on/off switch was a dimmer switch). But hey, at least the hub can talk to my Z-Wave devices again, which the last one hasn’t done for the better part of two months.

@rogersmj

I’d avoid working on this right now until they get their system fixed:

"Delayed Device Reports and SmartApp Processing

Investigating - We’re currently experiencing slower than usual processing of SmartApps and delayed device reporting. We are investigating, and will update shortly."

Sep 18, 15:13 EDT

http://status.smartthings.com/