Starting a couple weeks ago I began seeing massive slowdowns with a-wave switches. My zigbee devices worked just fine. This coincided with switching all 150+ devices over to the new hub after they deprecated the V1 hub. Support, after a couple weeks of back and forth, tells me they believe it’s an issue with security protocols on a mix of z-wave devices.
They think the old hub didn’t use security encryption, so the devices that did have security were downgraded to work without security…so they all worked together in the mesh.
Now that security is supported, half my z-wave devices won’t talk to the other half…so I have massive lag.
I don’t know how to tel which ones are secure, and/or how to change that. Anyone have any ideas?
Hmmm…that’s not how zwave is supposed to work according to the independent third party protocol: you can have devices at any of the different security levels and they should all be able to handle normal network traffic without bypassing anything with two exceptions:
you can only set up direct association between devices at the same security level
Devices with the highest security level, normally door locks, may not accept commands from some devices at a lower security level, typically handheld remotes.
But other than that, any mains powered light switch, for example, should be happy to pass along a message from any sensor regardless of their relative security levels. That’s part of the “backwards compatibility“ that is built into the standard. Here’s a statement from the company that now owns zwave and designs the security specifications:
Z-Wave never compromises on backwards compatibility. We want existing devices to interoperate smoothly with new controllers. If you come across a 10-year-old Z?Wave device, you will find that it includes and operates seamlessly with even the newest controllers on the market. We believe that the chosen security solution follows that tradition and strikes the best possible balance between security and interoperability. We have a clear path forward for evolving security across the entire ecosystem to become even more robust without sacrificing convenience and interoperability.
So what you are seeing is not supposed to happen just because you have different zwave devices at different security levels.
That said, I have no idea what happens in the SmartThings implementation of the S2 standard, the cloud architecture introduces some idiosyncrasies.
If smartthings engineering verifies that there’s an issue if you have devices at different zwave security levels, I believe them, but then they should be publicizing that much more widely since it’s a very significant deviation from the standard.
Hopefully someone who is more familiar with the smartthings details will respond. Quite a few people have been reporting over the last couple of weeks that they are seeing significant Z wave slowdown, but yours is the first time I’ve seen any explanation given.
The only way I could imagine this as an issue is if the v1 hub didn’t support Z-Wave S0 security, and now devices that have S0 (and not S2) are pairing as such. S0 security uses 3x the messages and can cause Z-Wave mesh issues.
This reply you got was written by me. The S2 devices you have Included all were reporting as “S2_FAILED” which is going to be an issue for their performance.
Presently, you have a bunch of non-secure devices (pre-S0), a handful of S2 devices that have all failed S2 enrollment, and a small amount of S0 devices. I feel that it is a fair summary to say that security level issues due to a mixed bag of Z-Wave devices likely explains at least some of your issues.
I never said it was impossible for your current setup to be operable on a newer V3/Aeotec Hub, just that at present you are having several issues that you would not have been aware of on a V1, due to how old the Z-Wave Radio on that unit was. On a V1, everything would have downgraded to pre-S0, non-secure communication, which would have served you well since I would say offhand 60% of your Z-Wave devices fall into that category inherently (back of napkin math, I did click into a lot of your devices, but not all).
The delays speak to one of two things (in my opinion):
More messages than can be handled efficiently and a backed up “queue.”
Insufficient physical coverage by the mesh or distance related issues.
Since you have many devices, and seemingly didn’t have issues prior, I feel that #1 is the more likely explanation (this is also to @jlv point above).
Let’s continue troubleshooting, I don’t think my theory will turn out to be completely wrong, but there may be more to the story. The first step we provided was a “Z-Wave Network Repair” after we had deleted two ghost Node IDs for you. The next would be getting your S2 devices properly enrolled as S2-Authenticated or S2-Unauthenticated devices (depends on model). After that, if there are still issues we can certainly walk you through getting Hub Logs to identify if a particular device (or devices) are “chatty” and causing a queue to build, which would lead to delays.
I was working through Samsung support, and it took a long time to get to this ‘security’ answer. They then told me they were at the limit of what they could do and said I should contact the forums for help.
There was discussion about S0 and S2, but I don’t understand how they’re not backwards compatible. Many of my z-wave devices are older (several years) so I would assume they’re less sophisticated and would (in theory) be a non-issue for the new system.
I don’t know if perhaps altering the device type to something generic would help this issue or not.
Hence why I’m posting here…hopefully someone has additional info!
No problem. Thanks @JDRoberts for the tag on this.
First, what you will want to do is Exclude the devices using the Hub (Open App → Find Hub device tile → Click it → Three dot menu in the upper right → “Z-Wave Utilities” → “Z-Wave exclusion”
Once Excluded, you can add them as you normally would. Depending on distance from your Hub, you may want to move your Hub closer to give it a better shot as success (hopefully it is on WiFi).
Your older Z-Wave devices will not support NWI (Network Wide Inclusion) which is a great feature of more modern devices. This means you need direct range to the Controller (SmartThings Hub) for Inclusion to be successful.
If you remember which devices you’ve recently purchased/installed (should be S2 and all S2 devices I looked at on your account failed) or you click into your Z-Wave devices in the IDE and look at the “networkSecurityLevel” field, it will show as “S2_FAILED” which means you should re-enroll them.
It sounds like there may just have been some misunderstandings. That can definitely happen, this stuff is complicated.
@garrett.kranz says in his post in this thread that his current troubleshooting theory is not because you have some S2 and some S0 devices on the same network: it’s because your S2 devices didn’t join properly, so they got the “failed“ message. That’s a whole different issue than just being an S2 device.
“Enrollment” is a technical engineering term for “add to the network.“ Same thing as “include,“ or “join.”
“Disenrollment” is the opposite: a technical term for “remove from the network.“ Same thing as “exclude” or “disjoin” (yes, that’s a word, even if only engineers use it. )
So he is saying that in order to fix the S2 failure message, you have to remove the device from the network and add it back in again. i’ll leave it to others to discuss exactly how to do that in a smartthings environment.
Also, someone will need to help you learn how to sign into the web interface to your smartthings account, the “IDE.” That’s where you can see the current status of the devices including their security levels.
Alright. As another pain point, it’s nearly impossible for me to get the app to work properly. It crashes all the time. This, I use a stand alone remote to exclude devices from the hub. Is this a contributing issue? Must I use the hub’s z-wave exclude to have success here?
You should be using the Hub to Exclude devices, otherwise the Hub has no idea the device is removed. This is probably why I found “ghost” nodes on your Hub.
You can use the IDE linked above to perform an Exclusion as well if you are having App issues.
Unfortunately I’ve tried both and neither works well to exclude devices. Since the new hub is WiFi, I guess I’ll have to move it around the house. It’s usually wired directly to the router. I’m in the IDE a lot over the years trying to keep this stuff working so I’m at least somewhat familiar.
When adding devices back in, is there anything specific to do to find success? I usually just use the ‘scan’ command. Is this sufficient?
That would definitely contribute to the creation of ghost devices. You might be factory resetting the end device but not actually removing it from the hub’s network tables. And that can cause a lot of problems in the future. normally we’d only do that if you didn’t have the old hub anymore, so you didn’t care about any ghosts that it had.
Once you get into the IDE as I mentioned above, I believe you can still run a general exclude from there if the app isn’t functional for you. But I believe you do need the app to add the device back again, so you’ll definitely need to get that straightened out as well.
Oops! Crossposted with @garrett.kranz so, yeah, what he said.
BTW, he is a very experienced Zwave engineer as well as thoroughly familiar with the smartthings platform, so you’re in good hands there.
“Scan nearby” is more so meant for WiFi devices/Samsung Appliances/BLE/etc. I would use either the “Generic Z-Wave device” selection or use the “By brand” and locate as close to your actual device as possible for the best performance with Z-Wave devices.
Ok. I’ll try that. Maybe moving the hub around the house is going to be the fix for this. I had a terrible time trying to remove or add anything when I moved them all across to the V3 hub. About the only thing that worked was the ‘scan nearby’. Again, I’m not a big fan of the app.
Hopefully I’ll have some time today to work on this a bit.
I’ll report back. Thank you for the help. It’s a real bummer to have a well-working, large, heavily integrated system become so in-useful. I’m. It wild about rewriting all the automations but if I need to replace the devices to make this work I guess that has to happen.
Best advice for adding zwave devices to any hub, regardless of security level, is to focus on building a strong backbone.
From the wireless range and repeaters community FAQ:
zigbee repeats only for zigbee and zwave repeats only for Z wave. The repeaters form the “backbone” of the network. So there will be one backbone for zigbee and one backbone for zwave. When installing several devices at once, install the repeaters first, beginning closest to the hub and working outward, so that the other devices will be able to use them to reach the hub. Then go back and install the battery powered devices. Using this method, zwave plus Devices should be able to be paired in place.
.
Some Z wave classic devices may need to be moved close to the hub to pair, and then moved to their desired location (or you can move the hub close to the device). Z wave locks will need to be paired very close to the hub so that they can exchange an encryption key.
.
If you do have to change the location of any devices, including moving the hub around, during the join process, you will then need to run a Z wave repair utility once everything is in place to update the neighbor tables.
OK, after going through my devices on the IDE, there were only 4 that showed as ZWAVE_S2_FAILED. Nearly every other z-wave switch I have shows up as ZWAVE_LEGACY_NON_SECURED. In the interest of fairness, I excluded and then added three of the four (couldn’t get one of them, it’s just too far out).
The three I added back now show up as either NON_SECURED, or S2_AUTHENTICATED.
I’ve run a z-wave repair and the only errors I see now are 4 devices that failed to update mesh, but no security failures.
Nothing appears to be fundamentally different at this point. Still virtually no control from the app, or at least very delayed responses. If I try to control them with an Alexa echo, she tones as if the command went through (as opposed to stating the device is unresponsive), but nothing changes until a couple minutes later…as with the app, if it functions.
When I manually turn on/off switches and check their status in the IDE, they change upon refresh, after a short (a few seconds) delay.
FWIW, I started seeing the issue you described right about the time that Samsung advised of the July 12th issue. Basically, I lost control of ALL of my Zwave switches via the app. My network has been essentially unchanged Zwave-wise for quite some time, so it’s unlikely any particular switch configuration would suddenly be contributing to an issue that had never occurred before.
My issue continued past when Samsung announced that they had resolved the issue, but I’m convinced that I was still suffering from it despite their indication that it was resolved. Today my network is almost back to normal, but I do have intermittent control issues. All of my switches are cloud-based, as they use DHs, so it wouldn’t surprise me that there is something going on server-side.
I’m bringing this up because I wonder if you’re suffering from cloud issues and don’t’ have local issues at all. I realize your problem has been going on for a lot longer, however, but that still doesn’t mean there isn’t a server-side issue.
Of course, this isn’t going to help you resolve the problem, unfortunately. But my thinking is that if things were running just fine for a period of time and then performance degraded when you didn’t make any changes on your side, that suggests to me that your problem may not be local.
)