I would say, the consumer would have returned the product to Best Buy, but I said that I am allowing for three strikes before I move on, that was the first…
2 Likes
I wonder if Smartthings undertakes any market research in this regard.
I would love to see a group of 10 or so average joe consumers using an average deployment of ST and see how that plays out.
And no, not to see the meltdowns, but to understand the issues they encounter and any breaking points as average consumers.
1 Like
ST is trying to offer a plug and play experience, but obviously they are not there yet. I am starting to wonder if my little experiment is a little premature, @alex did say, the improvements will be come in the upcoming ‘days, weeks and months’.
2 Likes
I don’t think so. Shows what issues still yet exist, what needs to be overcome to make this average consumer ready. I can’t really ask for much more from ST than to have a truly consumer ready setup - then the issues that remain as a result of me trying to over-leverage the system are mine to own and deal with. If we could get there, ST would be the first eco system to do so and I would be very happy.
I love that it also documents the very real issues, that so many seem to forget they even have or encounter and believe they have a fully functional system. Cognitive dissonance is a real issue for many here that are technically adept and really love the tinkering part of the hobby.
2 Likes
That’s what got me started on this little project. One big advantage for ST, is that it gives you some tools to make your system work, even when things get broke. With Wink, Nest, Amazon and Harmony, you are at the mercy of their support to fix the problems, and often times, that never happens. I’ve been in the situations where updates were deployed and existing set up stopped working. The only way to get going again, is either wait for a fix or rethink your HA automations, both of which, often take more than four hours. The key here is, how many times things break with Wink, for example vs. ST. This is the goal of my project. Say good bye, or embrace STs wonkiness…
2 Likes
I shared this with Alex and some other leadsrship at SmartThings.
1 Like
I really appreciate this deep engagement and, as @slagle mentioned, we are looking at your issues directly.
2 Likes
Right, but don’t you have a line that says "zigbee devices (mostly ZLL) "?
That’s the one that I’m suggesting should be broken into two lines, one for ZHA connected directly to the SmartThings hub and one for ZLL (probably connected to a Hue bridge).
Excellent point. So truthfully, reliability needs to be x days without a failure (like organizations that say x days without a lost day of work/incident/work-related injury).
I like tinkering on my terms, but not because it didn’t work the way it was suppose to. Also, I could care less if 1 automation failed out of the 25 that were scheduled - failure is failure when it’s the mainstay. Perception is reality - if my car starts 29 times each month, but fails that one time - I’m still irritated with it (better call Tesla). I would expect and accept more failures with ST, but simply put, if something fails every day, the perceived reliability is low regardless of how much actually worked. If someone says that it’s 90% reliable. Who cares - it also means it’s 10% unreliable. It needs to work 99.9% of the time so that the truly random failure is a fluke and not an artifact of the platform. Things fail and I’m OK with that. It just needs to be a fluke and not a reoccurring event.
5 Likes
Oops, I left the headers off on the device section the ‘mostly zll’ note, is for Wink controller and the N/A means I don’t have any zigbee devices connected to Wink. So if I split into two lines, I would have N/A on both Wink and ST because I don’t have any devices to monitor in either controller. I am just going to put Zigbee ZHA and leave ZLL out altogether. Good point…
2 Likes
I’ve updated the pics to reflect your suggestions Thanks…
2 Likes
Week 3 Update: Maintenance free week! No major issues to report. A minor issue was observed due to a bug in Smart Lighting. Changing the schedule of an instance from 6:30 AM to 8:00 AM made the automation fire at 6:30 AM & 8:00 AM. I spent less than 30 min troubleshooting; deleting and recreating the Smart Lighting instance fixed the problem.
@JDRoberts …I’ve added reliability rate based on days (see updated table on first post 
)
4 Likes
At SDC I confirmed ST does in fact support ZLL, however their existing bulb fingerprinting only works with ZHA 0104. However, it is possible to pair via ZLL if you use a custom built device type.
2 Likes
I was just thinking that I cheated last week. I should have opened a ticket when the light came off at 6:30 instead of 8:00 and see how long would have taken to get it resolved. A regular consumer would not have looked at app state in ide, figure out that the instance is currupt, deleted and recreated the instance…oh well…
2 Likes
I am going back and forth with support right now on several such tickets.
Part of it is I don’t want to fix it for them. The other part is I don’t have time with travel.
I won’t post the responses verbatim, but will PM them if @slagle or anyone else wants to see. But one person asked me “Is this perhaps related to the recent stability issues?”
Well, heck… I don’t know… I am asking smartthings. Why would support put it back on me to make wild conclusions about cause…
The other issue is, after they applied some “updates” whatever those were and the problems persisted they said just delete and rebuild the rules. Well, I haven’t done that yet - because of time and also because I feel like there isn’t any reason to say its a corrupt rule and if there is they should state why. I think it’s just a stab in the dark. Hoping and praying it goes away and they don’t have to truely understand what the cause is, etc. Is there evidence of SL rules being corrupted? How wide spread is this? What is the cause? Has ST considered developing a methodology to hunt for such corrupted rules? So they can proactively identify them?
So I asked - should I delete all my smartlighting rules and rebuild / should all ST customer delete al their SL rules and rebuild? etc. Of course, the answer received back is - no, just those ones I know there are problems with. The reality is I don’t know which ones are working and which aren’t. The problems are intermittent and I can’t monitor all of them. Only the ones that impact me in a noticeable way - such as my bed fan turning off. I won’t know if my outdoor landscaping lights turn off early or late because I don’t watch them at 2 am. The expectation is I put in the rules properly and the system would work properly.
1 Like
News to me. I know some bulbs use ZLL profile in fingerprinting and we can send commands that are part of the ZLL extended set, but I had no idea it’s possible for a device to actually see the hub as a ZLL coordinator. My understanding was that the ZLL devices would just fall back to ZHA to maintain compatibility.
1 Like
Updated 4 weeks performance tracker results. Is SmartThings’ reliability at 87% , 93% or 97%?
I’m going with negative 43
Did you read my updated first post?