Performance Tracker (Final)

At SDC I confirmed ST does in fact support ZLL, however their existing bulb fingerprinting only works with ZHA 0104. However, it is possible to pair via ZLL if you use a custom built device type.

2 Likes

I was just thinking that I cheated last week. I should have opened a ticket when the light came off at 6:30 instead of 8:00 and see how long would have taken to get it resolved. A regular consumer would not have looked at app state in ide, figure out that the instance is currupt, deleted and recreated the instance…oh well…:slight_smile:

2 Likes

I don’t see it on the ZLL certifications list from the zigbee alliance.

@sticks18 might know more.

I am going back and forth with support right now on several such tickets.

Part of it is I don’t want to fix it for them. The other part is I don’t have time with travel.
I won’t post the responses verbatim, but will PM them if @slagle or anyone else wants to see. But one person asked me “Is this perhaps related to the recent stability issues?”

Well, heck… I don’t know… I am asking smartthings. Why would support put it back on me to make wild conclusions about cause…

The other issue is, after they applied some “updates” whatever those were and the problems persisted they said just delete and rebuild the rules. Well, I haven’t done that yet - because of time and also because I feel like there isn’t any reason to say its a corrupt rule and if there is they should state why. I think it’s just a stab in the dark. Hoping and praying it goes away and they don’t have to truely understand what the cause is, etc. Is there evidence of SL rules being corrupted? How wide spread is this? What is the cause? Has ST considered developing a methodology to hunt for such corrupted rules? So they can proactively identify them?

So I asked - should I delete all my smartlighting rules and rebuild / should all ST customer delete al their SL rules and rebuild? etc. Of course, the answer received back is - no, just those ones I know there are problems with. The reality is I don’t know which ones are working and which aren’t. The problems are intermittent and I can’t monitor all of them. Only the ones that impact me in a noticeable way - such as my bed fan turning off. I won’t know if my outdoor landscaping lights turn off early or late because I don’t watch them at 2 am. The expectation is I put in the rules properly and the system would work properly.

1 Like

News to me. I know some bulbs use ZLL profile in fingerprinting and we can send commands that are part of the ZLL extended set, but I had no idea it’s possible for a device to actually see the hub as a ZLL coordinator. My understanding was that the ZLL devices would just fall back to ZHA to maintain compatibility.

1 Like

Updated 4 weeks performance tracker results. Is SmartThings’ reliability at 87% , 93% or 97%?

I’m going with negative 43

Did you read my updated first post? :slight_smile:

Duh, of course I didn’t.

Great job. I don’t see how you did it.
I got in the shower this morning, my motion sense was working, it was showing in the app, the rule showed true, but neither the fans or the lights would come on… I was pissed!

So leaving aside Life360, everything failed on exactly the same day at exactly the same time?

Otherwise, if the keen vent routine failed on Tuesday and the sensor lighting failed on Wednesday, the system failed on Tuesday and on Wednesday.

I’m just not sure I’m following the report since it appears to be reporting by device rather than by day.

Thats a good point! Luckily I’ve had only 4 days (including life 360) and each category failed separately. Otherwise it would have been difficult to assess the overall based on records.

Four days with failures out of 30 is 86.6% reliability. I didn’t see that number in the report except on the Life 360line, unless I just missed it.

Since Life360 is an officially supported advertised integration, I think it counts as a system failure.

I guess I was mostly confused by this:

The ST presence app has had 4 days with at least one incident (not caused by the Life 360 service), which triggered a lot of angst among the family members - making routines to run and modes to change unexpectedly. Other categories with issues were related to unannounced changes in device handlers and occasional server hiccups.

If I were to remove the Life 360 app, the overall performance of my system was …93% (or 2 days with issues).

If the presence app had 4 days with errors not caused by life 360, where is that number showing up? Or am I just misreading that sentence?

By the way, none of this Little detail stuff is to take away from the value of the information you provided, and I’m sure it was a lot of work. You’re sharing it with the community is very much appreciated. :sunglasses:

2 Likes

Like a good challenger, @JDRoberts is there! Love it…

You don’t see the overall number anywhere, because it wasn’t my intent to produce one, but rather to assess the strengths and weaknesses.

As I was updating my post I realized that with exception of Life 360 failures, I really only two distinct days when all incidents happened…(well now that you made me think, there were actually 3 distinct days :slight_smile: ) So you’re right again. I removed the 93% blurb…

2 Likes

This was very interesting. 86% up time is considered unacceptable in today’s world by an measuring stick. While it’s a major improvement from what the %'s probably looked like 2 months ago, still work needs to be done. As long as we continue to see improvements towards a 99.9% uptime which is what most org’s strive to be at.

1 Like

99.9%!? … not sure what I would do on my nights and weekends if I did not have to spend time anymore to do troubleshooting and fix my setup. Life would be really boring! :confused:

5 Likes

Here are some suggestions for you:

OR you can do what @bamarayne likes most :slight_smile: …start rebuilding your system…

3 Likes

Ask Alexa - done! Kids love the presence report…
Designing new rules engine - following with lots of interest! Doing some testing. @ady624 is an awesome coder who also did the ATT Digital Life integration - something I had been waiting for more than 2 years!
Rebuilding System - Dont mention this… WAF (or SAF!) will go down the drain

3 Likes

Speaking of kids, we already planned building a robot this weekend with my 4 year old that will have magical power. So when Alexa tells the robot to turn the lights off, it will do it…

2 Likes

You’re right, 86% is pretty bad, but it is a hell of improvement. what I hope it will disappear soon, are the incidents that require significant digging around for a resolution. I can deal with occasional hiccups and lag, but even one incident per month like the change in device handler is too much.

2 Likes