Frustrated by the state of things

As a vera user seeing if the grass is greener over here I am sad. Veras need add-ons like PLEG to provide good scripting and each software update is as likely to remove useful features as add new ones. I was hoping things were better over here.

We need some competition to drive improvements in both platforms. Unfortunately we seem to have less a race to the top and more a staggering run in place.

Maybe vera ui8 or smartthings 2.1 will be good enough to drive the other.

2 Likes

Are you sure itā€™s not the size of the system? Cause Iā€™m pretty sure that size mattersā€¦ A small system may not work as well a large systemā€¦ And sometimes the large system my not work as well as a smaller system. Iā€™m think that the structure the system is in could make a difference in if the size matters as much. J/s

1 Like

Unfortunately, thereā€™s less competition today than there was a year ago. Revolv - gone, NinjaBlocks - gone, Wink - bankrupt, Staples Connect - almost dead. Vera is still limping. SmartThings is having hard time. Google and Apple are still on the sidelines. No one seems to able to crack this nut just yet. Philips did a good job with Hue, but it only covers lighting. Makes me wish they would expand into other areas. Oh wellā€¦

2 Likes

I am still trying to figure out ā€œbest practicesā€ for this stuff.

I routinely (sorry) use routines and modes and things seem to be working okay. One thing I did learn was that a routine wonā€™t fire if the mode is already set to the expected mode change. That was an interesting discovery.

Did not know about the specific time / hour-quarter hour thing though so thanks!

For now my routines either only set modes (nothing else) or do some timed actions with no mode checking.

The smartapps only do actions (with mode checks sometimes).

Iā€™ve found that custom device types can mess up your schedule as well so I use them as sparingly as possible.

Like Larry I also employ redundancy - I have for example a ā€œNight Checkā€ routine that kicks off at around 1 am that simply makes sure all the lights are off that I want to be off etc.

Sorry wanted to clarify - my routines either set a mode OR do an action (which could include checking a mode). I do NOT set a mode and do an action - I keep them separate.

Thatā€™s a lot of ā€œdonā€™tsā€ for a platform that comes offering all of those options.

2 Likes

Iā€™d have to add Presence to the marketing list.

2 Likes

I wish for a more stable SONOS implementation. Had to remove all my sonos speakers from ST after scheduling and rooms starting acting flaky.

Interestingly my presence stuff (2 old key fob sensors and 2 android phones) seems to be working okay at the moment. I have a fairly simple implementation - just an ā€œeveryone leavesā€ sets the ā€œaway modeā€ and a ā€œfirst returnā€ sets the home mode.

Especially when Iā€™ve gone against all of these rules and have had 0 issues (knock on wood). Plus I use modes every day, including for triggering SHM.

3 Likes

I think they point of my original post is clear from the back and forth happening on this thread.

One person says it doesnā€™t workā€¦
next person says it does if you do x and y
third person says I did x and y but still have issues.
forth person says I didnā€™t do x and y and I have no issues anyway

Basically the server changes are making everyoneā€™s hacks and solutions unstable also.

Currently my setup has been stable for a while. But I have zero confidence that it will remain that way. I hope this changes but it is the uncertainty that is limiting my use of this system.

3 Likes

There needs to be a ā€œHippocratic Oathā€ for iterative released software / firmware: Do no harm! ā€“ New features should never be an excuse for the introduction of bugs to old functionality or overall instability. Even bug fixes donā€™t justify new bugsā€¦

While anytime you change code there is a risk of what we call ā€œregression bugsā€, the use of modern source code control systems help minimize this risk (and permit very quick rollback in many situationsā€¦). But a key ingredient is very thorough ā€œregression testingā€ and ā€œQA Testingā€, to the level of detail appropriate for the scope of the change.

One of the problems is that SmartThings does not pre-publish the Release Notes in advance to help us in the Community who are particularly conscious of the system, be prepared to recognize the impact in detail.

Or, like many Software As Service platforms, they donā€™t offer a fully public ā€œBeta Channelā€ where releases can be pre-depoloyed regularly in advance to a voluntary subset of living-on-the-edge Customers. Some people are quite willing to run the Beta channels of Firefox and Chrome for example (and many, many other such examples out there).

Weā€™re currently planning on providing an ongoing Beta pre-release channel for SmartTiles. Itā€™s effort to maintain, but Iā€™d much rather do support calls and debugging for 100 users than several thousand or more. Ohā€¦ please donā€™t request to be invited to the SmartTiles beta yet (weā€™ll give plenty of notice for invitees when weā€™re ready).

2 Likes

In engineering, itā€™s called ā€œDonā€™t Kill the Fish.ā€ :wink:

You can make any changes you want to an automated aquarium system as long as you DKTF.

5 Likes

Image

1 Like

ā€¦a new feature to disable previously working features?

3 Likes

Used to be Iā€™d agree with you, but now, facing the reality of it, Iā€™d be more inclined to say that the introduction of bugs to old functionality is never intentional, hard to detect, and sometimes hard to fix. You can go on all you like about regression testing and all of that, but youā€™re dealing with an untestable system (separate topic, why that is). It isnā€™t possible to test a covering set of functionality with any confidence that itā€™s the right covering set. Itā€™s an NP complete problem.

By that Iā€™m referring to testing in an ST system. Asynchronous, event driven, distributed, complexā€¦

Sureā€¦ but the use of an ā€œin-the-wildā€ stream of volunteer Beta testers that are a substantial (but fractional) subset of Customers would significantly increase the probability of discovering and resolving bugs before a full rollout. Even a week of pre-release in this Beta stream would subject the system to a wide range of diversely random testing, proportionate to the sample size used.

Iā€™ve done pre and post installation testing for sensor nets with all of the above characteristics.

You canā€™t catch everything, but you can catch a lot. Use case-based regression testing would catch a number of the areas that weā€™ve seen come through from ST in the last three months.

So I canā€™t give them a pass on that basis.

Take the oauth issue for the UK. Universal, reproducible, obvious, for an advertised feature.

If they launched not knowing the problem existed, thatā€™s one testing philosophy.

If they launched knowing it existed, thatā€™s another.

Either way, they killed the fish.

There is zero confidence in your statement that it ā€œwould significantly increase the probabilityā€, mathematically. If the probability is very low to begin with (due to complexity), then throwing more testing at it isnā€™t going to move the needle. Itā€™s the same dilemma the company faces in the first place, how much testing is ā€œworth itā€. Obviously, they could go bankrupt testing and still not have come close to having confidence that it all works. Tough management call, to say the least.

So you spend however much it is on testing, you deploy when you reach some level of confidence, and then a storm descends on you from things you missed. Fun. What do you change the next time? More testing? Everyone here thinks they know the answer ā€“ they donā€™t.

1 Like

I totally disagree that some in-the-wild Beta testing couldnā€™t make a big difference.

As @JDRoberts points out (and I agree), a significant number of the bugs we discover post-release are ones that were, well, extremely obvious and easy to discover post-release.

It only took a few days before this bug was discovered. It could have been fixed before the release instead of 23 days ā€¦ and counting.

I didnā€™t say that it ā€œcouldnā€™t make a big differenceā€. All I said was that mathematically there is no way to know if you are right or wrong. You may well be right. The problem that ST management faces is still the one I describe: Itā€™s a judgment call whether or not to do something like a beta testing program. You have your opinion about it, and evidently someone in ST management has a different opinion.

It is impossible to say whoā€™s right, you or ST. But, we donā€™t have to, do we? :grinning: