I started evaluation of SmartThings on Feb 1st to assess how reliable the platform is on all fronts. I took the effort to observe and test each and every feature that is installed in my home. The idea was to see how often did I notice something failing. Some of the failures aligned with the outages and platform upgrades etc. While some just had no explanation, most of the things aligned with the outages and platform upgrades etc.
The reliability shown here may not be the true % Reliability, but gives a good idea of how the platform performs. Over all I think SmartThings platform worked pretty well for me. I had to add some redundancies using Rule Machine and I am sure that helped too. I also left some routines without any redundancies and the evaluation is based off those.
I spent 15-20 minutes everyday evaluation each automation and also did a general evaluation based on daily usage and observation. I would make notes on my phone and then update the excel sheet each night. Not sure how many folks will agree with this evaluation. Or do you think your perceived experience is better/worse?