SmartThings Community

Scheduled jobs failing (again) (again 😥) (Ongoing Known Issue)

(Brad Langhorst) #107

My rule to turn on my garden lights at 7 PM tonight didn’t work - it has failed a couple of times in the last few days now.

(John) #108

Well the app can’t load (times out, gives unexpected error). This preceded the last catastrophic schedule failure. Hold on, looks like we might be in for another bumpy ride.

Trouble with RBoy Lock Smart App
(Kevin) #109

The trip from my office through the living room to the kitchen… every trip through tonight has resulted in failed lights turning on or off with every trip. Think thats about 12 or so attempts of an automation working right. 0 for 12. I guess thats acceptable. Yes I’m Mr PassiveAgressive. Just getting tired of wondering if this is going to work or not. I wonder if this is considered OK by the upper management now? We hear from a few people in the trenches and I appreciate that as I am a IT pro myself. But the status page has been ‘yellow’ for what, a month now. Even if it is complicated, a known bug shouldn’t be that long lived.

This all just really disappoints me

1 Like
(James) #110

Yep… I haven’t been able to access my home location via the app all day. :fearful: Just times out with “Unexpected Error” and options to Logout (like that will help :wink: ) or Retry. Really it’s been coin flip for the last week or so. I have quite a few “devices” I’ve made for my home lighting system, so I imagine I’m well above “average” in that regard. But still, I’ve never been locked out like this.

1 Like
#111

Somehow everything managed to fail last night. My wife isn’t home and is being reported as present, I am home, but being reported as not. 1 of 4 outside lights shut off last night, the alarm never armed, and the door sensors are in various states of who knows what.

Everything was working too smoothly and I was almost tricked into thinking they had finally scaled up to the level they convinced everyone they were already at. Very disappointed in the whole platform.

(JJG) #112

The Cassandra nightmare continue…

My download routines (dumping about 450 history events every 5 days) failed again today.
Out of the 11 temperature sensors my routine should dump, I can only dump 2 “by hand” from the IDE before getting the dreaded :
"java.lang.RuntimeException: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.ReadTimeoutException: Cassandra timeout during read query at consistency LOCAL_ONE (1 responses were required but only 0 replica responded) @ line 112"
or sometimes :
"java.lang.reflect.UndeclaredThrowableException @ line 112"

As explained before in the other Cassandra timeout during read query thread, this always happens during the history database access :
--------------------- abstract from “JJ’s Test2” smartapp ---------------------
111 for (tempSens in temperatureSensors) {
112 def allEvents = tempSens.eventsBetween(startDate, endDate, [max:2000]).findAll{it.name == “temperature”}
--------------------- abstract from “JJ’s Test2” smartapp ---------------------

This is especially irritating since this dump routine is critical for me : it is the only way I have found to overcome the limitation to 7 days of history imposed by SmartThings.
So now SmartThings, or somebody at SmartThings, makes my workaround ineffective.
Reminds me of some SF novel by Frank Herbert (Whipping Star) where a whole “Bureau of Sabotage” was dedicated to prevent the administration of becoming too efficient and too complacent.
SmartThings should tone down its own “Bureau of Sabotage” !
Anyway, any geek naming a mission critical service “Cassandra” should have been preventively sacked for attempting to sabotage the future !

1 Like
(ActionTiles.com co-founder Terry @ActionTiles; GitHub: @cosmicpuppy) #113

Why don’t you write the events to a database as they happen?

i.e., Subscribe to all the temperature sensors and call a REST-API to write to a database like Firebase or use InitalState or dozens of similar services?

Not saying that SmartThings shouldn’t get Event History queries to work, but that you could avoid the issues with it.

(JJG) #114

Well, I do not need real time download (I integrate the downloaded text files by hand from emails into Excel), and I am NOT sure I would have any less problems that way : any SmartApp I schedule has about 20% chances of not properly executing due to some SmartThings cloud problem (my statistics over the last year), and losing 90 events due to a trickle of incidents over 5 days would not be as visible as losing 450 once every 5 days.
At least, the way I proceed, I see when I do not get what I should.

And anyway I do not want to start anew and develop a completely new integration solution, after spending weeks developing the current one, which worked more or less (exempting random missed schedules) until SmartThings sabotaged me (again… :frowning: ).

(ActionTiles.com co-founder Terry @ActionTiles; GitHub: @cosmicpuppy) #115

An Event driven SmartApp (ie, subscription to Device Attribute Events) is much less likely to fail than a Scheduled SmartApp in our experience.

But I understand it’s a bother to rewrite existing functionality.

I’m posting this though as a recommended approach for other folks who are considering data collection… ie, don’t rely on Event History: SmartThings may arbitrarily truncate it at anytime.

1 Like
#116

My routines don’t work too :frowning: Is it common problem? If it is common ST staff has to inform all customers by email. Many people doesn’t know this forum.

(Realy Living Dream) #117

They do, you just have to sign up for notifications. http://status.smartthings.com/

1 Like
(3one4) #118

I use Smart Rules as redundancy just in case ST automations fail .

(Jeffrey) #119

My schedules seem to be firing pretty well this week from Core. App also seems more back to normal more or less. A bit of slowness here and there and IDE is a bit slow but almost back to what I expect

#120

My Motion sensor event worked well yesterday, but sunset/sunrise automations don’t work :frowning:

(Realy Living Dream) #121

Sunset sunrise have been hit or miss for about a week. Turn off at sunrise seems to have more issues ( failures) than turn on at sunset.
Power limit off after x minutes is not working ( for me ) either.

1 Like
(Eric) #122

I don’t have an attention span long enough to open the app.

1 Like
(Eric) #123

:sob:

:dizzy_face:

Should I come back?
(JJG) #124

For the third consecutive week, my periodic history download SmartApp failed to execute.
The only difference is that this time, running it by hand from the IDE, I got an “error” notification in the log :

java.util.concurrent.TimeoutException: Execution time exceeded 20 app execution seconds: 122413839485071 @ line 112

which resulted in the usual IDE return code :

java.lang.reflect.UndeclaredThrowableException @ line 112

I wonder if somehow, the way I wrote my lines 111-112 in my “JJ’s Test2” SmartApp does not result in parallel threads competiting for the same DB access, and when not getting through, then going into time-out.

111    for (tempSens in temperatureSensors) {
112        def allEvents = tempSens.eventsBetween(startDate, endDate, [max:2000]).findAll{it.name == "temperature"}

But if it is the case, how can I modify my code to force serial execution and avoid this concurrent.TimeoutException ?
Sorry if it is a dumb question, but Java/Groovy is not my forte… :frowning:

But obviously, this yet does not explain why the same code ran without this problem for 2 years, and systematically fails to execute since 3 weeks; somebody at SmartThings must have done (or not done) something…

Cassandra timeout during read query?
(ActionTiles.com co-founder Terry @ActionTiles; GitHub: @cosmicpuppy) #125

As far as I know, “eventsBetween()” is just a very expensive function that runs long in “real time” unless the SmartThings database is running optimally. So this is likely a temporary problem.

I don’t think it has anything to do with concurrently or anything you can control … unless you write a convoluted SmartApp that only fetches smaller batches of Events, each with a new SmartApp invocation (using runIn or some other loop callback).

1 Like
(JJG) #126

Well, the java.util.concurrent package is clearly used by either Groovy and/or SmartApps, since it generates an error message !.
The question is only, does this multithreaded execution happens at the lowest DB access level (in that case, I cannot do much, and I agree with you), or is it somehow happening at a higher level, meaning some different Groovy code writing, with the same semantic, could overcome the timeout.
This happens frequently at SQL level, so I suppose it could happen too when accessing a SQL (supposedly) cloud DB through Groovy and the SmartThings cloud engine.

1 Like