Scheduled jobs failing (again) (again 😥) (Ongoing Known Issue)

I figured as much. Wanted to mention it because throughout these new issues I’ve seen things break then go away then break again. I originally submitted a ticket back when these issues started.

1 Like

Just adding a “me too” to this thread. Routines and CoRE pistons randomly failing over the last few days, especially “I’m Back”. Just like what’s posted, the app sees I am home but the I’m Back routine doesn’t run so setting off intrusion alarms as well.

App is super slow, wife is getting feed up.

This morning around 9:00 AM CDT a significant part of the mitigation work was enabled. Since that time there haven’t been any event read time outs:

Write timeouts are also way down - though the change today was targeted at reads. Write timeouts are down because of the overall decreased pressure on the database with the read change. There were a total of 20 timeouts since the change went out.

We’re currently running in a mode where if any major issues happened with mitigation we could quickly revert back. After ops is confident that the fix is stable for the long term the final mitigation for writes will be deployed as well.

I would expect people to notice an overall improvement in successful SmartApp/DTH executions and a snappier IDE/Mobile App when accessing devices & looking at previous events.

If you are still seeing frequent failures please open up a support ticket and/or post in this thread.

@Jason_Brown if you are willing to provide some more details it would be greatly appreciated - were you seeing issues today after 09:00CDT? What specifically was slow & for how long (is it still slow?). Feel free to DM me your username - I may not get back to you directly but will take a look to see if anything stands out. Another user brought up continued sluggishness in the app in the dev slack earlier.


Tonight my mobile app is back to being reasonably responsive.

Pfew, my mobile phone did not explode. Maybe it’s because it’s not a Samsung Note 7?! :wink: checking app speed

Much much faster, thank you!

Average (15 min window) execution time for all my pistons:

My outdoor lights turn on at around 6:30pm (sunset) and the color changes every 30s for 9 distinct bulbs/strips. I guess a 2.5s average execution is okay/acceptable, though less than 1s would be nicer :wink:

EDIT: this is the morning timeframe, up until sunrise, lights taking around 5s on average. Definitely an improvement in execution time too:

And the best part, nothing exploded :wink: great job!

1 Like

I still have my note 7… I’m waiting patiently for it to explode… daddy needs to get paid!

1 Like

Thanks goes to @peter & @rappleg!

1 Like

Thank you Black Ops team and @rappleg :slight_smile:


I just manually ran my goodnight routine, never seen it respond that quick. It was almost instant from pushing the button to lights off.


One of my lights took a minute to respond.

The app is faster though.

1 Like

The app is definitely faster and seems my routines processed a bit faster this morning too.

1 Like

Things are very snappy on this end. Response times like this actually remind me just how awesome SmartThings could be. Great work guys.

1 Like

I still can’t rebuild my pistons in CoRE. @ady624 - Am I the only one?

c0ddeb9a-7e1c-4d64-9b6e-739042b53ead 1:50:40 PM: error java.util.concurrent.TimeoutException: Execution time exceeded 20 app execution seconds: 166274079983119 @ line 3467

The app doesn’t seem any faster.

It took the SmartApps page 16 seconds to load for example.

Uh-oh :frowning: :frowning:

To be more transparent and clear, this is a complex piston, but it rebuilt fine prior to the recent issues.

It is 100% repeatable. Every time it fails. This happens on a bunch of other pistons also.

@ady624 - Another question, what is a reasonable memory usage for CoRE? Mine sits between 25% and 45%.

BTW - I noticed a CoRE update. Same issue, new line number.

c0ddeb9a-7e1c-4d64-9b6e-739042b53ead 2:01:51 PM: error java.util.concurrent.TimeoutException: Execution time exceeded 20 app execution seconds: 166945835485021 @ line 3468

Each SmartApp gets about 100kB of state space. 45% means it’s using 45kB of data. All is good as long as it doesn’t get too close to 100%.

You may be suffering from read timeouts, a problem @vlad knows about - ST has implemented some changes yesterday to help with that, maybe Vlad knows more?

Try the new update and let me know if things improve please.

I updated prior to my last post. Still didn’t work…

BTW - I am curious about the numbers you posted. None of my execution times are anywhere near yours. Mine are way less.

But I can also tell you it took 18 seconds to open the screen below from the previous screen.

1 Like

Trust me, update again :smiley: mwahahaa