New scheduler (Codename: Ticker) being rolled out

ALL THINGS OPERATIONAL

Imagine that! Was a lesson learned? It sure doesn’t seem to be that way.

Automatons aren’t failing. They just aren’t firing at all! I guess if they don’t fire, than they can’t possibly fail. Which means you can post this for the world to see. …

I wonder if the root cause for this one is the same as the melt down in march. You remember, that’s the one that was caused by. … what was it again. …oh yeah, it was. … wait. … ummmm… yeah. … I’m pretty sure we were told what it was. … right. … we were told. … chirp, chirp, chirp.

2 Likes

Had issues this morning. 2 Hue Bulbs turned on at 5:25am like they were supposed to. At 6:00 they are supposed turn off… This morning they didn’t turn off. Had to manually turn them off. All local automations ran as scheduled.

My morning rule to turn off the alarm didn’t run this morning. Had been running perfect since I created it a week or so ago.

Same here…

Well taking a look at the amazing new notifications/activities log in the app, of command are being sent, but nothing is turning off.

This is affecting all of my smart apps equally, and it’s not just pending off.

Smart lighting had a simple sweeper tile to turn of devices in the morning. It failed for the second day in a row.

Also failed to change modes twice last night.

Don’t think this is ticker, deck lights didn’t turn off, delay off fired, but didn’t actually execute.

Handler	Scheduled Time	Actual Time	Delay (msec)	Execution (msec)
delayRuleFalse	2016-04-26 5:08:21 AM PDT	2016-04-26 5:08:37.464 AM PDT	16464	93440

16 second delay, 93 second execution?

2 Likes

Hey, they thought you were too happy last week with your system, so they sent you a reminder that we are not out of the woods, just yet. Or as @JDRoberts put it, some may forget that reliabily is still a big issue…

2 Likes

Ha, ha, ha…is Amazon 's AWS wishing Robert good luck in his new role…

4 Likes

Yeah, they have to make to not show favoritism lol.

@JDRoberts is actually incorrect. Reliability is not an issue. Our perception of reliability is.

The ST system is consistent that it reliably fails in some way on a daily basis. It is extremely stable in it’s instability.

Everyone gets a trophy, remember.

1 Like

just reviewing job history for other scheduled events, the delays and execution times are all over the map.
Delays ranging from 14ms to 50s and exec times from 170ms to 40s, and that was just one app…

Looking deeper here - it does look like ticker did its thing but i don’t see the unlock called anywhere so some problem downstream I guess

			    stopHandler
    			2016-04-26 7:15:00 AM EDT
    			2016-04-26 7:15:03.886 AM EDT
    			3886
    			133
    		
    			startHandler
    			2016-04-26 7:10:00 AM EDT
    			2016-04-26 7:10:03.359 AM EDT
    			3359
    			140766

Execution time of 140766msec is about 10x longer than any others.

The worst part is, whatever is happening/happened it’s affecting both runIn() and [delay: msecs]

.#AllSystemsOperational

2 Likes

Same here. After weeks of flawless behavior, my outside lights didn’t go on at sunset yesterday.

We discovered an issue that was impacting some schedules and have corrected it. If you are currently having issues with schedules, PM me or send a note to support@smartthings.com.

3 Likes

@jody.albritton - when specifically was it corrected? I assume we should only send notes to support for any issues after that time.

Thanks.

I got a note from one of the engineers at 10:15 AM PST on 4/26. If you are having scheduling issues after this, please open a ticket or message me.

1 Like

Thanks @jody.albritton. However, if you got a note about this, then clearly you guys were aware there was an issue yet nothing was posted to status.smartthings.com nor via email.

2 Likes

I can confirm the fix worked for me for my rule that had a delayed power off after 5 mins… kept failing over and over today… just tested and it worked. I turn on a virtual switch after a door is opened and then do a delayed turn off (5 mins) as part of the same actions and it worked. Thank you! :slight_smile:

I had seen some isolated reports of schedules failing starting late last night, but this issue was not impacting everyone. It was only after reports increased about an hour ago that we discovered the issue and steps were taken to correct the scheduler health. The status page will only change when there are platform wide outages/service disruptions. I realize that this leaves some gaps in information dissemination and we are working on our processes to address that.

I highly recommend using something like Facebook or Twitter for smaller issues that are still under investigation. It’s quick and easy and can reach many followers.

1 Like