My ST 7" tablet integration


(Glen King) #1

I’m documenting the basics here in case others wish to use ideas from it. Most of those ideas of course came from others who have done similar projects; can’t name you all, but thank you.

So my integration seems to have centered on a cheap 7" android tablet. It started there hoping to use bluetooth as a virtual presence sensor/detector for multiple devices (phones, pod, etc). That works, but not consistently. Will devote more energy to that in the future. But I also imagined using its camera and screen, and those things are shaping up nicely.

In particular, the camera. I’m using IP Webcam Pro to capture 15 seconds of video any time there is motion, and the motion sensor output from that program is also triggering, via the Secure Settings plugin for Tasker, screen wake up on the android tablet.

Right now I’m using Sharptools widgets as my control panel. Might eventually migrate to Smarttiles or something, but for now this is working.

Additionally I’m using OK Google for voice command, with the tablet always listening. Aside from occasionally having to retrain the OK Google phrase, that works very well. It turns lights on/off, locks and unlocks my front door, etc.

I also have Tasker, via AutoNotifications, using tablet notifications to trigger all sorts of things. It sends emails to notify of events, it turns on lights when certain friends call ahead, it can also unlock my door that way, etc.

Will add to this thread as I progress.


(Joshua Lyon (SharpTools.io Dashboard)) #2

Very cool! Thanks for sharing! It’s pretty amazing to see what people come up with and how they use community developed apps like SharpTools!

If you have some pictures or video clips, I would love to see them!


(Glen King) #3

So now that I’ve outlined what I’m doing, here are some of the challenges… and possible approaches for solving them.

  1. The bluetooth virtual presence either works not well enough, or too well. By that I mean it sometimes does not acquire an approaching presence soon enough to unlock the door… or it re-acquires existing presences, and unlocks the door sporadically with no real cause.

The possible solutions to this are multiple. First clearly is going to involve restricted timeframes in Tasker, to at least get it under control. Beyond that I’m going to have to play with variables. Dunno yet how I’m going to structure this, but the general idea is: %ipodnear=0 if that device has not been seen within the past many searches. So that if it is seen, bluetoothnear triggers the door unlock and %ipodnear gets set to =1… And if %ipodnear>0, then bluetoothnear acquiring the ipod on the next search results in no action.

The related problem is that iPods and phones get turned off while being present. lol

  1. GoogleNow comes to the fore, and stays there, when you use it.

I guess I want Smartthings to return to the fore within a few seconds… or I want the home screen, with the ST widgets, to return to the fore. I need to think that through a bit. Some sort of instruction that says “if no activity after X time, return to home screen” or something.

Of course there are also unanticipated gains. For example: using the android camera as a motion sensor allows me to take the ST motion sensor away, and use it in a different room.


(Joshua Lyon (SharpTools.io Dashboard)) #4

This may be a configuration option within AutoVoice as my Google Now commands show up temporarily and then disappear after they are processed.

It sounds like you are on the right track with using variables to track this. I would consider using variables with uppercase letters to make them global, so they can be used across tasks and profiles. The iPods being turned off is particularly tricky - I would be interested to hear what you come up with.


(Glen King) #5

As for the latter issue, turns out to be easy. Once you start thinking, that is lol

I have MotionDetector sending intent to Tasker to keep the screen bright. So if the screen dims, it means no one is using it… So at that point, the Go Home command takes me to the home screen thereby sending any Google windows to the background. Let android manage the ‘dead’ background apps, which it supposedly does very well.

Still thinking on the virtual presence issue. Things might improve somewhat now by the fact that the CPU is always on, never sleeping.


(Glen King) #6

I’m temporarily suspending the efforts to have presence sensors open locks and lights on arrival. The ST presence sensor doesn’t trigger routines either at this time, so I figure I’ll wait until platform stability and improvements come along before continuing the bluetooth effort with phones, iPods etc

The tablet itself is coming along ok. Here’s a list of what it has and does so far:

  • tools being used are Tasker, Sharptools, MotionDetector, AutoApps, ScreenFilter, googleNow, hangouts, SendSilentMail, BluetoothAutoConnect

  • screen brightens and dims via MotionDetector, which works via the front camera using Intents. An app called Screen Filter dims it almost to total dark.
    Bonus, it functions as a motion- sensing nightlight!

  • that camera can take single still shots using MotionDetector, receiving Intents triggered by ST events (such as a window sensor opening) combined with its own motion detection. I was using IPwebcamPro, but the vids were too blurry

  • Those images are instantly uploaded to cloud storage. Cloud image access is not directly integrated into ST, but iMO it’s not at all important that it be so.

  • ST access secured via Tasker app lock

  • returns to home screen when screen goes dim, sending GoogleNow windows into background

  • intercepts OK Google commands to turn on lights, unlock doors, etc in ST

  • speaks all notifications and alerts out loud

  • sends emails via SendSilentMail for any ST intrusion detections to my work email, using smtp.mail.yahoo.com

  • Control Center using ST icons and widget on the home screen for now.

  • using Hangouts Dialer, alerts and notifications can dial particular phone #s.

  • tells news and weather onscreen and audibly

  • can be used as a speakerphone using hangouts, for both incoming and outgoing calls


(Glen King) #7

Improvements to the above will be:

  • have any images recorded during the above process instantly and automatically emailed to my work email etc, so that I could know right away whether the person in the image belongs in the house and act accordingly.
    Presumably, the date string variables would allow that to be easily automated.

  • auto-activate a siren sound, and play it on a bluetooth-connected speaker, in cases of intrusion. I will of course wait until all the kinks are worked out prior to this; last thing I need is a siren blasting in the night if there is no intruder.

  • Eventually, some “control panel” user interface that allows access to a bunch of varied scenes. Not important yet, as everything currently being done fits into the 15 available page icon slots plus four available system tray icon slots (I’m actually only using ten of those nineteen so far).

Sooo… what functions do you all consider crucial for such a tablet implementation?


(Glen King) #8

Just to continue logging progress and issues:

  • using autoinput, the tablet can reliably make automated outgoing calls via hangouts. So it sends sms alerts, email alerts, and phone alerts.
  • MotionDetector usually merely lights up the screen when someone passes by. But using Autoinput, it can conditionally (such as if an intrusion has been detected) switch to taking pictures too.
  • MotionDetector ‘extra’ intents don’t work. Have written app author multiple times, got no response.

Next step: get device handler for Schlage lock so I can use its built-in alarm, rather than a bluetooth speaker, as the alarm. I’ll want to include an onscreen routine where I can enter a code and thereby prevent the alarm from going off, or stop it if it has gone off.


(Glen King) #9

Found out (thanx 2 this forum) that the Schlage lock alarm cannot be triggered from external software. Bummer.

Yesterday I stumbled on a BT speaker called 808 Thump. For $20, it’s pretty good. So I set it up to do two things: a) play a siren-alarm sound if a particular Tasker context is met (such as a security alert), and b) function as the general audio output for the tablet. The speaker times out and shuts off if it receives no audio program material for 15 minutes, so I have Tasker sending it a faint beep every 12 minutes as a keep-alive.

Sort of a “poor man’s Echo”, with OK GOOGLE playing the role of Alexa. Once EchoDot becomes generally available I’ll likely get one or two, just because the tablet’s mic hears less well than I’d like. But for now this is working great.


(Glen King) #10

Just tracking status again. SmartTiles is installed, it is awesome. I’m configuring the first page with device controls and door/window closed statuses. A second page will have temperature and battery statuses. Another page will have SHM, modes, and routines.

I’m going to do some sensor substitutions. When I first got ST, of course the multi sensors went on front and back doors. But I can accomplish the same thing with the $11.50 Iris contact sensors, so those will be moved tonight. The ST multi sensors will go on the garage door and washing machine. I will set up a voice alert to speak “your laundry wash cycle is complete” thru the bluetooth speaker when the vibration stops on the washer - I can’t wait to see my wife’s face when that first happens :slight_smile:

I’m also ordering the go control package, and using the two contact sensors as moisture devices. One will be a leak sensor, the other will tell me whether the bathtub is full. The motion sensor will eventually also find a home.


(Glen King) #11

Another update. The garage door getting a sensor is big. Even if I never automate it in any other way, it’s still big. Because with Smarttiles on the control tablet by the front door, the question “is the garage open?” always can be answered accurately; never again does someone have to go out in the rain or snow, only to find the door is closed.

I also mounted my BT speaker on the ceiling. Wife does not like the mount, so I have to change it lol. Oh, and I added an Iris sensor to my doorbell.


(Glen King) #12

Alright, here’s a totally cool Tasker function: you can invert multiple holidays.
For example, in my area tomorrow is garbage and recycling pickup. So in Tasker I chose a context of State -App - Calendar Entry. I listed the three holidays that can occur on days other than Monday or Thursday, separated with semicolons: Christmas Day; New Years Day; Independence Day. The calendar being referenced is Google Holiday Calendar. I also chose a Weekday context of Wednesday, and a Time context of 6:30am.

If I don’t invert the State context, none of the three contexts are active.
But if I invert the State context… right now it is active!! So it knows today is not one of those three holidays. And that state will remain active through tomorrow. And at midnight, the Wednesday date context will activate… and at 6:30am, the system will speak to remind me to put out the trash and the recycling.

I could of course also have it post an onscreen message, and/or flash lights, etc… Now we’re getting somewhere with our smart home.


(Glen King) #13

Following up on the Tasker “garbage reminder” idea, I have two things going on. First is that the tablet will remind me, at 6:30am on garbage days, WHICH garbge needs to go out (yard waste, recycling, rubbish) this morning.

I added to it: now I can ask it at any time, and it will tell me the next one. For example, yard waste is Monday morning and recycling is Wednesday morning. So if I ask on Sunday night “ok Google, next garbage day” it responds “Monday, garbage and yard waste”. If on Monday afternoon I ask again, it replies “Wednesday, garbage and recycling”.

Not bad so far.
But not yet good enough. Because Memorial Day is coming, and if I ask the question on Saturday that weekend it will say “Monday, garbage and yard waste”. Which is incorrect because there is no garbage pickup on Memorial Day! It needs to be smarter than that; the next garbage day from that Saturday is actually Wednesday.

So I’m working with the idea of future holiday handling in Tasker. 86400 is the relevant number, but I have yet to work out the actions such that they do calculations based on the Google holiday calendar.


(Glen King) #14

My irregular update (that doesn’t sound right lol):

A Tasker voice event (via OK Google) that adds items directly to my Wunderlist shopping list!
The trigger event is a State - AutoVoice Recognized commnd. The command filter is “add to grocery list”, and the action grabs whatever word(s) I say immediately afterward. The variable is %avcommnofilter.

So if I say “add to grocery list raw almonds”, the task grabs “raw almonds”. Then it sends an intent to wunderlist, and the text of that intent is of course the variable %avcommnofilter. Then it uses AutoInput to click ‘add’ on the pop up screen. If that pop up has been configured to hit my shared Shopping list (which it has), then the item gets recorded to that list… and when Wunderlist syncs, then it hits my phone and my wife’s phone and my iPad and my desktop.

So this is yet another “Alexa function” that can be built in Tasker.


[RELEASE] Ask Alexa
(Glenn Morrison) #15

Nice setup you have going there. My inner nerd is uber jealous.


(HONGTAT TAN) #16

Can you share your configuration? It would be easier for those noobs to tasker, Auto**, sharptools, etc, to modify and try out. :sweat_smile:


(Glen King) #17

Hongtat, here’s one of my profiles, one of three related to local trash collection.

Profile: Next Garbage Mon Wed Fri Am (40)
Event: AutoVoice Recognized [ Configuration:Command: "next garbage day"
Last Cmd Id Regex: false
Last Cmd Id Invert: false
Not on Normal: false
Not on Continuous: false
Contains All: false
Use Regex Replacements: false
Variable Names: yard waste, recycling, rubbish
Do Google Now Search: false ]
State: Not Calendar Entry [ Title:columbus day; labor day; memorial day; presidents day; christmas day; new years day; independence day Location:* Description:* Available:Any Calendar:Google:Holidays in United States ]
Time: From 00:00 Till 06:45
Day: Mon, Wed or Fri
Enter: AM garbage reply (42)
A1: Wait [ MS:0 Seconds:2 Minutes:0 Hours:0 Days:0 ]
A2: Say [ Text:F right now garbage and yard waste Engine:Voice:default:default Stream:3 Pitch:5 Speed:4 Respect Audio Focus:On Network:Off Continue Task Immediately:Off ] If [ %DAYW ~ monday ]
A3: Say [ Text:F right now, garbage and yard waste Engine:Voice:default:default Stream:3 Pitch:5 Speed:4 Respect Audio Focus:On Network:Off Continue Task Immediately:Off ] If [ %DAYW ~ monday ]
A4: Say [ Text:F right now, garbage and rubbish Engine:Voice:default:default Stream:3 Pitch:5 Speed:4 Respect Audio Focus:On Network:Off Continue Task Immediately:Off ] If [ %DAYW ~ friday ]

Here’s one that uses AutoNotification to announce when the wash is complete. The timeframe was included because the day after I made this profile, a lightning storm set off the announcement at 4am (not good for the wife acceptance factor lol)

Profile: Washing Machine Done (16)
Event: AutoNotification Intercept [ Configuration:Event Behaviour: true
Has Reply Action: false
Notification Text: Washing Machine is finished
Get All Fields : false
Get Internal Actions: false
Get Internal Actions Big: false ]
Time: From 07:05 Till 22:03
Enter: Say Done (22)
A1: Say [ Text:F .Laundry washing machine cycle has been completed! Engine:Voice:default:default Stream:3 Pitch:5 Speed:4 Respect Audio Focus:On Network:Off Continue Task Immediately:Off ]