A TRUE Smart Home...HAS JARVIS!

Wiki Projects & Stories
,
1

For a LONG time now I’ve been trying to make a smart home for as LITTLE money as possible. Unfortunately, that’s VERY hard to do unless you know how to make all the stuff on your own. I’m not a coder or an engineer…but I’ve got an awesome imagination and problem solving skills. Here’s how I’ve made my small two bedroom apartment…SMARTER than all the rest:

What I have:

  1. Smart Things Starter Kit (I want to add more, can’t afford it at the moment)
  2. Jarvis (LINKS Mark II at www.ai-dot.net)
  3. Philips Hue Lights (Six lights total, also want more, can’t afford it)
  4. Teamspeak (you’ll understand in a moment)
  5. Eventghost (if you’re on a mac…well…lolwut? sorry)
  6. Sentry (just a windows security app).
  7. IFTTT (www.ifttt.com)

So, the first thing I did was install and configure Jarvis with TONS of questions and answers and tasks to do on voice command. THEN, I installed my own Teamspeak 3 server and client, and (as I demonstrate in this video: https://www.youtube.com/watch?v=8otzlMDW5DY) connected Jarvis to it and installed the Teamspeak client on my phone so that I can communicate and get responses from jarvis anywhere, anytime.

Now, with Eventghost and the pushbullet plugin you can do A LOT, because without the pushbullet plugin it’s very hard to communicate from eventghost with IFTTT, which is what I need if I want Jarvis to integrate with IFTTT things.

SO…I say, “Jarvis, lights on.” Jarvis says, “Yes sir, Turning all lights on.” and creates an event in eventghost, which uses pushbullet to send a text to the IFTTT sms number with a hashtag #lightson, which then triggers almost immediately WITHOUT IFTTT’s 15 minute refresh wait-time, my philips hue lights to all turn on, all this within about five seconds or less of my vocal command.

But here’s the cool part!!!

With Smart Things, I now have a door sensor, a personal sensor, and a motion detector, so when I come home, Jarvis says, “Welcome home Eric. I hope you had a fantastic day. How may I be of assistance to you?”

If there is motion detected at night or when I’m away, then IFTTT throws a pushbullet to eventghost, which plays an audio file with a VERY loud siren on it, my lights turn red and start flashing, and Jarvis says, “Intruder alert. Alert. Alert. There is an intruder detected in the building. Intruder Alert. Contacting local police automatically for emergency response. Intruder alert. Alert.” And this continues until I command him to stop.

My goal is to put a motion detector in every single room of my house, put a Philips hue in every room, and if I can, i want to figure out how to use the motion sensors and personal presence devices in conjunction so that you can tlel who is where in the house, and from there I can script for teamspeak to direct all messages for that user in the channel they’re in. This way you get some sort of sense of privacy with the system, like if I can place a phone call over the teamspeak system so that you don’t need to have your phone on you constantly.

Additionally, I intend to have a separate smartphone device in every room (they don’t need phone access, just internet wifi access to connect to teamspeak). At this point I’ll hook up a speaker to each of them and configure some muting for specific devices across the platform to decrease the potential for echoing.

When this is done, I will have Jarvis (who sounds like Jarvis —using Ivona voice Brian British English) as a personal assistant able to make appointments, remind me of appointments, turn on and off IFTTT recipes (another script I’ll have to write since IFTTT won’t let you do it automatically), and I’ll have an actual smart house, that talks to me, and I can talk back to and have it do specific things for me.

Another idea is to have a fitbit scale in the bathroom where Jarvis can keep record of my weight in order to query a database on my current workout routine with fitbit to determine what food I should eat and when, make a grocery list, and order for delivery my groceries online automatically (with confirmation of course).

What do you guys think? Any suggestions on how to make this more awesome?

-Eric

12 Likes
2

Can you rename Jarvis to Carson (Downton Abbey)? :wink:

3

you can call the LINKS program anything you like, give it a name, and even change the Ivona voice on your computer or it. So, if you want SARAH from ‘Eureka’ then you can do that too :wink:

1 Like
4

Wow, a Eureka Reference! Good Job! :smile:

1 Like
5

Very cool project and thanks for sharing! Is the primary method of communication through the phone (TeamSpeak client)? Do you currently have a voice interface directly to the PC (eg. Kinect or other long-range microphone)?

6

Actually Joshua, no, I don’t have a dedicated microphone on the actual computer, and here’s why:

The goal in the future for me is to be able to have 3D mapping and detection of individuals in any room in the house, that’s future, but that means there has to be a microphone and speaker in every room as well, but I had to figure out a way to make them both unified and segregated as well.

This is my first step towards that goal, anything after this will be incredibly complicated, but once it’s done it will be easy to implement anywhere else. Teamspeak gives me the option to create a channel for all the devices to communicate on, and anyone can give commands to the jarvis system and therefore the house as well anywhere they’re at, and Jarvis can respond to anyone in the house at any time as well.

So my goal is to have it so that someone can set privacy in their room, so that they can have a phone call or watch a video without interfering with anyone else in other rooms of the house. This is going to be difficult because Jarvis only has one session and anyone talking to jarvis will get the same response. The only way, right now, to fix this would be to setup a physical (or virtual) box that has an individual jarvis and teamspeak client on it, so that they can either join one room or the other or be in one setting together. All music and files will be shared among those systems so that everyone can play their own music if they choose.

This is going to take a lot more resources and a much more powerful computer to operate, but the best part about it is that Jarvis and teamspeak are free, so is virtualbox, and the only expense would be getting a single server powerful enough to run an individual VM for each room.

However, right now, this computer is my personal computer, so I often have to change things back just so I can watch a movie or listen to music on my surround sound speakers. That should change once I can segregate Jarvis.

7

Cool project, once i will be home, i´m going to follow your Project !!!

8

@shawneric - thanks for the response. It sounds like a cool project with potential for really interesting features in the future!

I wrote an app for Android that I use for voice commands with SmartThings, among other things. The app is called SharpTools and is a set of Tasker plugins and Widgets for SmartThings. I paired it with AutoVoice to get some cool voice command features, but I have been amazed with the creative uses other users have come up with!

I am interested to see what other cool ideas you (and others) come up with using a PC based solution!

9

oh that’s pretty neat Joshua!!!

The Jarvis program I’m using is still in beta and uses a great deal of system resources, so it’s difficult to manage it, much less more than one. I would certainly have to get a beefy server that can run multiple virtual machines in order to get my entire system up and running.

I was looking for a tasker app for this. Really what I REALLY need is a program that works with teamspeak and bluetooth. I would LOVE to have a bluetooth headset on that could talk with jarvis at all times, but I would have to map my keys with my phone in order to get it to work. It’s weird O.o

1 Like
10

I’m using Hey Siri with IFTTT so the core process is quite similar to what you’re doing, but without natural language. So it sounds a bit clunky (“hey siri. Tell house hashtag TV”) but works well and has no extra cost if you have an iPhone with unlimited text.

Other people are using Ubi or Ivee or @joshua_lyon 's sharptools for android.

You might find this topic interesting:

1 Like
11

I used the auto voice and google now app on my android to do the same thing. The only problem is I can’t specify what I want it to say back to me. I want to make this as realistic a ‘smart house’ as I can, so i don’t want it to be restricted to my phone.

12

Has anyone had any luck on your end doing this?

13

Luck doing what specifically?

14

My project in the original post.

15

Nope. I tried to install Jarvis, and it won’t even launch. I read on their forum that is a common problem and tried a couple “fixes”. None worked, and I didn’t have patience or time for a program that needs all kinds of love just to start.

16

If you are using Tasker, you can have it say whatever you want using the ‘Say’ action in Tasker. For a more natural voice, check the ‘Network’ box which will use network synthesis resulting in better quality (but with a slight delay).

If you are looking for the machine learning side of things, then today it seems the PC (or the cloud) would be the best place for that. With the Tasker setup, you’ll have to configure your commands and expected responses. I setup a whole list of actions I can ask my phone like “what temperature the AC is set to”, “is XX turned on” (example), “is the front door locked”, etc and the phone responds with (pre-programmed) natural language like “the temperature is set to 76 degrees”, “the bedside lamp is off”, “the front door is locked”.

1 Like
17

Hmm…ok. I haven’t had any issues installing or running Jarvis on my computer, I’ve used four different OS platforms when doing it as well.

18

Well, now you’ve got me wondering what I’ve installed because the site I went to states the program is currently Windows only even though it has Mac and Linux posted. The links just take you to a “coming soon” donation request.

19

Ah, I’m sorry. When I say ‘OS Platforms’ I mean different Windows OS Platforms. Not every windows OS works the same, and this software has worked on all of mine. But mac and linix, I’m not sure how they’d work with it. You could probably get it to work within a virtual machine.

20

I installed it on my Windows 8.1, but that’s where I’m having the trouble and can’t seem to find a solution. The suggestions on the forum for those having the same were to install SP1 (already have), turn off anti-virus (don’t have any aside from Windows), and to fresh install windows which is not going to happen.

I will keep my eye out for their next update to the app and try it again then.