Very cool project and thanks for sharing! Is the primary method of communication through the phone (TeamSpeak client)? Do you currently have a voice interface directly to the PC (eg. Kinect or other long-range microphone)?
Actually Joshua, no, I don’t have a dedicated microphone on the actual computer, and here’s why:
The goal in the future for me is to be able to have 3D mapping and detection of individuals in any room in the house, that’s future, but that means there has to be a microphone and speaker in every room as well, but I had to figure out a way to make them both unified and segregated as well.
This is my first step towards that goal, anything after this will be incredibly complicated, but once it’s done it will be easy to implement anywhere else. Teamspeak gives me the option to create a channel for all the devices to communicate on, and anyone can give commands to the jarvis system and therefore the house as well anywhere they’re at, and Jarvis can respond to anyone in the house at any time as well.
So my goal is to have it so that someone can set privacy in their room, so that they can have a phone call or watch a video without interfering with anyone else in other rooms of the house. This is going to be difficult because Jarvis only has one session and anyone talking to jarvis will get the same response. The only way, right now, to fix this would be to setup a physical (or virtual) box that has an individual jarvis and teamspeak client on it, so that they can either join one room or the other or be in one setting together. All music and files will be shared among those systems so that everyone can play their own music if they choose.
This is going to take a lot more resources and a much more powerful computer to operate, but the best part about it is that Jarvis and teamspeak are free, so is virtualbox, and the only expense would be getting a single server powerful enough to run an individual VM for each room.
However, right now, this computer is my personal computer, so I often have to change things back just so I can watch a movie or listen to music on my surround sound speakers. That should change once I can segregate Jarvis.
Cool project, once i will be home, i´m going to follow your Project !!!
@shawneric - thanks for the response. It sounds like a cool project with potential for really interesting features in the future!
I wrote an app for Android that I use for voice commands with SmartThings, among other things. The app is called SharpTools and is a set of Tasker plugins and Widgets for SmartThings. I paired it with AutoVoice to get some cool voice command features, but I have been amazed with the creative uses other users have come up with!
I am interested to see what other cool ideas you (and others) come up with using a PC based solution!
oh that’s pretty neat Joshua!!!
The Jarvis program I’m using is still in beta and uses a great deal of system resources, so it’s difficult to manage it, much less more than one. I would certainly have to get a beefy server that can run multiple virtual machines in order to get my entire system up and running.
I was looking for a tasker app for this. Really what I REALLY need is a program that works with teamspeak and bluetooth. I would LOVE to have a bluetooth headset on that could talk with jarvis at all times, but I would have to map my keys with my phone in order to get it to work. It’s weird O.o
I’m using Hey Siri with IFTTT so the core process is quite similar to what you’re doing, but without natural language. So it sounds a bit clunky (“hey siri. Tell house hashtag TV”) but works well and has no extra cost if you have an iPhone with unlimited text.
Other people are using Ubi or Ivee or @joshua_lyon 's sharptools for android.
You might find this topic interesting:
I used the auto voice and google now app on my android to do the same thing. The only problem is I can’t specify what I want it to say back to me. I want to make this as realistic a ‘smart house’ as I can, so i don’t want it to be restricted to my phone.
Has anyone had any luck on your end doing this?
Luck doing what specifically?
My project in the original post.
Nope. I tried to install Jarvis, and it won’t even launch. I read on their forum that is a common problem and tried a couple “fixes”. None worked, and I didn’t have patience or time for a program that needs all kinds of love just to start.
If you are using Tasker, you can have it say whatever you want using the ‘Say’ action in Tasker. For a more natural voice, check the ‘Network’ box which will use network synthesis resulting in better quality (but with a slight delay).
If you are looking for the machine learning side of things, then today it seems the PC (or the cloud) would be the best place for that. With the Tasker setup, you’ll have to configure your commands and expected responses. I setup a whole list of actions I can ask my phone like “what temperature the AC is set to”, “is XX turned on” (example), “is the front door locked”, etc and the phone responds with (pre-programmed) natural language like “the temperature is set to 76 degrees”, “the bedside lamp is off”, “the front door is locked”.
Hmm…ok. I haven’t had any issues installing or running Jarvis on my computer, I’ve used four different OS platforms when doing it as well.
Well, now you’ve got me wondering what I’ve installed because the site I went to states the program is currently Windows only even though it has Mac and Linux posted. The links just take you to a “coming soon” donation request.
Ah, I’m sorry. When I say ‘OS Platforms’ I mean different Windows OS Platforms. Not every windows OS works the same, and this software has worked on all of mine. But mac and linix, I’m not sure how they’d work with it. You could probably get it to work within a virtual machine.
I installed it on my Windows 8.1, but that’s where I’m having the trouble and can’t seem to find a solution. The suggestions on the forum for those having the same were to install SP1 (already have), turn off anti-virus (don’t have any aside from Windows), and to fresh install windows which is not going to happen.
I will keep my eye out for their next update to the app and try it again then.
that’s to bad. I couldn’t find an alternative that worked as well as Jarvis does. I sure hope they can fix this issue.
I started thinking about how to do this as well, and came up with a question…specifically about your implementation of PushBullet. It appears you only use PushBullet as the ‘glue’ to get to ITFFF so you can control the SmartThings devices, correct? It would seem you could replace this with an app and some get/post http code that would ‘push’ directly to the SmartThings cloud, similar to how SmartTiles or Simple Rules builder works. I know you mentioned you are not a coder, but I copied @625alex and @JoeC on this as they are the creators of the apps mentioned.
I just started reading up on web to SmartThings interfacing and believe I understand how the basics on how it is supposed to work, but wanted to verify with you both of your thoughts on how this might be implemented. I envision a simple SmartThings app that allows you to choose which devices can be accessed via the web. The more difficult part for me to understand is the http code that is required to ‘trip’ those devices…I know would need to use OAuth to accomplish this, but from there I don’t quite know what is next.
In addition, Eric’s example in this thread, I would see his EventGhost sending a ‘macro’ to a web page (PHP? Java?) with some sort of parameter that would trip the device. Since I am familiar with get/post, I see something like this “http://webserver.com/code.php?device=Kitchen&command=off”. However, I assume you wouldn’t want to call a web page and instead do some sort of curl command?
Can either of you give some pointers on how this might work? Also, since you already did a lot of the heavy lifting with getting this to work would you be interested in starting off some base code that has simple functionality? I could see this being used heavily by others as a ‘template’ to get them started building web-to-SmartThings interfaces.
The best place to start is to look through Web Services Guide. It shows you how to create endpoints. Once you obtain the endpoints, you plug them into any other software you want to interface with SmartThings. You can trigger the endpoints directly, there is no need for intermediate layer.
SmartTiles is just an endpoint on steroids. The actual control of devices happens once mappings redirects execution to a method. In that method you could do anything you need.
This is the place in SmartTiles where the magic happens.
OAuth is only used for authentication, it’s not an important part of this process. If it’s easier, you could manually authenticate by calling
createAccessToken() in your SmartApp. It will assign a new access token and save it in
state.accessToken, which you then print in the logs, for example.
I’m not sure what exactly you are trying to achieve, but there are many, many creative ways to do it.
Ok…I will create a small app (one switch) and see if I can build the interface. The documentation was not as clear as I would have liked, but will get through it.
Are you open to questions (here or PM) to assist if I run into some issues?