# Cortana (Xbox, Windows (Phone/OS) Integration (with Simple SmartThings App)

**URL:** <https://community.smartthings.com/t/cortana-xbox-windows-phone-os-integration-with-simple-smartthings-app/72990>\
**Category:** Ideas and Suggestions\
**Created:** [January 10, 2017, 3:11pm UTC](https://community.smartthings.com/t/cortana-xbox-windows-phone-os-integration-with-simple-smartthings-app/72990 "2017-01-10T15:11:14Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![Jimmy\_A](https://avatars.discourse-cdn.com/v4/letter/j/e95f7d/32.png) [@Jimmy\_A](https://community.smartthings.com/u/Jimmy_A)\
**Post date:** [January 10, 2017, 3:11pm UTC](https://community.smartthings.com/t/cortana-xbox-windows-phone-os-integration-with-simple-smartthings-app/72990/1 "2017-01-10T15:11:14Z")

</div>

I might just get back into programming just for this, perhaps if I get some support with command structures that ST works with so I can look into creating a voice command definition file for an Xbox background app for ST that takes voice commands from Xbox and passes it to ST hub to execute on any of its connected devices one is trying to interact with. The feedback from ST hub will also be returned to the background app and on to Cortana to render as desired (audiovisually). And Since Xbox One apps can be Universal Windows Platform (UWP) apps, it can be installed across any current Windows platform (they all have Cortana anyway, althought I’m more interested in Xbox and perhaps phone). It will be great if the existing official SmartThings app from Samsung can be extended to do this, but whatever, I guess. The more platforms that support ST, the better. Meanwhile: [https://channel9.msdn.com/Blogs/One-Dev-Minute/Integrating-Cortana-with-your-App](https://channel9.msdn.com/Blogs/One-Dev-Minute/Integrating-Cortana-with-your-App)

---

<div class="post-metadata">

**Author:** ![Jimmy\_A](https://avatars.discourse-cdn.com/v4/letter/j/e95f7d/32.png) [@Jimmy\_A](https://community.smartthings.com/u/Jimmy_A)\
**Post date:** [January 10, 2017, 3:11pm UTC](https://community.smartthings.com/t/cortana-xbox-windows-phone-os-integration-with-simple-smartthings-app/72990/2 "2017-01-10T15:11:54Z")

</div>

> **[Integrating Cortana in Your Apps](https://mva.microsoft.com/en-us/training-courses/integrating-cortana-in-your-apps-8487?l=20D3s5Xz_5904984382)**
>
> Explore Cortana and text-to-speech technology, learn to build speech-enabled mobile apps with Visual Studio for Windows devices.

---

<div class="post-metadata">

**Author:** ![tonesto7](https://yyz2.discourse-cdn.com/smartthings/user_avatar/community.smartthings.com/tonesto7/32/114859_2.png) [@tonesto7](https://community.smartthings.com/u/tonesto7)\
**Post date:** [January 10, 2017, 3:29pm UTC](https://community.smartthings.com/t/cortana-xbox-windows-phone-os-integration-with-simple-smartthings-app/72990/3 "2017-01-10T15:29:38Z")

</div>

I would be down to work on something like this. I have 2 XB1’s and about 15 Windows 10 devices so getting this working with Cortana would be nice

---

<div class="post-metadata">

**Author:** ![Jimmy\_A](https://avatars.discourse-cdn.com/v4/letter/j/e95f7d/32.png) [@Jimmy\_A](https://community.smartthings.com/u/Jimmy_A)\
**Post date:** [January 10, 2017, 3:40pm UTC](https://community.smartthings.com/t/cortana-xbox-windows-phone-os-integration-with-simple-smartthings-app/72990/4 "2017-01-10T15:40:18Z")

</div>

Wow! That’s a ton of devices. Lol. Sounds good, we can certainly work together. Are you in anyway an ST insider? We’ll need to start gathering required details for the voice command definition file.

I am also looking into (wireless) microphone integration options placed around the house and connected to the Xbox so one can issue voice commands from anywhere in the house and really do centrally managed HA right.

---

<div class="post-metadata">

**Author:** ![Jimmy\_A](https://avatars.discourse-cdn.com/v4/letter/j/e95f7d/32.png) [@Jimmy\_A](https://community.smartthings.com/u/Jimmy_A)\
**Post date:** [January 10, 2017, 3:51pm UTC](https://community.smartthings.com/t/cortana-xbox-windows-phone-os-integration-with-simple-smartthings-app/72990/5 "2017-01-10T15:51:31Z")

</div>

SmartThings developer docs will come in handy as well: [http://docs.smartthings.com/en/latest/](http://docs.smartthings.com/en/latest/)

---

<div class="post-metadata">

**Author:** ![seattle505](https://avatars.discourse-cdn.com/v4/letter/s/58f4c7/32.png) [@seattle505](https://community.smartthings.com/u/seattle505)\
**Post date:** [January 10, 2017, 4:42pm UTC](https://community.smartthings.com/t/cortana-xbox-windows-phone-os-integration-with-simple-smartthings-app/72990/6 "2017-01-10T16:42:29Z")

</div>

I would be interested in this as well. I am currently resisting getting an Echo to wait for a MS Cortana solution. I have heard there is a Cortana speaker coming. But I would really like this to be on the Phone-Tablet-xBox as well. We have an Xbox Windows 10 phone and 5 W10 tablets in the house. We just bought a Fire to control the ST via the Smart tiles but having a Cortana version would be great.

I did briefly try the “Triggers” app and it worked, but my ST was brand new and I truly had not figured out how to properly use it yet. By the time I had worked out the uses and processes for it, the triggers app was gone. 😢

---

<div class="post-metadata">

**Author:** ![tonesto7](https://yyz2.discourse-cdn.com/smartthings/user_avatar/community.smartthings.com/tonesto7/32/114859_2.png) [@tonesto7](https://community.smartthings.com/u/tonesto7)\
**Post date:** [January 10, 2017, 4:46pm UTC](https://community.smartthings.com/t/cortana-xbox-windows-phone-os-integration-with-simple-smartthings-app/72990/7 "2017-01-10T16:46:03Z")

</div>

> [@Jimmy\_A](#):
>
> Are you in anyway an ST insider?

I like to think I know my way around quite well 🙂 I am the creator of [Nest Manager](https://community.smartthings.com/t/release-nest-manager-4-0/60052/1)

---

<div class="post-metadata">

**Author:** ![ciroip](https://yyz2.discourse-cdn.com/smartthings/user_avatar/community.smartthings.com/ciroip/32/46004_2.png) [@ciroip](https://community.smartthings.com/u/ciroip)\
**Post date:** [January 10, 2017, 9:04pm UTC](https://community.smartthings.com/t/cortana-xbox-windows-phone-os-integration-with-simple-smartthings-app/72990/8 "2017-01-10T21:04:15Z")

</div>

If you are a developer the project is trivial:  
Smartthings have “Web services smartapps” infrastructure in place.  
[![](http://docs.smartthings.com/en/latest/_images/method-2.png) ](http://docs.smartthings.com/en/latest/_images/method-2.png)  
You can “expose” all your devices with few line of groovy (sigh) code and just hammer the ST servers with what you need:  
**_curl -H "Authorization: Bearer " "/switches"_**  
would return plain JSON like  
_**[{“name”:“Kitchen 2”,“value”:“off”},{“name”:“Living room window”,“value”:“off”}]**_  
and control them:  
**_curl -H "Authorization: Bearer " -X PUT "/switches/on"_**  
I was thinking about writing a GUI for the XBOX, WebOS tv and Kodi but not being able to publish the Smartapp on a public repository really killed my enthusiasm about it.

---

<div class="post-metadata">

**Author:** ![Jimmy\_A](https://avatars.discourse-cdn.com/v4/letter/j/e95f7d/32.png) [@Jimmy\_A](https://community.smartthings.com/u/Jimmy_A)\
**Post date:** [January 11, 2017, 4:03pm UTC](https://community.smartthings.com/t/cortana-xbox-windows-phone-os-integration-with-simple-smartthings-app/72990/9 "2017-01-11T16:03:13Z")

</div>

Sorry, I could not respond yesterday since this site prevented me because there is a limit to how much new members can post. Yes, here is the Cortana-enabled Harman/Kardon speaker scheduled for release this year: [https://www.youtube.com/watch?v=bikRuaJAv5g&feature=youtu.be](https://www.youtube.com/watch?v=bikRuaJAv5g&feature=youtu.be) . I’m assuming ST Cortana integration will kick off then and easily extend to Xbox or something since most of the code behind would already be done so they can just add a UI/UX to it if needed (for at least configuration pages) while keeping it as a background conduit-like app.  
I just don’t get why existing center of the home devices such as the millions of Xbox units out there are not front and center in HA voice commands already.

---

<div class="post-metadata">

**Author:** ![Jimmy\_A](https://avatars.discourse-cdn.com/v4/letter/j/e95f7d/32.png) [@Jimmy\_A](https://community.smartthings.com/u/Jimmy_A)\
**Post date:** [January 11, 2017, 4:22pm UTC](https://community.smartthings.com/t/cortana-xbox-windows-phone-os-integration-with-simple-smartthings-app/72990/10 "2017-01-11T16:22:16Z")

</div>

Great information! Although I’ve never heard of the “groove” programming language until yesterday as I looked into the ST dev docs. Interesting! I’m still looking into how apps will interact with ST and get all necessary details such as devices, rooms, routines, etc to quickly get up and running once the app is installed and set up on the Xbox to interact with the ST hub.

---

<div class="post-metadata">

**Author:** ![Entityxenon](https://yyz2.discourse-cdn.com/smartthings/user_avatar/community.smartthings.com/entityxenon/32/24647_2.png) [@Entityxenon](https://community.smartthings.com/u/Entityxenon)\
**Post date:** [January 11, 2017, 4:58pm UTC](https://community.smartthings.com/t/cortana-xbox-windows-phone-os-integration-with-simple-smartthings-app/72990/11 "2017-01-11T16:58:18Z")

</div>

Looks like an interesting project… I’d be keen to know what would or could be passed to ST from the app… “Groovy” is basically a variant of Java if you have any experience there…

For reference Ask Alexa or Google Helper developed by @MichaelS which is the Echo integration is probably the closest to what you want to achieve… I assume it’ll “just” be a different integration point… Tagging Michael in case he has any interest, although expect he’s already got a lot on his plate with both Google and Amazon integrations!

Also see the code from @N8XD which I think kicked off the Alexa integration, perhaps a good starting point…

> [@AskHome - Alexa/SmartThings programmers kit. Alexa can do anything](https://community.smartthings.com/t/askhome-alexa-smartthings-programmers-kit-alexa-can-do-anything/46613):
>
> Hi Guys, In real life you view your home as a structure filled with rooms, people, and objects. This kit helps you use Alexa to interact with your house by talking about those things - and not a list of devices. …which is great if you are helping someone non-technical make a smarthome. This is not a supported release, but a “share”. I’m sharing what I’ve done, and giving you the code so you can try it yourself if you want. Here is a short video with a few sample interactions from my home - and the kit code: documentation, smartApp, cut/pastes for alexa skills and lambda is here – [https://github.com/n8xd/AskHome](https://github.com/n8xd/AskHome). The demo code is configured to work with some things at my house…to make it YOUR AskHome, you’ll have to program your house into it. The AskHome kit is for groovy pro…

---

<div class="post-metadata">

**Author:** ![seattle505](https://avatars.discourse-cdn.com/v4/letter/s/58f4c7/32.png) [@seattle505](https://community.smartthings.com/u/seattle505)\
**Post date:** [January 11, 2017, 5:58pm UTC](https://community.smartthings.com/t/cortana-xbox-windows-phone-os-integration-with-simple-smartthings-app/72990/12 "2017-01-11T17:58:54Z")

</div>

Insteon already has Cortana integration as part of its base functionality. The MS store even sells their hardware directly.

However I have heard they have connection and other issues as wells as less integration with third party devices so last year I made the decision to go with ST. I know a couple who use Insteon and other than the investment of using their stuff they are happy and the Cortana works as advertised. So seems like we are just waiting for someone to develop this connection. I use Cortana on most of my devices. Xbox is still limited so we use it for mostly game stuff, but on my Phone PC and tablet I use it daily for all sorts of stuff, similar to what we would use Echo for, If I could ask Cortana to arm or disarm that would make me very happy.

I would be willing to Pay for this, but I am not sure how many others there who use this. Alexa seems to have cornered the market al least for now. As usual MS has a better product, but 2 years behind the competition

---

<div class="post-metadata">

**Author:** ![Jimmy\_A](https://avatars.discourse-cdn.com/v4/letter/j/e95f7d/32.png) [@Jimmy\_A](https://community.smartthings.com/u/Jimmy_A)\
**Post date:** [January 11, 2017, 6:05pm UTC](https://community.smartthings.com/t/cortana-xbox-windows-phone-os-integration-with-simple-smartthings-app/72990/13 "2017-01-11T18:05:54Z")

</div>

Insteon products are generally problematic as many of their users have come to realize and have been moving away from it. Too bad they are the ones that have embraced the Xbox, which is a powerful machine just sitting in most living rooms waiting to be leveraged for HA and other things outside of gaming and entertainment. And I know MS will gladly work with the SmartThings folks to get a SmartThings app out for this purpose in no time.

---

<div class="post-metadata">

**Author:** ![Jimmy\_A](https://avatars.discourse-cdn.com/v4/letter/j/e95f7d/32.png) [@Jimmy\_A](https://community.smartthings.com/u/Jimmy_A)\
**Post date:** [January 11, 2017, 6:38pm UTC](https://community.smartthings.com/t/cortana-xbox-windows-phone-os-integration-with-simple-smartthings-app/72990/14 "2017-01-11T18:38:07Z")

</div>

Nice! I will certainly look into the rest of this. So far, I have been looking into the voice command file and find that it certainly looks good but unsure what I will need from it as the Cortana voice command file I have seen and have been working on needs much less. For example, no need for Operands, Operators, Inquisitors, etc. I guess those are taken care of by Cortana. Here’s an example in simple XML-based as, being what is needed in the Cortana voice command file:

 SmartThings Show Security Status 

```
<Command Name="whatIsSecurityStatus">
  <Example> What is The Current Security Status </Example>
  <ListenFor RequireAppName="BeforeOrAfterPhrase"> what is the [current] security status </ListenFor>
  <ListenFor RequireAppName="ExplicitlySpecified"> what is the [current] {builtin:AppName} security status </ListenFor>
  <ListenFor RequireAppName="ExplicitlySpecified"> cancel [{builtin:AppName}] security status </ListenFor>
  <Feedback> Canceling security status... </Feedback>
  <Navigate />
</Command>

<Command Name="whatIsTheIndoorTemp">
  <Example> What is the current temperature?</Example>
  <ListenFor RequireAppName="BeforeOrAfterPhrase"> what is the current [indoor] temperature</ListenFor>
  <ListenFor RequireAppName="ExplicitlySpecified"> what is the current {builtin:AppName} temperature </ListenFor>
  <ListenFor RequireAppName="ExplicitlySpecified"> cancel [{builtin:AppName}] current temperature </ListenFor>
  <Feedback> Cancelling current indoor temperature...</Feedback>
  <VoiceCommandService Target="SmartThingsVoiceCommandService"/>
</Command>

<Command Name="open">
  <Example> Open (garage) door </Example>
  <ListenFor RequireAppName="BeforeOrAfterPhrase"> open {room} {device} </ListenFor>
  <ListenFor RequireAppName="ExplicitlySpecified"> open {builtin:AppName} {room} {device} </ListenFor>
  <ListenFor RequireAppName="ExplicitlySpecified"> cancel open [{builtin:AppName}] {room} {device} </ListenFor>
  <Feedback> Cancelling [{room}] {device}... </Feedback>
<VoiceCommandService Target="SmartThingsVoiceCommandService"/>
</Command>

<Command Name="close">
  <Example> Close (garage) door </Example>
  <ListenFor RequireAppName="BeforeOrAfterPhrase"> close {room} {device} </ListenFor>
  <ListenFor RequireAppName="ExplicitlySpecified"> close {builtin:AppName} {room} {device} </ListenFor>
  <ListenFor RequireAppName="ExplicitlySpecified"> cancel close {builtin:AppName} {room} {device} </ListenFor>
  <Feedback> Cancelling close [{room}] {device}... </Feedback>
<VoiceCommandService Target="SmartThingsVoiceCommandService"/>
</Command>    

<PhraseList Label="room">
  <Item>Main Living Room</Item>
  <Item>Kitchen</Item>
  <Item>Dining Room</Item>
  <Item>Powder Room</Item>
  <Item>Second Floor Stairway</Item>
  <Item>Second Floor Hallway</Item>
  <Item>Master Bedroom</Item>
  <Item>Master Bathroom</Item>
  <Item>Kids Bathroom</Item>
  <Item>Bedroom 1</Item>
  <Item>Bedroom 2</Item>
  <Item>Basement Stairway</Item>
  <Item>Basement Living Room</Item>
  <Item>Laundry Room</Item>
  <Item>Basement Bathroom</Item>
  <Item>Basement Bedroom</Item>
  <Item>Garage</Item>
</PhraseList>

<PhraseList Label="device">
  <Item>Fireplace</Item>
  <Item>Thermostat</Item>
  <Item>Sony Speaker</Item>
  <Item>Front Door Sensor</Item>
  <Item>Back Door Sensor</Item>
  <Item>Kitchen Window Sensor</Item>
  <Item>Living Room Window Sensor</Item>
  <Item>Philips Hue Ambiance</Item>
  <Item>Reading Light</Item>
  <Item>Chandelier</Item>
  <Item>Recessed Light</Item>
  <Item>Front Yard Camera</Item>
  <Item>Back Yard Camera</Item>
  <Item>Front Door Lock</Item>
  <Item>Back Door Lock</Item>
  <Item>Table Lamp</Item>
  <Item>Garage Door</Item>      
</PhraseList>

<PhraseList Label="routine">
  <Item>Good Morning</Item>
  <Item>Good Night</Item>
  <Item>Goodbye</Item>
  <Item>I'm Back</Item>
</PhraseList>

```

---

<div class="post-metadata">

**Author:** ![Jimmy\_A](https://avatars.discourse-cdn.com/v4/letter/j/e95f7d/32.png) [@Jimmy\_A](https://community.smartthings.com/u/Jimmy_A)\
**Post date:** [January 11, 2017, 10:43pm UTC](https://community.smartthings.com/t/cortana-xbox-windows-phone-os-integration-with-simple-smartthings-app/72990/15 "2017-01-11T22:43:41Z")

</div>

A great Cortana foundation (keep in mind, we are not trying to launch an app, more of a background conduit app between Xbox and ST hub, starts at the 38th minute mark): [https://channel9.msdn.com/Events/Build/2015/3-716](https://channel9.msdn.com/Events/Build/2015/3-716)

---

<div class="post-metadata">

**Author:** ![ciroip](https://yyz2.discourse-cdn.com/smartthings/user_avatar/community.smartthings.com/ciroip/32/46004_2.png) [@ciroip](https://community.smartthings.com/u/ciroip)\
**Post date:** [January 12, 2017, 4:38am UTC](https://community.smartthings.com/t/cortana-xbox-windows-phone-os-integration-with-simple-smartthings-app/72990/16 "2017-01-12T04:38:16Z")

</div>

> [@Jimmy\_A](#):
>
> I just don’t get why existing center of the home devices such as the millions of Xbox units out there are not front and center in HA voice commands already.

because Microsoft…and, in this case, because Samsung/Smartthings  
The Microsoft UWP apps (Universal Windows Platform aka write once execute everywhere) works flawless on the xboxone. The whole network stack is there and full access to internet: too bad they didn’t want developers to publish on the store. On the other hand Smartthings already have everything implemented ready for developers to access every device connected to the hub but again they didn’t let developers publish on their store.  
Note for the layman: if a platform have a store is pretty much impossible running anything else if you dont have developer superpowers. Really iimho there are no technical reasons.  
I’m sorrt. Im going back to sleep.

---

<div class="post-metadata">

**Author:** ![Jimmy\_A](https://avatars.discourse-cdn.com/v4/letter/j/e95f7d/32.png) [@Jimmy\_A](https://community.smartthings.com/u/Jimmy_A)\
**Post date:** [January 12, 2017, 2:26pm UTC](https://community.smartthings.com/t/cortana-xbox-windows-phone-os-integration-with-simple-smartthings-app/72990/17 "2017-01-12T14:26:58Z")

</div>

I find that hard to believe because there are other apps that do voice commands on the Xbox directly with other products. I learnt Insteon has theirs as well as two others I am aware of that don;t meet the needs we seek here. Except if you have further details you are yet to share, there is no reason why a perfectly fine app that does not violate security and privacy concerns will not be allowed onto the store. It just does not compute. More clarifying details as to exactly why what you said may be the case will be appreciated.

---

<div class="post-metadata">

**Author:** ![ciroip](https://yyz2.discourse-cdn.com/smartthings/user_avatar/community.smartthings.com/ciroip/32/46004_2.png) [@ciroip](https://community.smartthings.com/u/ciroip)\
**Post date:** [January 12, 2017, 3:01pm UTC](https://community.smartthings.com/t/cortana-xbox-windows-phone-os-integration-with-simple-smartthings-app/72990/18 "2017-01-12T15:01:12Z")

</div>

> [@Jimmy\_A](#):
>
> there is no reason why a perfectly fine app that does not violate security and privacy concerns will not be allowed onto the store

Microsoft promised to make UWP apps running on every device (when Windows phone were still a thing the holothing video were all over internet and they were completing the xboxone shift to windows 10 infrastrucure).  
I am super impress by how complete the framework is:it is a joy watching apps running without any modification on the xbox  
[[https://www.youtube.com/watch?v=Hk6pPNUsHCU](https://www.youtube.com/watch?v=Hk6pPNUsHCU) ]  
but they didn’t let me access the relative store (6 months ago).  
I’m a developer and I have a couple of apps in the Microsoft app store but last time I checked (again, months ago) Microsoft didn’t allow me to publish UWP apps for the XboxOne without being “x big company” and, at the time, they were specifically looking for games.  
It is really possible that the store is now open to Joe Schmo developers.  
I’m (slowly) porting my apps to UWP, I let you know if something changed.  
I apologize if I’m wrong and for the tone of my past posts: I’m always a bit bitter when as a developer I get frustrated by (usually obscure and unexplained marketing decisions).

---

<div class="post-metadata">

**Author:** ![Jimmy\_A](https://avatars.discourse-cdn.com/v4/letter/j/e95f7d/32.png) [@Jimmy\_A](https://community.smartthings.com/u/Jimmy_A)\
**Post date:** [January 12, 2017, 4:31pm UTC](https://community.smartthings.com/t/cortana-xbox-windows-phone-os-integration-with-simple-smartthings-app/72990/19 "2017-01-12T16:31:48Z")

</div>

Ok, I get your point and yes that was the case a while back. However, now the language has changed from just “games” to “games and apps”. And UWP apps are now a part of the options even during sign up for ID@Xbox (Independent Developer). Check it out at [http://www.xbox.com/en-US/developers/id](http://www.xbox.com/en-US/developers/id)

Getting started dev info: [https://msdn.microsoft.com/en-us/windows/uwp/xbox-apps/getting-started](https://msdn.microsoft.com/en-us/windows/uwp/xbox-apps/getting-started)

---

<div class="post-metadata">

**Author:** ![ciroip](https://yyz2.discourse-cdn.com/smartthings/user_avatar/community.smartthings.com/ciroip/32/46004_2.png) [@ciroip](https://community.smartthings.com/u/ciroip)\
**Post date:** [January 12, 2017, 6:47pm UTC](https://community.smartthings.com/t/cortana-xbox-windows-phone-os-integration-with-simple-smartthings-app/72990/20 "2017-01-12T18:47:28Z")

</div>

Excellent: I already deployed a couple of application on the XBOXOne.  
I could write a Smartthings remote in few minutes querying a websmartapp.

[Next page](https://community.smartthings.com/t/cortana-xbox-windows-phone-os-integration-with-simple-smartthings-app/72990.md?page=2)