j64Harmony - Amazon Echo + Harmony Hub Integration

I’ve written an app that will allow you to control a Harmony Hub via the Amazon Echo. Yes, you can currently do this by setting up different activities in the harmony hub to control channels and such. That all works OK, but I found it a little difficult to setup activities to control the volume and more importantly I really wanted to learn a little more about writing LAN connected smart apps and device types. So, I set out to build a few new device types that allow you to toggle the mute button on/off, change the channel, pause the DVR and of course to automatically “surf” channels for you. Here is a screen shot of the various devices in the mobile app.

I used simple device types (switches) in this app so that it would be easily integrated with the Amazon Echo. This lets you say things like “Alexa, turn off sound” or “Alexa, Turn on ShowTime” or “Alexa turn on Channel Surfing” and it will send the appropriate commands to the harmony hub. When I am away from the remote, my favorite and most used voice command is to tell Alexa to turn off the sound. My wife is even starting to appreciate that command so the overall acceptance factor has definitely been raised in my house.

I’ve been using this app for the last couple of weeks and it works great for me so I thought I would put it out here and invite others to try it out. I am very interested to know if it works well for you, what other commands you think would be useful, any issues you have getting it running, etc.

You can download the code and associated apps from my github: https://github.com/joejarvis64/j64.Harmony. There are some pre-reqs for this app and it does require that you run a gateway app on a local machine so it can talk to the Harmony hub. I created a GettingStarted document that provides instruction on how to install and get everything running.

Once you have installed the app you will be able to define channels and names that the Amazon Echo will recognize using a web app i’ve created. The nice thing about this app is that it has the smarts to synchronize with the smart things hub and properly create devices that the Echo will recognize.

Let me know if you decide to give it a spin!

5 Likes

I’m giving this a whirl.

I have istalled this on a Windows 10 machine.

I get to

I enter my info and hit connect to Harmony Hub, I then get a blank screen.

I get 1 failed in the CMD window.
fail: Microsoft.AspNet.Diagnostics.ExceptionHandlerMiddleware[0] An unhandled exception has occurred: One or more errors occurred. System.AggregateException: One or more errors occurred. ---> System.Net.Sockets.SocketException: No such host is known at System.Net.Sockets.Socket.EndConnect(IAsyncResult asyncResult) at System.Net.Sockets.TcpClient.EndConnect(IAsyncResult asyncResult) at System.Threading.Tasks.TaskFactory1.FromAsyncCoreLogic(IAsyncResult iar, Func2 endFunction, Action1 endAction, Task1 promise, Boolean requiresSynchronization) --- End of inner exception stack trace --- at System.Threading.Tasks.Task.ThrowIfExceptional(Boolean includeTaskCanceledExceptions) at System.Threading.Tasks.Task.Wait(Int32 millisecondsTimeout, CancellationToken cancellationToken) at System.Threading.Tasks.Task.Wait(Int32 millisecondsTimeout) at j64.Harmony.Xmpp.Hub.EnsureConnection() in D:\My Files\Downloads\Compressed\j64.Harmony-master\src\j64.Harmony.Xmpp\Hub.cs:line 337 at j64.Harmony.Xmpp.Hub.OpenConnection(String address, Int32 port) in D:\My Files\Downloads\Compressed\j64.Harmony-master\src\j64.Harmony.Xmpp\Hub.cs:line 111 at j64.Harmony.Xmpp.Hub.StartNewConnection(String userName, String password, String address, Int32 port) in D:\My Files\Downloads\Compressed\j64.Harmony-master\src\j64.Harmony.Xmpp\Hub.cs:line 89 at j64.Harmony.WebApi.Controllers.FirstTimeConfigController.Edit(FirstTimeConfigViewModel ftcViewModel) in D:\My Files\Downloads\Compressed\j64.Harmony-master\src\j64.Harmony.WebApi\Controllers\FirstTimeConfigController.cs:line 42 --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at Microsoft.AspNet.Mvc.Controllers.ControllerActionExecutor.ExecuteAsync(MethodInfo actionMethodInfo, Object instance, Object[] orderedActionArguments) at Microsoft.AspNet.Mvc.Controllers.ControllerActionInvoker.<InvokeActionAsync>d__6.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Microsoft.AspNet.Mvc.Controllers.FilterActionInvoker.<InvokeActionFilterAsync>d__53.MoveNext() --- End of stack trace from previous location where exception was thrown --- at Microsoft.AspNet.Mvc.Controllers.FilterActionInvoker.<InvokeAsync>d__44.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Microsoft.AspNet.Mvc.Infrastructure.MvcRouteHandler.<RouteAsync>d__6.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Microsoft.AspNet.Routing.Template.TemplateRoute.<RouteAsync>d__27.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Microsoft.AspNet.Routing.RouteCollection.<RouteAsync>d__9.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Microsoft.AspNet.Builder.RouterMiddleware.<Invoke>d__4.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Microsoft.AspNet.Authentication.AuthenticationMiddleware1.d__18.MoveNext()
— End of stack trace from previous location where exception was thrown —
at Microsoft.AspNet.Authentication.AuthenticationMiddleware1.<Invoke>d__18.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Microsoft.AspNet.Authentication.AuthenticationMiddleware1.d__18.MoveNext()
— End of stack trace from previous location where exception was thrown —
at Microsoft.AspNet.Authentication.AuthenticationMiddleware1.<Invoke>d__18.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Microsoft.AspNet.Authentication.AuthenticationMiddleware1.d__18.MoveNext()
— End of stack trace from previous location where exception was thrown —
at Microsoft.AspNet.Authentication.AuthenticationMiddleware1.<Invoke>d__18.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Microsoft.AspNet.Authentication.AuthenticationMiddleware1.d__18.MoveNext()
— End of stack trace from previous location where exception was thrown —
at Microsoft.AspNet.Authentication.AuthenticationMiddleware1.<Invoke>d__18.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Microsoft.AspNet.IISPlatformHandler.IISPlatformHandlerMiddleware.<Invoke>d__8.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Microsoft.AspNet.Diagnostics.ExceptionHandlerMiddleware.<Invoke>d__6.MoveNext() ---> (Inner Exception #0) System.Net.Sockets.SocketException: No such host is known at System.Net.Sockets.Socket.EndConnect(IAsyncResult asyncResult) at System.Net.Sockets.TcpClient.EndConnect(IAsyncResult asyncResult) at System.Threading.Tasks.TaskFactory1.FromAsyncCoreLogic(IAsyncResult iar, Func2 endFunction, Action1 endAction, Task1 promise, Boolean requiresSynchronization)<---

It is not able to resolve the “Harmony Address” name that you are using. Be sure to set the address to either the IP address of your local harmony hub or a fully qualified domain name. It was defaulted to the address I use so yours is probably different.

That did it.

Everything was installed without any other issues, I will be able to try it later.

One thing I did notice is there is no Skip Forward or Skip Back. I even created a sequence that hits Skip Forward 7 times.

It would be great if we can say Alexa Skip Forward Sequence.

By skip forward i assume you mean hitting the fast forward button on the DVR for a certain number of seconds or something like that? It would not be too add something like that. I will also clean up the “host not found” message so it is a little more obvious what the issue is.

Got it working for volume control. It works pretty well. How can I do this for each of my three hubs though? I’m going to have a Dot in my bedroom and theater, my echo is in my living room. Since it automates setup I’m guessing I can’t easily run three asp instances of this? Or do that but change the Names of the devices for the first before adding the second, change those, then the third? Will I even be able to run multiple instances?

I’ll be trying this out when my Dot gets here, my main Echo is in a different location than my entertainment room.

If you want to run 3 instances of the app, the easiest way would be to make a seperate copy of j64.Harmony src directories, 1 for each instance. Then change the hosting.json file under j64.Harmony.WebApi directory to listen on a different port. Maybe 2065,2066,2067.

{
  "server.urls": "http://0.0.0.0:2065"
}

You could point each instance at a different harmony hub. However, the way it works right now you would have to use different device names for each hub so the smart app/device names do not get confused. You can configure those names via the configure->device names & configure->favorite channels menu options. Once you change the names be sure to hit the “Resync with smart things” button to get the actual device names changed on the ST hub. If you watch live logging in the ST IDE it will show you any errors that might pop up as devices are being renames.

I could change the app to more easily handle multiple harmony hubs so you would not need to run multiple instances of the app. However, I am not really sure how the Echo would deal with duplicate names in each of the different rooms. For example in the living room if i say “turn off sound”, what would i say in the media room to my other echo device. I only have one Echo device/Harmony hub right now so am open to any ideas on how to best handle those type of scenarios?

I have a new dot coming as well! I am pretty curious to see how well it works when you have multiple in the same home.

I have it set for DirecTV. I have normal Fast Forward and I have Skip Ahead which jumps 7 seconds. I use Skip Ahead allot more than FF. I created a Sequence in Harmony the strings 7 Skip Aheads at a time.

When I do the Mute/UnMute this works great! My sound device is my Yamaha receiver. Unfortunately I think the volume up and volume down do not work correctly.

If I say Alexa, set [Sound] to 40% it decreases the sound by 2 clicks of the button. If I say Alexa, set [Sound] to 30% it just decreases by another 2 clicks. If I say Alexa, set [Sound] to 60% it icreases by 2 clicks and if I say Alexa, set [Sound] to 70% by another 2 clicks.

Yeah, for the sound i don’t really have a way to set an actual percentage on the sound device (yamaha receiver in your case). It is not a 2 way comm channel so does not really know what percentage it is at or have the ability to send an IR command that says set it to X percent. So all that volume device really does with the percentage is move it up/down by a couple of clicks as you have noticed. If you know of a better way to handle the volume with a harmony hub via activities or something sequence of commands i would be interested to understand more about that.

Here is a link that will show you how to create a sequence of button clicks within the Harmony mobile app. There is another one for creating in the Harmony Desktop app.

https://support.myharmony.com/en-us/creating-button-sequences#harmonyapp

I tried creating New Channels like HGTV, Sci Channel, I hit the Resync with Smartthings button, but the do not show up in ST.

Pause and Unpause work great on my DirecTV. Can I also set it for my FireTV?

When clicking that resync button can you watch the “Live Logging” in the smart things IDE. It may be showing some messages that would give a better indication of why it did not perform the update.

Hi there, I got stuck on Smartthings Login. If it helps. I have a Denon receiver for volume and actually used the Panasonic TV for Channels (I’m a “cordcutter” and I don’t have and STB)

info: Microsoft.AspNet.Mvc.Controllers.ControllerActionInvoker[1]
Executing action method j64.Harmony.WebApi.Controllers.OAuthController.CreateSmartApp with arguments (j64.Harmony.WebApi.ViewModels.Configure.OauthViewModel) - ModelState is Valid’
fail: Microsoft.AspNet.Diagnostics.ExceptionHandlerMiddleware[0]
An unhandled exception has occurred: One or more errors occurred.
System.AggregateException: One or more errors occurred. —> System.ArgumentException: Can not convert Object to String.
at Newtonsoft.Json.Linq.JToken.op_Explicit(JToken value)
at j64.Harmony.WebApi.Repository.DeviceTypeRepository.d__22.MoveNext()
— End of stack trace from previous location where exception was thrown —
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)

That “looks like” a problem with the smart things platform timing out. That spot in the program is where it is trying to add the device type and smart app for you. They’ve had quite a few issues with stability the last couple of days. Have you tried more than once to get it to add? It does not break anything to add those apps/device types multiple times.

Thanks @joe64 ! I’m using a Mac btw - I tried several times an hour or so ago and and tried some again just now. If I put in a wrong password it would say “Could not connect to smart things using the supplied credentials” If I put in the right password it would just go blank on this URL - http://localhost:2065/OAuth/CreateSmartApp with the a long list of errors on the terminal in one instant - after this, there is no more activity on terminal

I use a Mac also. I just tried to run through the “Create Devices” step and it took a while but did finish for me without any issues so there must be something else going on with your scenario.

That step is just trying to add the smart app and device types to the smart things IDE. You can manually add those items by following these directions: https://github.com/joejarvis64/j64.Harmony/blob/master/Documentation/ManualInstallStApps.md

If you decide to try that let me know as I am curious to see if that moves you forward.

After doing a manual install of the Smartapp, I discovered that the Device handlers have been already installed. I might have interrupted the “auto install” during my first try?

Anyway, I did a manual input of the keys on localhost and now it works! I’m now playing around with setting things up. Thanks again!

I am not sure how to obtain a password for my Harmony Hub. What do I need to do?