[HOW TO] Control Harmony hub using Google Home (individual button presses, not activities)

requires_server
project_voice

(Chris) #1

This setup will allow you to issue individual button press commands by voice from a Google Home device to multiple Harmony hubs. Example uses:
“Hey Google, tell living room to pause”
“Hey Google, tell master bedroom volume up 6 times”
“Hey Google, tell living room left 3 times”
“Hey Google, tell master bedroom to mute”

There’s no support for launching or ending activities, didn’t see a need since it’s easy to turn them on or off already using either GH/SmartThings integration or GH/IFTTT/Harmony integration.

Much credit goes to sushilks from GitHub, he developed the js utilities for Harmony, I just created a simple web server that accepts GET requests and calls certain functions in his harmony-hub-util app. Alexa/Echo users may want to check out his alexaHarmonyApp, it uses a custom Alexa skill rather than IFTTT+CoRE to accomplish the same thing. I was using this with my Echo Dot until I got it working through GH.

This setup is probably not going to be a good solution for a casual or non-techie SmartThings user because it requires setting up an internal web server and most likely modifying the command set that I’ve included to customize it for your needs.

If that doesn’t scare you, read on…

Here’s how it works:

  1. You execute an IFTTT applet using Google Assistant.

  2. IFTTT triggers it’s Maker channel to make a web request to CoRE running on your SmartThings account, passing along your command, destination IP address for your Harmony hub, and a command counter (like volume up 5, left 3, etc) as variables.

  3. CoRE, acting as the gateway between the internet and your internal network, takes the incoming request and stores the included variable values, then turns around and makes a web request to your internal web server, passing along the same variable values.

  4. The web server application captures the variable values from the CoRE web request and passes them to the harmony-hub-util application to issue the proper commands to your Harmony hub.

Pre-requisites:

  • IFTTT account, with Google Assistant and Maker channels activated
  • Computer on your local network that stays turned on, logged in, and connected to your local network all the time. This will be your internal web server. note - all of my instructions are based on my experiences setting this up on a Raspberry Pi. Theoretically it should be possible on a Windows machine or a Mac, but additional steps may be necessary
  • Node.js installed on your internal web server (download here: https://nodejs.org/en/download/)
  • Static IP address for your internal web server and Harmony hub(s). Assign through your router.
  • CoRE smartapp installed with dashboard enabled

Since there are many moving parts to this installation, I’m going to divide them up into phases. This will make the setup process easier, and testing at the end of each phase will make troubleshooting easier as well.

Phase 1 - get the web server up and running

  1. Download the zip file from my GitHub site: https://github.com/destructure00/web-harmony. This is my first attempt at making my own app and sharing it on GitHub, so hopefully I didn’t mess it up. :cold_sweat:

  2. Extract the zip file to whatever folder you want. I’m on a Raspberry Pi, extracted to /home/pi/web-harmony

  3. Open a command/terminal window, navigate to your target folder, and run the command “npm install”. This will install the web server and it’s dependencies. It should go off and download the necessary dependencies for a minute or two, then drop you back at the prompt. You may get a warning message or two, hopefully no errors.

  4. Start the web server by running “node server.js” in the same folder. You should get a message that it’s listening on port 8079. I have it set up to listen on this non-standard port to avoid conflicts with other web apps I have running. You can change this as needed by editing the server.js file.

  5. Test to see if the web server is working correctly. Enter the following URL in a web browser: http://(your web server ip):8079/remote?command=mute&count=1&harmony_ip=(your harmony ip). You should receive a response like the following:

If this is all working correctly, you’re ready for phase 2. If not, let me know in the comments below and we can troubleshoot.

Phase 2 - Create a CoRE piston to make requests to your internal web server

  1. Create a new Basic piston

  2. Initialize local variables command, count, and harmony_ip, all as strings. Values don’t matter right now (you can enter whatever you want), we just need them to exist so they can be added to your piston action.

  3. Capability = IFTTT, Comparison = Executed, Value = harmony. Make sure you select the option to Import event data on true, this is what tells CoRE to store the incoming request values to variables.


  4. Action = Make a web request, URL = http://{web server ip}:8079/remote, method = GET, content type = FORM, Variables to send - command, count, and harmony_ip

  5. Save and exit

Here’s the final product:

Phase 3 - Create your IFTTT applets

Create your first applet to handle single-press commands like pause, play, select, menu, etc

  1. “this” = Google Assistant

  2. Select “Say a phrase with a text ingredient”.

  3. In the “What do you want to say” section, enter “tell living room to $”. You can customize this as desired, change room name, etc.

  4. If you want GH to repeat the command back to you, enter something like the following in the response section:

  5. “that” = Maker, Make a web request

  6. Open your CoRE dashboard in a separate tab in your browser. Copy the URL.

  7. Paste the CoRE URL into the Maker URL field. At the end of the URL, Replace “dashboard#/” with “ifttt/harmony”

  8. Set Method = GET, content type = application/json, body as shown, replacing my IP address with your Harmony hub IP address

Create your second applet to handle multiple-press commands like volume up 6, right 3, etc.

  1. “this” = Google Assistant

  2. Select “Say a phrase with both a number and a text ingredient”.

  3. In the “What do you want to say” section, enter “tell living room $ # times”. You can customize this as desired, change room name, etc. I added the word times to the end because without it, google gets confused between numbers and words at the end, two/too, four/for. Adding another word at the end seems to have fixed this issue.

  4. If you want GH to repeat the command back to you, enter something like the following in the response section:

  5. “that” = Maker, Make a web request

  6. Open your CoRE dashboard in a separate tab in your browser. Copy the URL.

  7. Paste the CoRE URL into the Maker URL field. At the end of the URL, Replace “dashboard#/” with “ifttt/harmony”

  8. Set Method = GET, content type = application/json, body as shown, replacing my IP address with your Harmony hub IP address

If you have additional Harmony hubs, repeat Phase 4 and change the room name and hub IP address. I have two Harmony hubs, so I have a total of 4 IFTTT applets. 2 start with “tell living room” and 2 start with “tell master bedroom”.

Phase 4 - customization
Commands are executed in the remote.js file. The last command received can always be viewed in the server.js terminal window or in CoRE by looking at the variable values. If you want to add additional phrases to execute existing commands, you can just add them to the switch block in remote.js. If you want to add new commands. I’d recommend installing harmonyHubCLI, also, from sushilks, and using that to find commands that are available for your activities. You can also download and dissect remote.js from his alexaHarmonyApp to see what other commands are available.

That should be it! Let me know if you run into trouble and I’ll try to help out.


webCoRE - Volume up or down with harmony hub?
Universal Smartthings Web Service
CoRE - Get peer assistance here with setting up Pistons
Calling all community members: New SmartApp Ideas
(Chris) #3

I think I’m done with the instructions…if anyone has trouble let me know and I’ll help troubleshoot to the best of my ability.


(Dana ) #4

Wish I could try this right now, but it’s a little above my pay grade and I don’t have time at the moment sit and push through doing this via mental brute force. :slight_smile: Reading stuff like this does make me think I’ll have to get a Raspberry Pi at some point and learn how to use it.

Thanks for putting this together!


(Brett C) #5

I will definitely be trying this out! Thanks @destructure00!


(Brett C) #6

@destructure00 - If I understand this correctly, CoRE is simply being used to connect the dots between the IFTTT Maker request on the web to your local network, right? Since this requires a server to be running at all times, could you in theory use something like ngrok to tunnel the commands from Maker to your local network, and bypass the need for a CoRE piston and changes to the CoRE code? Or does what we setup in CoRE format the HTTP request in a particular way that is necessary?

UPDATE: I think I answered my own question. I setup an ngrok tunnel, and using cURL I can send the command via ngrok just fine, but I get errors when sending the commands via IFTTT Maker :frowning:

TypeError: Cannot read property 'replace' of undefined
at Object.handle (C:\nodejs\node_modules\web-harmony-master\server.js:13:33)
at next_layer (C:\nodejs\node_modules\web-harmony-master\node_modules\express\lib\router\route.js:103:13)
at Object.urlencodedParser [as handle] (C:\nodejs\node_modules\web-harmony-master\node_modules\body-parser\index.js:70:44)
at next_layer (C:\nodejs\node_modules\web-harmony-master\node_modules\express\lib\router\route.js:103:13)
at Route.dispatch (C:\nodejs\node_modules\web-harmony-master\node_modules\express\lib\router\route.js:107:5)
at C:\nodejs\node_modules\web-harmony-master\node_modules\express\lib\router\index.js:195:24
at Function.proto.process_params (C:\nodejs\node_modules\web-harmony-master\node_modules\express\lib\router\index.js:251:12)
at next (C:\nodejs\node_modules\web-harmony-master\node_modules\express\lib\router\index.js:189:19)
at Layer.expressInit [as handle] (C:\nodejs\node_modules\web-harmony-master\node_modules\express\lib\middleware\init.js:23:5)
at trim_prefix (C:\nodejs\node_modules\web-harmony-master\node_modules\express\lib\router\index.js:226:17)

(Chris) #7

Theoretically it should work. The base web server code I used actually had ngrok support, I removed it because I wasn’t going to use it. The downside to ngrok is that every time your session ends (reboot, etc ) you have to launch a new session and get a new URL, and would have to modify the IFTTT applets each time to match. Using CoRE, everything stays constant. I can see about adding that back in though if it’s more desirable than modifying CoRE.


(Glen King) #8

Looks like fun! But also looks like a tough slog, especially considering that when you are watching tv you are typically not moving around and typically have a remote right there with you.

So this seems to be something that, for me, will wait until the logic of these systems can allow you to say “ok google, turn on the Patriots-Steelers game on the theater tv” and it will then do all the work for you of finding out what channel the game is on, what time, and then scheduling your systems to turn on appropriately when that game begins.

In the meantime, I’ve programmed my Denon to receive volume and mute voice commands via Alexa. Because those are useful when listening to music or whatever (such as while doing something else, like cooking) as well as when watching tv. The following discrete voice commands were required:
Alexa, trigger

  • volume low
  • volume medium
  • volume high
  • audio mute
  • audio unmute
  • volume up
  • volume down

The first three are fixed values that are comfortable for myself and my wife, and the mute commands are obvious. All those use a single Tasker profile. The up/down commands have their own profile. They check the existing volume, use Tasker to compute 5db up or down from that, and send the new value. Because the default increment on the Denon is 0.5db, and saying “Alexa, trigger volume up” ten times in a row to hear any useful change is just silly.


(Brett C) #9

Agreed, that is a big downside for ngrok. I was just trying to avoid a new piston because I already have ~50 and am starting to think it’s affecting my system performance (haven’t done any troubleshooting to confirm yet though).

I’ll give it a go with CoRE when time permits and let you know how it goes following your guide. Thanks again!


(Brett C) #10

Unfortunately I’m getting same error with CoRE. I can see in the dashboard that the variables are being update appropriately, but when it makes its way to node.js I get the error. Not sure if maybe I’m missing something in my node.js setup? I’m new to that territory… The fact that it works fine with cURL though is puzzling.

TypeError: Cannot read property 'replace' of undefined
at Object.handle (C:\nodejs\node_modules\web-harmony-master\server.js:13:33)
at next_layer (C:\nodejs\node_modules\web-harmony-master\node_modules\express\lib\router\route.js:103:13)
at Object.urlencodedParser [as handle] (C:\nodejs\node_modules\web-harmony-master\node_modules\body-parser\index.js:70:44)
at next_layer (C:\nodejs\node_modules\web-harmony-master\node_modules\express\lib\router\route.js:103:13)
at Route.dispatch (C:\nodejs\node_modules\web-harmony-master\node_modules\express\lib\router\route.js:107:5)
at C:\nodejs\node_modules\web-harmony-master\node_modules\express\lib\router\index.js:195:24
at Function.proto.process_params (C:\nodejs\node_modules\web-harmony-master\node_modules\express\lib\router\index.js:251:12)
at next (C:\nodejs\node_modules\web-harmony-master\node_modules\express\lib\router\index.js:189:19)
at Layer.expressInit [as handle] (C:\nodejs\node_modules\web-harmony-master\node_modules\express\lib\middleware\init.js:23:5)
at trim_prefix (C:\nodejs\node_modules\web-harmony-master\node_modules\express\lib\router\index.js:226:17)

EDIT: I just noticed that right above this error is a pair of empty brackets {}. Looks like the command, count, and address aren’t making it from CoRE to the node.js server for some reason.

EDIT2: FYI - Wireshark shows that the HTTP packet from CoRE has the data appended to the URL of the POST.


(Chris) #11

Can you post a screenshot of your modified CoRE code?

I wonder if I can get my web app to parse the URL instead of the request body, would negate the need for modifying CoRE.


(Brett C) #12

Sure thing:

Here’s what I see in Wireshark as well if it helps at all:


(Chris) #13

Hmmmm…everything looks correct in the code, but the CoRE isn’t putting the variables in the request body. This is the problem I was having that led me to modify the code to begin with. Did you publish the updated code after you modified? Check your IDE Smartapp page - does it say Edited or Published?

Before:

Publishing changes:

After:

The best answer may be for me to work around the lack of CoRE encoding support so no modifications need to be made. I’ll work on it this evening and report back.


(Brett C) #14

Yep, code is published, and using FORM in the piston. Interesting that it worked for you and not me. I am doing this on Windows which I thought might be causing issues, but since I can see the request coming through in the query I’m guessing that’s not it.

Let me know if you need anything else!


(Chris) #15

I’m interested in the fact that you have several instances of CoRE showing in your Smartapps page. I have 6 instances running in my account but only 1 showing on the Smartapps page in IDE. Maybe that has something to do with it. Regardless, I’ll try to make it so no CoRE modifications are necessary, probably should have taken that approach to begin with.


(Brett C) #16

That was probably due to my being a newb LOL. I just created a new SmartApp from code for each instance, but it looks like I can just go back in to the Marketplace in the ST app to create a new instance? Would hate to rebuild all of my pistons, but might be worth it to keep things clean!


(Chris) #17

The trick to this is changing line 59 singleInstance from true to false. When it’s set to true (default) and you tap on CoRE from your apps in the marketplace it takes you into your existing instance. If you change this to false, it creates a new instance.


(Brett C) #18

Excellent! Thanks for the tip even though it’s slightly off-topic :smiley:


(Chris) #19

Just did some searching and it should be easily doable to read the URL query parameters instead of the post request body. Actually will make web-harmony a bit lighter too, one less module required, no need to edit CoRE code, and no need to test with curl/postman, should be able to enter in a standard browser window instead. Gotta wait til I get home to try though, haven’t gotten remote access to my pi working right yet…


(Chris) #20

@Brett_C let’s give this a shot. Replace the entire contents of the server.js file with the code below, then kill and restart node server.js. Test by putting the following in your web browser:
http://192.168.86.40:8079/remote?command=volume%20up&harmony_ip=192.168.86.32&count=3

var express = require('express');
var app = express();
var remoteapp = require('./remote.js');
const serverPort = 8079;


app.get('/remote', function (req, res) {
  if (!req.query) return res.sendStatus(400)
  console.log(req.query);
  var command = req.query.command.replace(" ","_").toLowerCase();
  var hub_ip = req.query.harmony_ip;
  var amt = Number(req.query.count);
  if (command && hub_ip && amt){
    res.send('Received ' + command + ' x' + amt + ' command for ' + hub_ip);
    console.log("Received " + command + " x" + amt + " command for " + hub_ip);
    remoteapp.dostuff(hub_ip, command, amt, function(res) {
      console.log(res + " " + command);
    });

  }else{
    res.send('Error');
  }

})
app.listen(serverPort);
console.log("Listening on port "+serverPort);  

You should see output like this:

If that works, go edit your piston and change the method in your action from POST to GET and try the voice commands again. If this works for you, I’ll update the github file and the instructions.


(Brett C) #21

Works like a charm! Great job Chris!

One thing I did notice - I added a few iterations of commands to the remote.js file (i.e. “turn_the_volume_up” in addition to just simply “volume_up”), but when the server receives the command it only replaces the space with an underscore for the first space in the phrase. In other words it ends up sending “turn_the volume up” which then doesn’t match.

Can I leave the spaces in the remote.js file, or is there something that I need to change in the server.js file? I can see the replace command, but don’t see anything that would limit it to only one replace?

P.S. - this does work with ngrok now as well if anyone wanted to go that route, noting the downside of needing to update the IFTTT applet anytime your server gets restarted.