SmartThings Community

Best way to collect logs from the hub?


(AD) #62

[TL:DR] - Problems with local Splunk communications having to do with string length. Modified POST to re-order commands and include an explicit Content-Length in the header. See bottom of post for code snip

Sorry for the long post – I’ve made a few edits throughout the night :slight_smile:

So I’m running into a weird issue that I’m trying to troubleshoot.

I’m logging locally successfully for most of my events, however my MultiSensor device is not logging to splunk when I do the local logging. I change to External and everything logs successfully.

I’ve turned on the debug in the SmartApp to show the JSON, and the only thing I can think of is the JSON has more data (longer string) than my other devices. When I look @ my splunk search log, when I go through external, the successful log entry for my MultiSensor is 611 characters, while my others average around 570.

Could there be a limit to the sendHubCommand? I’ve seen in other versions of the sendHubCommand there’s a length variable that gets passed to the POST command…

Wondering if anyone has run into similar issues with the SendHubCommand…

*adding debug info from Wireshark capture: looks like the Multisensor comes back with a 400 Bad Request for some reason

Successful post into splunk:

POST /services/collector/event HTTP/1.1
Accept: /
User-Agent: Linux UPnP/1.0 SmartThings
HOST: internal.ip.addr:8088
Authorization: Splunk my-splunk-key-removed-for-privacy
Content-Type: application/json
Content-Length: 585

{“event”:{“date”:“Wed May 18 05:20:59 UTC 2016”,“name”:“switch”,“displayName”:“Downstairs Hallway”,“device”:“Downstairs Hallway”,“deviceId”:“92e89916-1834-47cc-ab74-fafc9603e8a1”,“value”:“off”,“isStateChange”:“true”,“id”:“4e3adee0-1cb8-11e6-a452-d052a87298f7”,“description”:“zw device: 05, command: 2503, payload: 00”,“descriptionText”:“Downstairs Hallway switch is off”,“installedSmartAppId”:“null”,“isoDate”:“2016-05-18T05:20:59.292Z”,“isDigital”:“true”,“isPhysical”:“false”,“location”:“myHome”,“locationId”:“0c6ee2e0-a119-135f-9b8e-27702b80e47d”,“unit”:“null”,“source”:“DEVICE”,}}HTTP/1.1 200 OK
Date: Wed, 18 May 2016 05:20:59 GMT
Content-Type: application/json; charset=UTF-8
X-Content-Type-Options: nosniff
Content-Length: 27
Connection: Keep-Alive
X-Frame-Options: SAMEORIGIN
Server: Splunkd

{“text”:“Success”,“code”:0}

Unsuccessful post:

POST /services/collector/event HTTP/1.1
Accept: /
User-Agent: Linux UPnP/1.0 SmartThings
HOST: internal.ip.addr:8088
Authorization: Splunk my-splunk-key-removed-for-privacy
Content-Type: application/json
Content-Length: 621

{“event”:{“date”:“Wed May 18 05:20:53 UTC 2016”,“name”:“temperature”,“displayName”:“MultiSensor UnderStairs”,“device”:“MultiSensor UnderStairs”,“deviceId”:“88c6e1d4-9573-49cd-a45f-93a937a87a93”,“value”:“74.5”,“isStateChange”:“true”,“id”:“4ae153a0-1cb8-11e6-87a9-062254430c79”,“description”:“zw device: 08, command: 3105, payload: 01 22 00 EC”,“descriptionText”:“MultiSensor UnderStairs temperature is 74.5…F”,“installedSmartAppId”:“null”,“isoDate”:“2016-05-18T05:20:53.690Z”,“isDigital”:“false”,“isPhysical”:“false”,“location”:“myHome”,“locationId”:“0c6ee2e0-a119-135f-9b8e-27702b80e47d”,“unit”:“F”,“source”:“DEVICE”,}}HTTP/1.1 400 Bad Request
Date: Wed, 18 May 2016 05:20:53 GMT
Content-Type: application/json; charset=UTF-8
X-Content-Type-Options: nosniff
Content-Length: 27
Connection: Keep-Alive
X-Frame-Options: SAMEORIGIN
Server: Splunkd

{“text”:“No data”,“code”:5}

… Few Hours Later …

After more debugging, I think I was correct in the length being a problem. The string that gets bounced is actually 622 characters long, however the HTTP request was putting in “Content-Length” of 621.

I ended up modifying the code and reordering some of the POST request to line up with a successful request, and added an explicit Content-Length in the header based on the json character length.

Here’s a snip:

def length = json.getBytes().size().toString()

sendHubCommand(new physicalgraph.device.HubAction([
method: “POST”,
path: “/services/collector/event”,
headers: [
‘Authorization’: “Splunk ${splunk_token}”,
“Content-Length”: “${length}”,
HOST: “${splunk_server}”,
“Content-Type”:“application/json”,
“Accept-Encoding”:“gzip,deflate”
],
body:json
]))

Will continue to monitor to see if I get my events

Need to figure out how to capture errors with the sendHubCommand.


(Jason Hamilton) #63

I’ll be putting in your fixes to the main repo as well. Thank you @adrabkin for your research into this. I had no idea that there would be a length restriction. Yeah I’m also trying to figure out how to debug that command as well. Debugging the http response was easy but not sure about this command.

Edit: Changes have been checked in to github as well. Thanks for the help with that, I don’t know if I would’ve ever noticed that since I don’t have a multi


(AD) #64

Note sure that’s needed in the POST command, I just saw that from a successful entry that was coming from the Cloud, so I decided to through it in for good measure.

After looking through some other posts, I think the key for capturing errors is in a callback and tying into physicalgraph.device.HubResponse, but I’m still trying a bunch of things to get it to actually fire on an error.


(Jason Hamilton) #65

Ah I gotcha, well its not hurting it :slight_smile: Might remove it if it does cause an issue. Yeah I’m looking through some posts as well to find a nice example of how this is accomplished since the documentation doesn’t really tell me much.


(Jason Hamilton) #66

Well I can get it to log a bunch of “null” lol with this
log.debug hubAction
hubAction

So I think that we’re headed down the right path just need to figure out how to get it to actually throw some text in there besides null


(AD) #67

Ran into a new snag with the code.

I added a Schlage lock to the Splunk App to log my lock/unlock. As soon as I lock/unlock I get the following error in the LiveLogging:

groovy.lang.MissingMethodException: No signature of method: script14636285660451520135105.genericHandlers() is applicable for argument types: (physicalgraph.app.EventWrapper) values: [physicalgraph.app.EventWrapper@126344a6]
Possible solutions: genericHandler(java.lang.Object), energyHandler(java.lang.Object)

Seems as soon as it hits the event handler it craps out.

I am using a custom Device Handler for the Lock.

Diving into the lock device handler to see if I can figure this out :slight_smile:


(Jason Hamilton) #68

Yeah sadly I don’t have a zwave lock (Yet, really considering a yale because well I like Yale locks) So I can’t really debug this. Let me know how the debugging goes and what the fix will be :).


#69

Has anyone tried to log Aeontec Smart Stip plug metering data?

I have a home brew setup that I use this for, and I wanted to log how many watts I am pulling during the brew process.

Is there a SmartApp that could be written that places all the log data on your phone, from which you then can download and do what you please with it?

Thank you!


(Ben W) #70

https://artik.cloud/ has some cool graphing and its free for “hobbyist” usage. ST integration was pretty easy through the smartApp. I only have a few devices on it but it seems pretty nice. There is a data export option.

My other plan is to set up this:

Figure I have artik up and running so I will give it a month or two.


#71

@desertblade

Ben,

Have you had much luck with Artik?

I synced it up with my ST and the Aeon Labs DSC11-ZWUS, and it collects usage data of my virtual switches just fine.

The thing I can’t figure out is how to have it collect the energy data. It currently only gives me switch on and off data.

I will keep trying, but wanted to see if you have any luck around this.


(Ben W) #72

Have used it for that yet. Use it mostly for temps.

I added my Aeon switch, and there was a “Energy” option in the chart. Does the DH support monitoring KW hours?


#73

Just out of curiosity, in the “Add Device” dashboard for Artik cloud, what did you add the Aeon switch as?

I ask, becuase I don’t see the check box for energy.

Thank you,
Shawn


(Ben W) #74

It was under Energy Meters.

I am using this: https://www.amazon.com/Aeon-Labs-DSC06106-ZWUS-Z-Wave-Energy/dp/B007UZH7B8

In the Device Handler, it lists wattage/KwH used.


(Hal Rottenberg) #75

Great work!

-hal @ splunk


(Jason Hamilton) #76

Thank you very much let me know if you have any trouble.


(Wilson Lu) #77

Hi there,

I am fairly new in trying to setting up splunk and programming with smart things. I setup a Splunk-light 6.4 instance, configured a new indexer and called it WilsonST. Can you help me understand if I set up the following correctly?

  1. SSL - From your code, this looks to be a boolean value. I used false in this section because I wanted HTTP instead of HTTPS for splunk.
  2. Local - From your code, this also looks to be a boolean value. I inputted True in this section because I want to connect to my splunk services.
  3. So these are my inputs for each of the 5 questions:

Local Server - False
Splunk Hostname/IP - WilsonST
Use SSL - False
Splunk Port - 8088 (default port)
Splunk Authentication Token - KeyEnteredFromSplunk

In the Smart Things logs, I am seeing: java.lang.SecurityException: Endpoint http://WilsonST:8088/services/collector/event is blacklisted. @ line 238

When I try to visit http://WilsonST:8088/services/collector/event, it doesn’t work either. Can you provide any assistance on what I am doing incorrectly please?

Thank you,


(Jason Hamilton) #78

You are correct that those items are boolean items. I put in the ssl request per request since initially we were sending the events from ST over the web to the box. But then I looked at how to use the local portion to have the hub just feed the events directly to splunk so that way we don’t have to have the port open on the FW for these events.

So if you have local server set to false and I’m assuming that WilsonST is your splunk server? Should you change local to be true? Also is WilsonST a dns resolvable name? In my case I just plugged in the IP address of the splunk box since I didn’t trust my DNS server :slight_smile: Let me know how I can help you get this working. Thanks.


(Wilson Lu) #79

Hey Jason,

Unfortunately I am not an expert with splunk. (its literately my first time trying to set this up) I was thinking my hostname would be WilsonSt, but that looks more to be the name of my token.

Can you help me understand where my splunk hostname is located?

I tried using my splunk light URL (https://prd-p-2mm2hg3qnhs8.cloud.splunk.com) and that gave me some new errors

Local Server - False
Splunk Hostname/IP - prd-p-2mm2hg3qnhs8.cloud.splunk.com
Use SSL - True
Splunk Port - 8089 (default port)
Splunk Authentication Token - Token value added for WilsonST

Error - org.apache.http.conn.ConnectTimeoutException: Connect to prd-p-2mm2hg3qnhs8.cloud.splunk.com:8089 [prd-p-2mm2hg3qnhs8.cloud.splunk.com/54.175.156.122] failed: connect timed out

Not sure where the address “/54.175.156.122” came from in the error logs.

I also tried the following two links into my firefox browsers to see what would happen:
prd-p-2mm2hg3qnhs8.cloud.splunk.com:8089/services/collector/event
Error: Secure Connection failed

prd-p-2mm2hg3qnhs8.cloud.splunk.com/services/collector/event
Error: 404 not found

BTW: Thank you once again for providing me support. I greatly appreciate it!


(Jason Hamilton) #80

I’m guessing that the 54.175.156.122 is the ip address of the splunk cloud instance.

I myself have a local splunk install. You might want to post on splunk answers and ask what is the best URL for splunk cloud to use the event collector. With a local instance I just point it to the local IP but I’m not sure when it comes to the cloud.

Post in the Splunk Answers and let me know what you get back from there. With the cloud instance it could be a different path which would require another boolean :slight_smile:


(Cpisano) #81

A neat little trick I found when writing code to get all of you event.

If you create your page, similar to this: (where selectedCapabilities = @Field final selectedCapabilities = [ “actuator”, “sensor” ])

page(name: "deviceAuthorization", title: "", nextPage: "devicesPage", install: false, uninstall: true) { section("Select Devices to Authorize") { for (capability in selectedCapabilities) { input name: "${capability}Capability".toString(), type: "capability.$capability", title: "${capability.capitalize()}Things", multiple: true, required: false } }
}  

So when you goto your app, all of your devices will be listed in either the actuator or sensor group. If you then put oauthPage: “deviceAuthorization” as the preference options, you can install the app via OAUTH. With that, you can have your SmartApp isntalled via OAUTH.

So, when you want to subscribe to the events you simply:

selectedCapabilities.each{ capability ->
   settings."${capability}Capability".each { thing ->
        
      if (subscribed[thing.id]) {
        return
      }
      subscribed[thing.id] = true
      thing.supportedAttributes.each { attribute ->
        log.debug "subscribe to attribute ${attribute.name}"
        subscribe thing, attribute.name, deviceEventHandler
      }
      thing.supportedCommands.each { command ->
        log.debug "subscribe to command ${command.name}"
        subscribeToCommand thing, command.name, deviceEventHandler
      }
      log.debug "subscribed to thing ${thing.id}"
    }

}

and you will subscribe to every single event the device throws out and commands as well. (the commands part is nice as you can verify if a command you sent went out) No need to hard code or manually enter all of the attributes. BTW, this also works well when creating generic Web Services.

I must say, much thanks to https://octoblu.com/ as their SmartApp had a lot of this. Just wanted to pass it along as my logging SmartApp is super small in code.