Best way to collect logs from the hub?

crlsmartthings · December 10, 2014, 9:52pm

Hi SmartThings developers,

I know there is no a data exporting tool supported by SmartThings. Do you have any suggestions to harvest logs/events from a hub so that I can do some data analysis? Your answer will help me greatly.

Best,
Joey

Billybnyc · January 8, 2015, 12:01am

Ditto! Anyone creating a log export / capture?

Thanks all,

-B

smart · January 8, 2015, 1:08am

I am doing it physically here in Jersey! Have to get the fireplace going… It’s far below negative with the wind chill… logs help.

frayes · January 28, 2015, 3:37pm

I’m looking to solve the same problem of analysing how often my heater is on.
Any solution to data logging?

klinquist · January 28, 2015, 11:15pm

If you have somewhere that you can run a web server/nodejs script/php file/etc., you can use a smartapp to subscribe to events, then when an event occurs, make an HTTP GET/POST to your server and run the analytics there…

tgauchat · January 29, 2015, 12:03am

Fetch recent events with: https://graph.api.smartthings.com/api/accounts/idHashString/events

Fetch lots (999…) of events with: https://graph.api.smartthings.com/api/accounts/idHashString/events?max=999

Get idHashString after logging into the API and running:

https://graph.api.smartthings.com/api/accounts

…CP / Terry.

btk · January 29, 2015, 2:39am

Is there documentation on this somewhere? I’m having a tough time finding it on the docs site.

tgauchat · January 29, 2015, 2:55am

I was hoping someone would ask that question!

There used to be a “work-in-progress” PDF / Google Doc that had details on a lot of the REST-API endpoints, but it disappeared, and the replacement documentation doesn’t seem to go into that area of detail … unless I am also missing some obvious link. Search of docs didn’t yield result.

@Jim or @Ben or @urman?

I would expect it to be linked from here:
http://docs.smartthings.com/en/latest/smartapp-web-services-developers-guide/index.html

Or here: http://docs.smartthings.com/en/latest/reference-material.html

But, in the meantime, someone made an unofficial copy here:

github.com

tmaiaroto/smartthings-unofficial-docs/blob/master/Documentation.md


# SmartThings Unofficial Documentation

Many of the endpoints (and parameters for them) documented below were discovered via trial and error methods. However, a more complete list of endpoints can be found in this [Titanium project here.](https://github.com/alanleard/ti.smart/blob/master/src/lib/smart.js) However, it should be noted that not all of those seem to work. The endpoints documented here have been tested. No promises things won't change and it should be expected that they will.

## Projects
Projects allow you to setup a community around what you're building for SmartThings. You are given the ability to setup blogs, galleries, and more.

You first need an account on [http://build.smartthings.com](http://build.smartthings.com) which is different than your account for using your device (which is used with http://graph.api.smarthings.com). Then you can go to create a project at the following URL: [http://build.smartthings.com/projects/create/](http://build.smartthings.com/projects/create/)

## SmartApps
SmartApps are the actual applications/widgets which users can install to the SmartThings mobile application. They use OAuth to authenticate and allow access to the API. This allows an end-user to individually disable/control access for any SmartApp. If you're familiar with Facebook apps - this is very similar.

You can create smart apps here: [https://graph.api.smartthings.com/ide/apps](https://graph.api.smartthings.com/ide/apps)

*Note: You may not have access to this section if you are not registered as a developer.*

## JSON API Endpoints
First off, you need to be logged into http://graph.api.smarthings.com … Which is simple enough. You can then go on to see a list of your hubs and events, among other things, in their dashboard. However, we're after JSON responses and all of hte data you see here is available in (seemingly open) JSON responses.

This file has been truncated. show original

…CP / Terry.

rob_a_landry · October 5, 2015, 11:14am

Just curious if this still works. I think i read on the forums that the new oauth flow made this method obsolete. I am trying to pull video feeds from the hub events but having little luck getting the hub events to display.

tgauchat · October 5, 2015, 7:57pm

It still works for me; though the various official alternative methods are worth looking into.

btk · October 6, 2015, 1:41am

I’ve been using a SmartApp I wrote to subscribe to all of my devices and ship the events off to a Logstash server, using Kibana as the front-end. This gets me all of the historical data I want and the ability to make pretty pictures out of it as well.

github.com

bkeifer/smartthings/blob/master/Logstash Event Logger/LogstashEventLogger.groovy

/**
 *  Event Logger
 *
 *  Copyright 2015 Brian Keifer
 *
 *  Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except
 *  in compliance with the License. You may obtain a copy of the License at:
 *
 *      http://www.apache.org/licenses/LICENSE-2.0
 *
 *  Unless required by applicable law or agreed to in writing, software distributed under the License is distributed
 *  on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License
 *  for the specific language governing permissions and limitations under the License.
 *
 */
definition(
    name: "Logstash Event Logger",
    namespace: "bkeifer",
    author: "Brian Keifer",
    description: "Log SmartThings events to a Logstash server",

This file has been truncated. show original

mikenicholson · October 27, 2015, 12:52pm

The links are broken 404 error. Was it removed on purpose?

btk · October 27, 2015, 1:52pm

Nope, just broke during a repo re-org. The link above should work now.

bridaus · November 6, 2015, 1:02pm

Anyone know of a cloud version of Logstash functionality? I’m hoping to inspire, or be inspired to create a cloud logger for the community to catch recent poltergeist issues.

I’m open to other methods as well, I realize you could leave the logger open in ST, but it’s hard to keep that up for history (and it crashes a fair bit lately). You could probably also send debug as an event, or as texts, but I feel like that might punish the platform, and cause unnecessary load.

Some quick research:

https://www.loggly.com/
https://papertrailapp.com/
Will add more as time allows…

awelch02 · November 10, 2015, 9:05pm

Has anyone wrote a SmartApp logger through the SmartThings Web Services directly? It could interface with pretty much any server side language or database of choice.

tgauchat · November 10, 2015, 9:50pm

[OBSOLETE] Automatically Capture Every SmartThings Event and Store it Forever (InitialState SmartApp) SmartApps & Automations

The Initial State SmartApp (if you don’t know what this is, read about it here) now lets you automatically capture every event from supported SmartThings devices and store it forever. The “unlimited data retention” feature was released today and costs $5/month. This lets you use SmartThings like a “black-box” to capture what has happened in your home, something that can be really useful for energy monitoring or analyzing how your home is being utilized. Plus, you can view all of this data in visualizations like dashboards, waveforms, etc. on your laptop or mobile devices. http://blog.initialstate.com/wp-content/uploads/2015/07/Screenshot-2015-07-28-18.11.43-1024x553.png

and

bridaus · November 10, 2015, 11:32pm

I use initial state, but it’s data oriented, not log oriented. I believe Thinglayer is the same (tried it out too before…).

Maybe I’m missing a functionality?

tgauchat · November 10, 2015, 11:48pm

Ohhhhh…

What logs are you trying to capture … the output of Live Logging, or the output of Events?

Not sure if there is any way to continuously stream Live Logging to a collection service. Perhaps some clever Javascript to scrape the output.

As for Events, the two example services can (theoretically?) capture all Events, not just State changes and Value reporting. Message Events are just text values + “descriptionText”, after all.

bridaus · November 11, 2015, 3:06am

I’ve seen a couple of cases where if I could go back a day in live logging, I’d be able to tell what happened.

Even if I had to put lines in code, it would be useful as a trace.

btk · November 11, 2015, 3:23pm

I’m using Logstash on one of my own servers to do this. In any SmartApp where I need to retain history, I add the following method and use it to log messages instead of log.debug():

gist.github.com

https://gist.github.com/bkeifer/2bfc50abb6a8d8ccd74c

stash.groovy

def stash(msg) {
	log.debug(msg)
	def dateNow = new Date()
    // def isoDateNow = dateNow.format("yyyy-MM-dd'T'HH:mm:ss.SSS'Z'", location.timeZone)
    def isoDateNow = dateNow.format("yyyy-MM-dd'T'HH:mm:ss.SSS'Z'")
    def json = "{"
    json += "\"date\":\"${dateNow}\","
    json += "\"isoDate\":\"${isoDateNow}\","
    json += "\"name\":\"log\","
    json += "\"message\":\"${msg}\","

This file has been truncated. show original

I also wrote another SmartApp that subscribes to all of my things and logs their events to the same Logstash server for easier correlation with the log entries:

github.com

bkeifer/smartthings/blob/master/Logstash Event Logger/LogstashEventLogger.groovy

/**
 *  Event Logger
 *
 *  Copyright 2015 Brian Keifer
 *
 *  Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except
 *  in compliance with the License. You may obtain a copy of the License at:
 *
 *      http://www.apache.org/licenses/LICENSE-2.0
 *
 *  Unless required by applicable law or agreed to in writing, software distributed under the License is distributed
 *  on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License
 *  for the specific language governing permissions and limitations under the License.
 *
 */
definition(
    name: "Logstash Event Logger",
    namespace: "bkeifer",
    author: "Brian Keifer",
    description: "Log SmartThings events to a Logstash server",

This file has been truncated. show original

Topic		Replies	Views
Collect logs to external database? SmartApps & Automations	5	3944	February 16, 2015
Data capture from Smartthings sensors Projects & Stories logging	6	1144	August 27, 2022
Live Logging Scraper / Capturing General Discussion	1	1300	April 25, 2016
Daily Event Log Export - Feature Request General Discussion archive_general	8	3462	September 17, 2015
FAQ: Logs and SmartThings - An intro into figuring out what went wrong FAQ log , troubleshoot , ide , livelogging	9	20899	December 14, 2018

Best way to collect logs from the hub?

Related topics