Automatically Capture Every SmartThings Event and Store it Forever (InitialState SmartApp)

smartapp_misc
logging

(Jamie Bailey) #1

The Initial State SmartApp (if you don’t know what this is, read about it here) now lets you automatically capture every event from supported SmartThings devices and store it forever. The “unlimited data retention” feature was released today and costs $5/month. This lets you use SmartThings like a “black-box” to capture what has happened in your home, something that can be really useful for energy monitoring or analyzing how your home is being utilized. Plus, you can view all of this data in visualizations like dashboards, waveforms, etc. on your laptop or mobile devices.


Download Events Data
Best way to collect logs from the hub?
ROUNDUP: Data Logging and Charting Solutions
[BETA RELEASE ROUND 2] Curb Energy Monitor Integration
(Bruce) #2

Can it capture and record the log.debug messages?


(Jamie Bailey) #3

No, ST doesn’t have the ability to extend their logging in the SmartApp SDK. However, another SmartApp developer could certainly send their own data to their Initial State account if they wanted.


#4

Just signed up! Looks great so far…


(Bruce) #5

Could you elaborate on this? One problem I run into is that sometimes things fail for some reason in the middle of the night, and looking back at device logs doesn’t reveal enough. These issues are with apps I have written (although most likely an ST issue), so writing log-like info to an external service has some appeal.


(sidjohn1) #6

Nope, basically if a smartapp can subscribe to it (open/close, motion, temp, acceleration, lock/unlock, humidity) Initialstate can present it over time


(sidjohn1) #7

Instead of sending event / errors to log.debug you can

sendEvent(linkText:app.label, name:"${evt.displayName} ${evt.name}", value:"${evt.value}",descriptionText:"${evt.displayName} ${evt.name}: ${evt.value}", eventType:"SOLUTION_EVENT", displayed: true)

It makes it easier to TS at a later time… up to 7 days :wink:


(Bruce) #8

Ah, cool. That’s a very good idea.


(Matthew) #9

Have this setup. Only started with my Aeon Multisensor. That worked great. However now I have added more devices and in the IDE Log I see that things keep getting added to the buffer for processing, but I now see this error in the log:
shipping events failed: net.sf.json.JSONException: There is a cycle in the hierarchy!

Any ideas? I am using the cloud to cloud app. I am going to try the DIY app that submits values as they happen.


(David Sulpy) #10

Hey @mander this is some issue that ST is aware of. Something with the way the JSON is being built and handled inside the httpPost methods available through the ST SmartApp SDK.

The DIY version does not have this problem. Since the DIY version doesn’t buffer events, it can do simple JSON string manipulation as opposed to handling a persistent array of serializable events.

This issue seemed to have just randomly popped up recently anyway.


(Matthew) #11

Thanks. Installing the DIY now. Will see how that works, and maybe someday ST will fix the problem with cloud to cloud, as that would work well for me, and use less resources on their services.


(David Sulpy) #12

@mander yeah, it’d be great to get the official integration fixed and working correctly. However, the DIY version works fundamentally the same way as the automatically installed version (at least, as for where it works in ST ecosystem). The only difference is the removal of buffering which makes the end users experience far superior for real-time nature events and consumes marginally more resources from ST (assuming an environment without hundreds of events a minute).


(ravi) #13

Did you get the DIY solution to work Mander?


(Matthew) #14

Ya it is working. Still playing with what I want to track… I really want multiple dashboards. So I think I need to publish the app for each set of counters.

I will have one dashboard for batteries. I really want to be able to see over time how quickly the different sensors drain. If there is an increase in some of my outside farther away sensors it could mean my mesh is broken and something dropped out.

Another for Temp/Humidity of all my sensors.

And the last for power utilization. Just ordered stuff for whole home as well as Washer/Dryer tracking. The new mini switches I am deploying report that as well.

I wish I could combine some of the features of Initial State with GroveStreams. Both have pluses and minuses.

So far though it looks good. I am struggling a little it seems like the ST platform doesn’t always send counters. Watching live events I can see some that never get picked up by grove app or InitialState app. Also been 24+ hours and a few of my devices haven’t pinged in with their battery usage yet, but they will at some point I am sure.


(David Sulpy) #15

@mander one thing I noticed when I developed this app and found the capability.battery capability, is that different devices don’t send battery levels as events at all. So it’s up to the developer of the device to have implemented battery event capability properly in order to inform the ST platform and therefore a user that batteries are low on a particular device. Likewise, I’ve noticed a lot of the SmartThings branded devices that I have will send really weird battery levels that don’t seem to make sense. For instance, a device will report it’s battery level as 38%, then 0% then 38% and it will oscillate between these two over time.

Here is an example of a SmartSense device that does exactly this:

You can see that instead of a nice line, I get these almost rhythmic spikes where the device is saying it has 38% battery and then saying it has 25%, then 38% and then all of a sudden stopped oscillating and stayed on 25%, but still reports it’s temperature (the line above).

I’ve pinged ST about it once before and they too thought this was odd behavior and mentioned that it should get fixed. But I’m not sure it ever did get fixed.

My point in all of this, is that unless you developed the SmartDevice code for the devices you want to track battery on, you may be disappointed with the battery tracking, I know I have been. The beautiful thing, though, is that I was able to use Initial State to see this problem and how it happens because the Initial State SmartApp simply listens to every event ST will give it and sends those to the visualization service. So, the visualization is a true reflection of what’s going on in my ST network. It provides a little bit of insight into some odd behavior sometimes.

Also, the ability to have different dashboard views of the data is something that we’re working on at Initial State. However, feel free to mention your exact use case in our “Feature Requests” forum of our support site at http://support.initialstate.com/forums/252778-feature-requests

Full Disclosure: I am the author of the SmartApps as well as the Founder and CTO of Initial State.

Cheers,
David


(David Sulpy) #16

Oh, and quick question, what would you like to combine with GroveStreams?


(Matthew) #17

Thanks David. Ya I have seen some flopping on the battery level for a few devices. That makes sense. It could just be the device itself, hard to say.

As for GroveStreams vs Initial State. I would like to be able to build a custom dash in Initial State that would allow me to graph multiple values in the same graph. I really want to see my in house temp/humidity sensors overlayed on the same graph. Which GroveStreams allows. However I really like the ability to drill into the data like Initial State does. Really with Initial State if I could make a dashboard and add the items I want to graph how I want to graph them it would be awesome. Let me know if you want more info.


(David Sulpy) #18

I see! Yeah, we’re working on some new Tiles in our dashboard that could allow just that. Something similar to what we do in the Lines application, which is what I use to check the effectiveness of my heating and cooling distribution across my house (since the SmartSense Multi’s all have temperature measurement capabilities).

Would you want to show different Y-scales for two or more different lines mapped in a graph? i.e. in Lines in Initial State right now, it uses a single Y-scale, but I suppose you could try and define multiple min-max Y-scales for two different measurement units like when comparing temperature and humidity.

I’m glad you enjoy the ability to visually dig into the data in Initial State, that’s exactly what we built the visualizations to do: visual data exploration and mining for analysis.

Anyway, feel free to let us know if you ever have more features you’d like to see or tweaks that would make the product even more interesting to you! We love having engaged users and feedback means a lot to us.

Cheers!


(Matthew) #19

Yes the ability to have to scales. So one thing I am trying to correlate right now that I am not having great luck on (Granted I probably just need to RTFM.) is the to increase the scale of “Heat on” from 1 to something I can see so I can get an idea of how long it currently takes for rooms to heat/cool once the HVAC kicks on. I am installing keen vents soon and want to be able to compare this information from before I put the system in till after. I guess I should also step up and pay for an account so I can get more than 24 hours of trending.

Let me know if you need any beta testers for those new tiles :smile:


(David Sulpy) #20

Ahh, I see exactly what you’re trying to do here. And that’s a really cool idea! One thing that I do when measuring temperature vs heater state is use waves so I can see when turning a small space heater on, I can see how long it takes to heat up a room with the door closed. I only just now started doing this, so I don’t have a lot of data as an example, but here is a zoomed in view of something so far:

You can see that in my screen shot, I’ve snapped my cursors to the edges of the heater switch so that I’ve isolated the time between when it turned on and when it turned off. You can also see that I’ve pulled up some stats on the temperature to understand what’s going on during this period of time. So, for 41.84 min, my heater is on in this room, and during that 41.84 min, my room’s temperature increases by 4 degrees F as indicated by the difference between the Min and Max.

This allows me to see how effective my heater is. Now, like I said, I only started tracking the heater yesterday. And I plan on putting a new switch control on it to also track it’s power consumption so I can understand the cost of heating my office.

Is this kind of what you’re looking to do?

Cheers!