SmartThings Community

SmartThings Data Visualisation using InfluxDB and Grafana

smartapp_misc
smartapp
charts
logging

(codersaur) #1

Hi all,

Here’s a write-up of how I’m using InfluxDB and Grafana to visualise SmartThings device states in charts and dashboards.

The solution uses a custom SmartApp I’ve written to send data from SmartThings to InfluxDB. Crucially it supports the database server being on the same private LAN as the SmartThings hub, so there’s no need to make your database server accessible from the internet.

I’m quite pleased with it so far. In conjunction with my Evohome (Connect) SmartApp I’m getting some great graphs of my home heating behaviour and energy usage.

Here’s an example of a temperature chart:

Here’s an Evohome Heating Zone chart with some overlays:

I aim to post another article covering some of the more-advanced Grafana features like templating and annotations in the next few weeks.

Z.


Sending data to Influxdb
Data Graph / Database?
(Updated) Efergy 2.0 (Connect)
Best way to collect logs from the hub?
GroveStreams SmartThings QuickStart
Z-Wave Smart Energy Meter - 9.99....Really?
[RELEASE] Honeywell Evohome Integration
SmartThings + Numerous = Awesome! (Update: Numerous is out of business)
Multi-room Temperature Logger
Aeon Labs Energy Meter clamps
MySQL Database logger
Can Smartthings data be polled for ingestion into InfluxDB?
Can Smartthings data be polled for ingestion into InfluxDB?
[RELEASE] Iris Smart Plug (3210-L) Zigbee Plug with Z-wave Repeater
Download Events Data
Power Meter Logging SmartApps (PlotWatt, ThingSpeak, SmartEnergyGroups)
Is anyone putting multisensors in every room, and gathering/using mean data from it all?
Log events to Google Sheets [see post 154 for current Github repo and v1.1]
[RELEASE] NST Manager v5.0
Aeontec Home Energy Monitor ZW095 - What do you use to monitor?
[RELEASE] Iris Smart Plug (3210-L) Zigbee Plug with Z-wave Repeater
[RELEASE] InfluxDB Logger
Summary of Power Usage?
Best/Cheapest Outlet for SmartThings?
Aeon Home Energy Meter - daily usage and cost summary
Log events to Google Sheets [see post 154 for current Github repo and v1.1]
Feature Request: Light, plug, switch monitoring
Cheapest zwave/zigbee plug to monitor energy usage?
Graphing from Smartsense Multi
(John) #2

Thanks for the write-up! This is great.


(Michael Hess) #3

Finally got around to setting this up, things are working well so far. I have one issue, any device with an ’ gives and influxdb error within grafana. Is there a way around that or will I need to reconsider my device naming conventions? For example Mike’s Office Light, can’t add to any of the graphs.

This is awesome though, I’ll replace both my Google Sheets and ThingSpeak logging with this!


(Anthony S.) #4

Yes thanks @zcapr17 for throwing all of this together. You saved me a lot of time I would have needed to try and learn another piece of software…

I do have a question because I suck at graphing.

What do you suggest for displaying motion or switch patterns?


#5

@zcapr17 Thanks for sharing this. This is freaking awesome and it’s saving me a few dollars a month from not having to use grovestream.


(tobfaced) #6

This is cool. I’m doing something similar with Elasticsearch+Kibana/Grafana hosted on a Raspberry Pi v2.

Things can be a real pain to get running on that platform, so if you’re looking for hints on using ELK and/or Grafana on a Pi, check out my new blog intranetofstuff.com.

Cheers.


(Mike) #7

The graphs look really cool. Do I understand correctly that the InfluxDB/Grafana approach basically requires a linux server (at home or online)? It’s not a Windows approach?

Thanks,

Mike


(Michael Hess) #8

Correct. I just run Ubuntu in a virtualbox instance on my primary computer that is on 24/7. You can use a pi as well.


(Matthew) #9

You could look in to installing natively or running Docker on your windows machine, but it does require the machine to be on 24/7.

I actually just set this up using Docker on my Synology, since it’s already on. Great writeup, @michaelahess. I’ve got some fine tuning to do, and want to start doing more monitoring points but it’s up and running.
Progress


(Michael Hess) #10

Wait, what did I write up?

And how’d you get this online with raintank.io, I’m very confused right now.


(Matthew) #11

Sorry, meant to thank @zcapr17 for the writeup.

As to raintank.io, that’s just a screenshot - Raintank is the umbrealla over Grafina.

Sorry for the confusion!


(tobfaced) #12

I take it you’re asking about “motion or switch patterns” in that these things are a binary state, either on (motion) or off (no motion).

I have graphed this case in Grafana:

  1. A switch’s status is fed into the database (I use ElasticSearch) with a status of 1 (on) or 0 (off). The same could be done with a motion sensor. This is the first step, getting data into your database.

  2. Put that data series on a graph. By default Grafana creates a histogram. Change the metric from “count” to “max” based on the field that represents the event value in your data (i.e., the field that has a 1 or 0 in it). You use “max” here so that for whatever aggregation bucket Grafana puts your data into the value will only reach “1” for “on” or “motion” because values greater than 1 don’t have any meaning (i.e., if you used “sum”)

  3. At this point Grafana gives you a saw-tooth graph pattern as your data swings between 0 and 1. You can fix this in the Graph/Display tab by selecting the “Staircase” option.

  4. Clean up the Y axis. Grafana supports 2 Y-axes on each graph, this assumes you leave it on the Left Y axis. I often put this binary series on the right Y axis instead so I can graph something else against it (like house fan on/off graphed against temperature). On the Graph/Axes tab, set Y-min and Y-max to 0 and 1, respectively for your desired Y axis. Optionally set a label for the axis and deselect “Show”.

The result is a clean graph of on/off or motion/no motion data.


(codersaur) #13

Sorry for not answering this question, I’ve been away for some time. @tcjennings answer is spot on.

Note, my InfluxDB Logger SmartApp will already create valueBinary for any pretty much any attribute that can be represented as a binary value. You just need to graph the series using the Staircase option. To achieve the Window Function / Optimisation / Is Heating overlays in my original post (above) I put the binary values on a second axis to get the overlay effect.


(Michael Hess) #14

I’m graphing door open/close stuff but it seems closed doors are a 1 and open a 0. Is there an easy way to reverse that?


(codersaur) #15

Yes, you can just change it in the handleEvent() method of the InfluxDB Logger SmartApp.

Depending on exactly which type of sensor you are using you’ll just need to change around the calculation of the binary values for the appropriate event (i.e. switch ‘1i’ and ‘0i’):

	else if ('contact' == evt.name) { // contact: Calculate a binary value (closed = 1, open = 0)
		unit = 'contact'
		value = '"' + value + '"'
		valueBinary = ('closed' == evt.value) ? '1i' : '0i'
		data = "${measurement},deviceId=${deviceId},deviceName=${deviceName},groupName=${groupName},unit=${unit} value=${value},valueBinary=${valueBinary}"
	}
	else if ('door' == evt.name) { // door: Calculate a binary value (closed = 1, open/opening/closing/unknown = 0)
		unit = 'door'
		value = '"' + value + '"'
		valueBinary = ('closed' == evt.value) ? '1i' : '0i'
		data = "${measurement},deviceId=${deviceId},deviceName=${deviceName},groupName=${groupName},unit=${unit} value=${value},valueBinary=${valueBinary}"
	}

Note, I decided to make closed == 1 and open == 0 to keep things logically consistent with all the other types of sensor, i.e. 1 == thing being sensed is there, 0 == thing being sensed is not there.


(Peter Cridland) #16

@michaelahess, did you ever since the apostrophe issue? I’m considering trying the InfluxDB route here and I have apostrophes in my device names, so keen to know if you got it to work?


(Michael Hess) #17

I renamed all my devices to not need them. Sucked for a few, but I’ve gotten used to bad grammar now. :slight_smile:


(codersaur) #18

Right all, the apostrophe issue…

I had assumed that the cause of the issue was either InfluxDB not liking tag values with apostrophes or with Grafana not liking them. Well, I just tried to re-create the problem by sticking an apostrophe in a random device name and to my surprise it showed up in InfluxDB and it showed up in Grafana too!

So, can those of you with this issue let me know if your values get into InfluxDB and hence whether the issue is with InfluxDB or Grafana? In either case, what versions of InfluxDB and Grafana are you using?

Meanwhile, as a work-around, you can simply add an additional string replace into the escapeStringForInfluxDB() function in the SmartApp code. This will replace apostrophes with underscores:

private escapeStringForInfluxDB(str) {
	if (str) {
		str = str.replaceAll(" ", "\\\\ ") // Escape spaces.
		str = str.replaceAll(",", "\\\\,") // Escape commas.
		str = str.replaceAll("=", "\\\\=") // Escape equal signs.
		str = str.replaceAll("\"", "\\\\\"") // Escape double quotes.
		str = str.replaceAll("'", "_")  // Escape apostrophes. << Add this line.
	}
	else {
		str = 'null'
	}
	return str
}

HTH


(Michael Hess) #19

When I do this, will it create new devices in the database or are they correlated to something else? When I first set this up, I swear I renamed an outlet and suddenly had two displaying, one with historical and the other current data. Then of course my Grafania queries weren’t working as the name changed.


(codersaur) #20

That will depend on how you’ve set up your queries in Grafana. If you’re grouping by DeviceName then yes, it will appear as new series, as the name will be different. However, if you group by DeviceId, then no, because the DeviceId will still be the same. HTH.