Succeeded to write a device handler parsing my arduino humidity and temperature webpage

Hi, just sharing the implementation of the devicehandler I wrote in order to parse the webpage my Arduino is feeding with humidity and temperature measurements.
I struggled with the HTTP request despite the multiple posts in this forum so I ended-up adapting a working HTTP request from [OBSOLETE] URI Switch -- Device Handler for controlling items via HTTP calls (thanks to the submitters @surge919 and @tguerena).
Then I faced difficulties to understand how to parse the webpage the GET received. I initially used html parsing example but this just returned the entire website content as a single string which wasn’t easy to parse.
I then moved to xmlparser and then extracted the content which I could browse and extract easily my 2 numbers.
Here is my website HTML:

<html>
<head>
    <title>Temperature and Humidity Sensors</title>
</head>
    <body>
        <h1>Temperature and Humidity readings</h1>
    <table border="0" cellspacing="0" cellpadding="4">
      <tr>
            <td>ID</td>
            <td>Timestamp</td>
            <td>Temperature</td>
			<td>Humidty</td>
      </tr>
      
<tr><td>29066</td><td>2016-12-09 14:55:05</td><td>21.60</td><td>48.10</td></tr>    </table>
    </body>
</html>
```
This was parsed as:

html[ attributes={};
value=[
head[
attributes={};
value=[
title[
attributes={};
value=[
Temperature and Humidity Sensors
]]]],
body[
attributes={};
value=[
h1[
attributes={};
value=[
Temperature and Humidity readings
]],
table[
attributes={border=0, cellspacing=0, cellpadding=4};
value=[
tr[
attributes={};
value=[
td[
attributes={};
value=[
ID
]],
td[
attributes={};
value=[
Timestamp
]],
td[
attributes={};
value=[
Temperature
]],
td[
attributes={};
value=[
Humidty
]]]],
tr[
attributes={};
value=[
td[
attributes={};
value=[29075]
],
td[
attributes={};
value=[
2016-12-09 15:40:23
]],
td[
attributes={};
value=[21.50]
],
td[
attributes={};
value=[47.70]
]]]
]]]]]]

I could then refer to my temperature and humidity values by the below hierarchy and convert into a float with the needed precision:
    log.debug "Temperature: ${html.body.table.tr[1].td[2].text()}"
    log.debug "Humidity: ${html.body.table.tr[1].td[3].text()}"
    def temp_float = html.body.table.tr[1].td[2].text()
    def humid_float = html.body.table.tr[1].td[3].text()
    
   
    
	sendEvent(name: "temperature", value:  temp_float.toString().format( java.util.Locale.US,"%.1f", temp_float.toFloat()))
	sendEvent(name: "humidity", value: humid_float.toString().format(java.util.Locale.US,"%.1f", humid_float.toFloat()))
    }

The complete code of my devicehandler is below:

/**
* Temperature_Humidity_sensor
*
* Copyright 2016
*
* Licensed under the Apache License, Version 2.0 (the “License”); you may not use this file except
* in compliance with the License. You may obtain a copy of the License at:
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software distributed under the License is distributed
* on an “AS IS” BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License
* for the specific language governing permissions and limitations under the License.
*
*/
preferences {

	section("Internal Access"){
		input "internal_ip", "text", title: "Internal IP", required: false
		input "internal_port", "text", title: "Internal Port (if not 80)", required: false
		input "internal_query_path", "text", title: "Internal query Path (/yourphppage.php)", required: false
	}
}

metadata {
    definition (name: "Humidity_Temperature", namespace: "xxxxx", author: "xxxxxxxxxx", category: "C2") 
    {
            capability "refresh"
            capability "polling"
            capability "capability.temperatureMeasurement"
            capability "capability.relativeHumidityMeasurement"
	}

tiles(scale: 2) {
   multiAttributeTile(name:"temperature", type:"generic", width:6, height:4) {
         // value tile (read only)
    tileAttribute("device.humidity", key: "PRIMARY_CONTROL") {
        attributeState("default", label:'${currentValue}%', unit:"%", backgroundColors:[
            [value: 40, color: "#153591"],
            [value: 50, color: "#1e9cbb"],
            [value: 60, color: "#90d2a7"],
            [value: 70, color: "#44b621"],
            [value: 80, color: "#f1d801"],
            [value: 90, color: "#d04e00"],
            [value: 95, color: "#bc2323"]])
   		}
   tileAttribute("device.temperature", key: "SECONDARY_CONTROL") {
        attributeState("default", label:'${currentValue}Âş', unit:"dC")
    	}
   
    
}
	standardTile("tempdetail", "device.temperature", width: 2, height: 2, canChangeIcon: false) {
            state "default", label: '${currentValue}Âş', unit:"dC", 
                  icon: "st.Weather.weather2", backgroundColors:[
            [value: 10, color: "#153591"],
            [value: 15, color: "#1e9cbb"],
            [value: 18, color: "#90d2a7"],
            [value: 20, color: "#44b621"],
            [value: 22, color: "#f1d801"],
            [value: 25, color: "#d04e00"],
            [value: 28, color: "#bc2323"]]
        }
	standardTile("listView", "device.humidity", width: 2, height: 2, canChangeIcon: false) {
            state "default", label: '${currentValue}%', 
                  icon: "st.Weather.weather12", backgroundColors:[
            [value: 40, color: "#153591"],
            [value: 50, color: "#1e9cbb"],
            [value: 60, color: "#90d2a7"],
            [value: 70, color: "#44b621"],
            [value: 80, color: "#f1d801"],
            [value: 90, color: "#d04e00"],
            [value: 95, color: "#bc2323"]]
        }

 	standardTile("refresh", "device.thermostatMode", decoration: "ring", width: 2, height: 2) {
 		state "default", action:"refresh", icon:"st.secondary.refresh"
 		} 

	main("listView")

 	}
}

def installed() {
log.debug "Executing 'installed'"
        
}

def updated() {
log.debug "Executing 'updated'"
    refresh()
}

def poll(){
log.debug "Executing 'poll'"
    refresh()
}

def parse(description) {
	log.debug "Executing 'parse'"
    def msg = parseLanMessage(description)
    log.debug msg.body
	if (msg.status == 200)
    {
    
    def xmlParser = new XmlParser()
    def html = xmlParser.parseText(msg.body)
    assert html instanceof groovy.util.Node 

	log.debug "Temperature: ${html.body.table.tr[1].td[2].text()}"
    log.debug "Humidity: ${html.body.table.tr[1].td[3].text()}"
    def temp_float = html.body.table.tr[1].td[2].text()
    def humid_float = html.body.table.tr[1].td[3].text()
    
   
    
	sendEvent(name: "temperature", value:  temp_float.toString().format( java.util.Locale.US,"%.1f", temp_float.toFloat()))
	sendEvent(name: "humidity", value: humid_float.toString().format(java.util.Locale.US,"%.1f", humid_float.toFloat()))
    }
 
}



private getCallBackAddress() {
    log.debug "Hub address" 
    log.debug device.hub.getDataValue("localIP") + ":" + device.hub.getDataValue("localSrvPortTCP")
	return device.hub.getDataValue("localIP") + ":" + device.hub.getDataValue("localSrvPortTCP")
}

// gets the address of the device
private getHostAddress() {
    def ip = getDataValue("ip")
    def port = getDataValue("port")

    if (!ip || !port) {
        def parts = device.deviceNetworkId.split(":")
        if (parts.length == 2) {
            ip = parts[0]
            port = parts[1]
        } else {
            log.warn "Can't figure out ip and port for device: ${device.id}"
        }
    }

    log.debug "Using IP: $ip and port: $port for device: ${device.id}"
    return convertHexToIP(ip) + ":" + convertHexToInt(port)
}

private Integer convertHexToInt(hex) {
	log.debug hex
    return Integer.parseInt(hex,16)
}

private String convertHexToIP(hex) {
	log.debug hex
    return [convertHexToInt(hex[0..1]),convertHexToInt(hex[2..3]),convertHexToInt(hex[4..5]),convertHexToInt(hex[6..7])].join(".")
}

private String convertIPtoHex(ipAddress) { 
    String hex = ipAddress.tokenize( '.' ).collect {  String.format( '%02x', it.toInteger() ) }.join()
    log.debug "IP address entered is $ipAddress and the converted hex code is $hex"
    return hex

}

private String convertPortToHex(port) {
	String hexport = port.toString().format( '%04x', port.toInteger() )
    log.debug hexport
    return hexport
}

def refresh() {
log.debug "Executing refresh"

    def host = internal_ip 
    def port = internal_port
    def hosthex = convertIPtoHex(host)
    def porthex = convertPortToHex(port)
    device.deviceNetworkId = "$hosthex:$porthex" 
    
    log.debug "The device id configured is: $device.deviceNetworkId"
    
    def path = internal_query_path
    log.debug "path is: $path"
    
    def headers = [:] 
    headers.put("HOST", "$host:$port")
 
  try {
    def hubAction = new physicalgraph.device.HubAction(
    	method: "GET",
    	path: path,
    	headers: headers
        )
        	
   
    // log.debug hubAction
    return hubAction
    
    }
    catch (Exception e) {
    	log.debug "Hit Exception $e on $hubAction"
    }
    
}


<img src="//cdck-file-uploads-global.s3.dualstack.us-west-2.amazonaws.com/smartthings/original/3X/8/4/849dafb4aa8eaab2cc70c43103cb252f093c7724.jpg" width="312" height="500">

<img src="//cdck-file-uploads-global.s3.dualstack.us-west-2.amazonaws.com/smartthings/original/3X/5/3/5384ff04cfa47da1269f85cf4dea10f3511fab85.jpg" width="312" height="500">

<img src="//cdck-file-uploads-global.s3.dualstack.us-west-2.amazonaws.com/smartthings/original/3X/1/6/16c1cbb3108219e08109538827602b5b4dc289a9.jpg" width="312" height="500">

<img src="//cdck-file-uploads-global.s3.dualstack.us-west-2.amazonaws.com/smartthings/original/3X/4/d/4d5f72db4bfae5b8eeab0fd3ce14c5cfeb08d198.jpg" width="312" height="500">

Hope it helps!
5 Likes

that looks good! how does the hardware look like? How did u install it?is it next to a plug? What sensor are you using for getting the temperature and humidity? I have done similar projects but always had some challenges taking measurements. I had to compare readings with other devices and try to calibrate the sensors in my software. Usually i use the HTU21D for temperature/humidity readings.

Thanks!
The hardware is based on Arduino Uno+ethernet shield+groove DHT22 which is way more accurate than the normal version DHT11, however, I am not doing specific calibration on this one as the results were pretty aligned with another device I have. It is then wired to a RJ45 plug and powered by a power adapter.

My setup is basically having the Arduino posting to my NAS hosted MySQL database the polled temperature and humidity (the below scripts are inspired from diverse internet examples on instructables.com or Arduino various forums, sorry for not greeting the authors, I didn’t plan to publish them originally).

#include "DHT.h"
#include <SPI.h>
#include <Ethernet.h>


    // Ethernet parameters and client config 
byte mac[] = { 0xDE, 0xAD, 0xBE, 0xEF, 0x12, 0x43 };
IPAddress ip(192, 168, 1, 54);

IPAddress NAS(192,168,1,4);

// fill in your Domain Name Server address here:
IPAddress myDns(192, 168, 1, 1);
IPAddress Subnet(255,255,255,0); 

EthernetClient client;

char NASServer[] = "192.168.1.4";
unsigned long lastConnectionTime = 0;             // last time you connected to the server, in milliseconds

//-----------------------------------------------------------------
// Temp and humidity sensor defines
#define DHTPIN 2     // what digital pin we're connected to

// Uncomment whatever type you're using!
//#define DHTTYPE DHT11   // DHT 11
#define DHTTYPE DHT22   // DHT 22  (AM2302), AM2321
//#define DHTTYPE DHT21   // DHT 21 (AM2301)

// Connect pin 1 (on the left) of the sensor to +5V
// NOTE: If using a board with 3.3V logic like an Arduino Due connect pin 1
// to 3.3V instead of 5V!
// Connect pin 2 of the sensor to whatever your DHTPIN is
// Connect pin 4 (on the right) of the sensor to GROUND
// Connect a 10K resistor from pin 2 (data) to pin 1 (power) of the sensor

// Initialize DHT sensor.
// Note that older versions of this library took an optional third parameter to
// tweak the timings for faster processors.  This parameter is no longer needed
// as the current DHT reading algorithm adjusts itself to work on faster procs.
DHT dht(DHTPIN, DHTTYPE);

void httpRequest(float temp,  float humidity) {
  // close any connection before send a new request.
  // This will free the socket on the WiFi shield
  client.stop();

    Serial.println("connecting to NAS");

  if (client.connect(NASServer, 80)){
    Serial.println("Connected to NAS");

    client.print ("GET /write_data.php?");
    client.print ("temp=");
    client.print (temp);
    client.print ("&&humid=");
    client.print (humidity);
    client.println(" HTTP/1.1");
    client.println("Host: 192.168.1.4");
    client.println("Connection: close");
    client.println();
    client.println();
    client.stop();
      Serial.println("End of NAS connection");
  }
  else {
    // if you couldn't make a connection:
    Serial.println("connection failed");
  }

  Serial.println("Reading answer...");
  while (client.connected()) {
    while (client.available()) {
      char c = client.read();
      Serial.print(c);
    }
  }
  Serial.println("");
}


void setup()
{

 dht.begin();

 // start serial port:
  Serial.begin(9600);
  Serial.println("--- Start ---");
  
  // give the ethernet module time to boot up:
  delay(5000);

 // give the ethernet module time to boot up:
  delay(1000);
  // start the Ethernet connection using a fixed IP address and DNS server:
  Serial.println("Ethernet.begin");

  while (Ethernet.begin(mac)==0)
  {
    delay(2000);
    Serial.println("Ethernet. dhcp failed");
    
    Ethernet.begin(mac);

    Serial.print("IP ");
    Serial.println(Ethernet.localIP());
   
  }
  Serial.print("IP ");
  Serial.println(Ethernet.localIP());

}
 
void loop()
{
 
  // Wait 5 minutes before every record
  delay(5*60000);

  // renew DHCP lease
  Ethernet.maintain(); 

  // Reading temperature or humidity takes about 250 milliseconds!
  // Sensor readings may also be up to 2 seconds 'old' (its a very slow sensor)
  float h = dht.readHumidity();
  // Read temperature as Celsius (the default)
  float t = dht.readTemperature();
  // Read temperature as Fahrenheit (isFahrenheit = true)
  float f = dht.readTemperature(true);


  // Check if any reads failed and exit early (to try again).
  if (isnan(h) || isnan(t) || isnan(f)) {
    Serial.print("Failed to read from DHT sensor!");
    return;
  }

  httpRequest(t,h);

}

As you see this calls a php script to insert the data in my MySQLDatabase (I keep logging things for later usage): write_data.php

<?php

    // Prepare variables for database connection
   
    $dbusername = "mysqlusename";  // enter database username, I used "arduino" in step 2.2
    $dbpassword = "mysqlpassword";  // enter database password, I used "arduinotest" in step 2.2
    $server = "localhost"; // IMPORTANT: if you are using XAMPP enter "localhost", but if you have an online website enter its address, ie."www.yourwebsite.com"

    // Connect to your database

    $dbconnect = mysql_pconnect($server, $dbusername, $dbpassword);
    $dbselect = mysql_select_db("mydbname",$dbconnect);

    // Prepare the SQL statement

    $sql = "INSERT INTO test.sensor (temp, humid) VALUES ('".$_GET["temp"]."', '".$_GET["humid"]."')";    

    // Execute SQL statement

    mysql_query($sql);

?>

Then the smartthings devicehandler query the last inserted data via another php script: get_lastdata.php

<?php
$url=$_SERVER['REQUEST_URI'];
header("Refresh: 5; URL=$url");  // Refresh the webpage every 5 seconds
?>
<html>
<head>
    <title>Temperature and Humidity Sensors</title>
</head>
    <body>
        <h1>Temperature and Humidity readings</h1>
    <table border="0" cellspacing="0" cellpadding="4">
      <tr>
            <td>ID</td>
            <td>Timestamp</td>
            <td>Temperature</td>
			<td>Humidty</td>
      </tr>
      
<?php
    // Connect to database
   $dbusername = "mysqlusername";  // enter database username
    $dbpassword = "mysqlpassword";  // enter database password
    $server = "localhost"; // IMPORTANT: if you are using XAMPP enter "localhost", but if you have an online website enter its address, ie."www.yourwebsite.com"
    $dbname = "mydbname";

   // IMPORTANT: If you are using XAMPP you will have to enter your computer IP address here, if you are using webpage enter webpage address (ie. "www.yourwebpage.com")

    $con = new mysqli($server, $dbusername, $dbpassword, $dbname);

    if ($con->connect_error) {
    		die("Connection failed: " . $con->connect_error);
	} 
       
    // Retrieve all records and display them   
    $result = $con->query("SELECT * FROM sensor ORDER BY id DESC LIMIT 1");
      
    // Process every record
    
    while($row = $result->fetch_assoc())
    {      
        echo "<tr>";
        echo "<td>" . $row['id'] . "</td>";
        echo "<td>" . $row['time'] . "</td>";
        echo "<td>" . $row['temp'] . "</td>";
	echo "<td>" . $row['humid'] . "</td>";
        echo "</tr>";
        
    }
        
    // Close the connection   
    mysqli_close($con);
?>
    </table>
    </body>
</html>

That’s all it does.

1 Like

pretty cool, very nice implementation!