Getting Ubi Sensor Data the Hard Way... Or how I stopped worrying and learned to love the XMLslurper

@chrisb
Not sure what you mean. Yes it is a devicetype, it creates a device that has the sensors from the Ubi.

Temp, humidity, luminance, Air Pressure and Sound Level.

These can then be accessed and updated on Poll()

This device will create tiles to display the data from the Ubi.

Your things are already on ST…

I must be missing something here. I added this as a device type in the IDE. No issues there. Now how do I access this device? Does just adding the devicetype automatically create a device in SmartThings?

This is beautiful. Thanks @pstuart!

No you then need to create a device and select this devicetype either in the ide devices or in the app.

Will this refresh regularly or only on demand update when I refresh app tile ?

It will not update regularly, devices have no built in access to schedules. Smartapps can do this. There is a Polling app

That can be used to get this data updated on a regular basis.

This is sweet. Nice hack job!

Some minor updates to allow sensor data to be logged via my Xively logger.

https://github.com/pstuart/smartthings/blob/master/Get%20Ubi%20Sensors

How do I use this if I have more than one UBI device?

You don’t. Official API calls now support getting all sensor data in json.

Just waiting on Ubi to allow everyone the ability to access the api

What do you use for the device_id @pstuart, I am getting java.lang.NullPointerException: Cannot invoke method writeTo() on null object @ line 137

DeviceID isn’t used, so you can set it to anything unique.

Code’s working for me, is it possible your username and password are wrong? Also, how many ubi’s do you have? If you have more than one, it might be the issue.

Must have had a typo in there somewhere. I double checked all the settings and it is working. Do you have API access for the UBI yet? I have requested it, but have not heard anything back from them.

Yes, I have it, waiting to release code until others will have it. PM me an email address, and I can set you up with the API code and see if it will work using my API access.

I’m getting the same “java.lang.NullPointerException: Cannot invoke method writeTo() on null object @ line 137” error and I suspect you’re correct about the multiple Ubis (I have three).

Ok, making some progress…

Had to change line 137 from:

doc[0].children[1].children[5].children[1].writeTo(x)

To:

doc[0].children[1].children[6].children[1].writeTo(x)

to adjust for restructured page.

Also made the following change to cater for the fact that I have the Ubi Portal report in Celsius rather than Fahrenheit (and again a slightly different page structure):

// Following was original code - based on Ubi Portal reporting temps in Fahrenheit
//        v.temp = linesAsList[6].minus("document.write(toFahrenheit(").minus("));") //temp in C
//      v.temp = Math.round((Double.parseDouble(v.temp) * 1.8 + 32)*100) /100 // Temp in F

// This is my code - based on Ubi Portal reporting temps in Celsius and wanting to report in Celsius
        v.temp = linesAsList[4].minus("document.write(").minus(".toFixed(2));") //temp in C
        v.temp = Math.round(Double.parseDouble(v.temp))

Note this is only reporting sensors for the first Ubi listed in the Portal…more kludging to come…

As is typical for this type of screen-scraping - it’s very sensitive to changes at the Ubi Portal. It will be good to have a proper API based approach :slight_smile:

I installed both the device type(shows up in Things on app) and the SmartApp which also shows up.
The problem is that on the app after the authentication which is successful when I press “Next” it doesn’t show up anything.In the Connect smartapp code in the final line there is a clientid.Do I have to change that there?

Also when you copy-paste the code it gives you an error.You go on App Setting s and enable OAuth so as to be published.
Am I missing sth?

@DarcRanger - Here is my version of @pstuart’s ps_Ubi which has been modified to support multiple Ubis.

Once you have the device type installed and published, you create multiple Ubi devices, one for each Ubi you have, so each Ubi becomes a “thing”. When you configure each device you need to specify which “Ubi number” it is (1, 2, 3, …), based upon the order in which it appears in the Ubi portal.

Note that I’ve also changed the code to work with temperatures in Celsius rather than Fahrenheit (we use the metric system here downunder). If you need Fahrenheit then hopefully you can see what needs changing back in the code.

/**
 *  sn_Ubi
 *
 *  Portions Copyright 2014 patrick@patrickstuart.com
 *  Portions Copyright 2014 Steven Newnham
 *
 *  Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except
 *  in compliance with the License. You may obtain a copy of the License at:
 *
 *      http://www.apache.org/licenses/LICENSE-2.0
 *
 *  Unless required by applicable law or agreed to in writing, software distributed under the License is distributed
 *  on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License
 *  for the specific language governing permissions and limitations under the License.
 *
 *  Based upon ps_Ubi by Patrick Stuart
 *  Changes support multiple Ubis and use of temperatures in Celsius
 *
 */

preferences {
    input("username", "text", title: "Username", description: "Your Ubi Portal username (usually an email address)")
    input("password", "password", title: "Password", description: "Your Ubi password")
    input("ubinumber", "number", title: "Ubi number", description: "Which Ubi is this in the Portal (1, 2, 3, etc.)")
    }
    
metadata {
    definition (name: "sn_Ubi", namespace: "snewnham", author: "Steven Newnham") {
        capability "Polling"
        capability "Relative Humidity Measurement"
        capability "Illuminance Measurement"
        capability "Temperature Measurement"
        
        
        attribute "airPressure", "string"
        attribute "soundLevel", "string"
    }

    simulator {
    
    }

    // Note: Value entries for background colours changed to match Celsius scale
    //       Not a direct Fahrenheit to Celsius conversion as original values
    //       didn't make much sense at the high end
    tiles {
        valueTile("temperature", "device.temperature", width: 1, height: 1, canChangeIcon: false) {
            state("temperature", label: '${currentValue}°', unit:"C", backgroundColors: [
                    [value: 0, color: "#153591"],
                    [value: 7, color: "#1e9cbb"],
                    [value: 15, color: "#90d2a7"],
                    [value: 25, color: "#44b621"],
                    [value: 30, color: "#f1d801"],
                    [value: 35, color: "#d04e00"],
                    [value: 40, color: "#bc2323"]
                ]
            )
        }
        valueTile("humidity", "device.humidity", inactiveLabel: false) {
            state "default", label:'${currentValue}%', unit:"Humidity"
        }
        valueTile("illuminance", "device.illuminance", inactiveLabel: false) {
            state "luminosity", label:'${currentValue}lux', unit:"Light"
        }
        valueTile("airPressure", "device.airPressure", inactiveLabel: false) {
            state "default", label:'${currentValue}Kpa', unit:"Air Pressure"
        }
        valueTile("soundLevel", "device.soundLevel", inactiveLabel: false) {
            state "default", label:'${currentValue}db', unit:"Sound"
        }
        standardTile("refresh", "device.poll", inactiveLabel: false, decoration: "flat") {
            state "default", action:"polling.poll", icon:"st.secondary.refresh"
        }
        
        
        main "temperature"
        details(["temperature", "humidity", "illuminance", "airPressure", "soundLevel", "refresh"])
    }
}

// parse events into attributes
def parse(String description) {
    log.debug "Parsing '${description}'"
    // TODO: handle 'humidity' attribute
    // TODO: handle 'illuminance' attribute
    // TODO: handle 'temperature' attribute
    // TODO: handle 'carbonMonoxide' attribute

}

// handle commands
def poll() {
    // log.debug "Executing 'poll'"
    
    // TODO: handle 'poll' command
    login()
}




def login() {   
    def params = [
        uri: 'https://portal.theubi.com/login.do',
        headers: [
        'Content-Type': 'application/x-www-form-urlencoded',
        'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8',
        'Accept-Encoding': 'sdch',
        'Host': 'portal.theubi.com',
        'DNT': '1',
        'Origin': 'https://portal.theubi.com/login.jsp',
        'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/28.0.1500.95 Safari/537.36'
            ],
        body: [j_username: username, j_password: password]
    ]

    data.cookies = ''

    httpPost(params) { response ->
        log.debug "Request was successful, $response.status"
        //log.debug response.headers
        response.getHeaders('Set-Cookie').each {
            String cookie = it.value.split(';')[0]
            //log.debug "Adding cookie to collection: $cookie"
            data.cookies = data.cookies + cookie + ';'
        }
        //log.debug "Cookies: $data.cookies"

    }
       

    def params2 = [
        uri: "https://portal.theubi.com/room/",
        headers: [
          'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8',
          'Accept-Encoding': 'plain', 
          'Cookie': data.cookies        
          ],
        ]

    httpGet(params2) { response2 ->
        log.debug "Request was successful, $response2.status"
        def doc = response2.data
        def x = new StringWriter()
        doc[0].children[1].children[6].children[1].writeTo(x)
        def linesAsList = x.toString().minus(" ").split( /\n|\r|\n\r|\r\n/ ).collect { it.replace( "'", '' ) }
//      log.debug linesAsList
        def v = [:]
        
        // Calculate an offset into the list of lines based upon which Ubi we're after
        int ubiOffset = (ubinumber.toInteger() - 1) * 15;
        
// Following was original code - based on Ubi Portal reporting temps in Fahrenheit
//        v.temp = linesAsList[6].minus("document.write(toFahrenheit(").minus("));") //temp in C
//      v.temp = Math.round((Double.parseDouble(v.temp) * 1.8 + 32)*100) /100 // Temp in F

// This is my code - based on Ubi Portal reporting temps in Celsius and wanting to report in Celsius
        v.temp = linesAsList[(ubiOffset + 4)].minus("document.write(").minus(".toFixed(2));") //temp in C
        v.temp = Math.round(Double.parseDouble(v.temp))

        v.humidity = Double.parseDouble(linesAsList[(ubiOffset + 11)].minus("Humidity").minus("%")) // humidity
        v.airPressure = linesAsList[(ubiOffset + 12)].minus("Air Pressure").minus("KPa") //Air Pressure
        //v.airPressure = Double.parseDouble(v.airPressure) / 100// into bar
        v.soundLevel = Double.parseDouble(linesAsList[(ubiOffset + 13)].minus("Sound level").minus("dB")) //Sound Level
        v.lightLevel = Double.parseDouble(linesAsList[(ubiOffset + 14)].minus("Light level").minus("lux")) //Light Level
        log.debug v
        
        sendEvent(name: 'temperature', value: v.temp, unit: "C")
        sendEvent(name: 'humidity', value: v.humidity)
        sendEvent(name: 'illuminance', value: v.lightLevel, unit: "lux")
        sendEvent(name: 'airPressure', value: v.airPressure.toString())
        sendEvent(name: 'soundLevel', value: v.soundLevel)
        
        }
    
}

You should be aware that this is screen-scraping the Ubi Portal HTML page, so it has a high likelihood of breaking in the future as changes occur to the Ubi Portal website. Also, each Ubi device will fetch a copy of the page, so it’s not the most efficient.

@pstuart has an API based approach in the works, pending the release of the API functionality by the Ubi powers that be - that will be a much better long term approach once it becomes available.

Which app are you talking about? If you’re talking about the “Ubi (Connect)” app in @pstuart’s Github repository, you don’t need that to use the ps_Ubi device type.

Once you’ve installed and published the ps_Ubi device type via the IDE you can create a device based upon it. When you configure it you need to provide your Ubi portal UserID and password. The device works by logging in to the Ubi portal and scraping the sensor data from the HTML page.

Once the Ubi thing is in your list of things, you can use it like any other SmartThings sensor - e.g. it will appear as a temperature sensor and a hygrometer (humidity sensor) in the ActiON (Web Dashboard) smartapp, etc. You can open the Ubi thing in your phones SmartThings app to see the various sensor readings available (temperature, humidity, illuminance, air pressure, sound level).

I had to change the code for it to work for me - most likely because I have more than one Ubi whilst @pstuart only has one at the moment. The layout of the HTML page from the portal didn’t match the original code for me. My changed code is posted in a prior post on this thread.

My UBI died. Support has been less then supportive. Trying another resurrect tonight, but no real way to factory reset this thing.

Getting pretty fed up with the device. Not ready for prime time.