[ST Edge] Web Requestor: a driver to issue local POST and GET HTTP requests

[TAustin]

I’ve pushed out a driver update to the shared channel.

As mentioned previously, this update contains 2 items:

1. To address issues with Settings returning erroneous data to the driver, you must now pause a minimum of 10 seconds between each Settings field update when you are creating and editing your pre-configured web requests. If you try to make changes any faster than that, the change will be ignored by the driver. Unless you are working really fast, I don’t think this delay will affect you too much, but I am hoping it will reduce the cases where your changes don’t seem to be working and you have to go back in and re-edit the web request string.
   In the event that you do experience cases where your web request doesn’t seem to be working, but the URL string should be correct, try the procedure I mentioned in a prior post: change to something else → save it → change it back to what you want → save it → test.
2. Automations for POST or GET URL commands using custom (as opposed to preconfigured) request strings should be now working. In the mobile app Automations screen for the Web Requestor device, you should have 3 Then action options presented to you:
   -1) Web request (to select a preconfigured URL)
   -2) POST URL (to send a POST for a specified URL)
   -3) GET URL (to send a GET for a specified URL)
   There continue to be random issues on iOS regarding this showing up in Automations correctly, so please report here any problems.

[Terri_Baker (TeBa)]

I don’t know much about web requests so please forgive me if this is completely incorrect!

Could this be used in place of my current Roku smartapp to control on/off for a Roku TV?

[Alwas]

If you open Live Logging in the IDE and see Web Requests being made when you use your smartapp then most probably yes it would work for you. In one of the Roku smartapps I see “hubaction” so again, probably yes.

def hubAction = new physicalgraph.device.HubAction(“”“GET /query/apps HTTP/1.1\r\nHOST: $host\r\n\r\n”“”,

[TAustin]

@iridris Following up on your request for HTTPS support. I’ve confirmed that I can indeed support HTTPS from an Edge driver, so I will add the code and would like you to test it, if you would. And if you don’t mind, can you download the CLI to a machine (Windows/Linux/Mac) so that we can have visibility to the logging? It’s quite easy to do; I can give you a step-by-step if that would help.

What I will do is push this test version out through a separate testing channel so you can get it loaded to your hub separately from the official ‘published’ version.

DM me and let me know when you could be ready.

Thanks.

[TAustin]

All - I’ve just realized I neglected to state that there is a 3-second timeout currently configured for these web requests. This means that the POST or GET is sent, and if an acknowledgement is not sent back from the receiver within 3 seconds, you will see a ‘timeout’ error on the device details screen. You’ll always get this timeout if you try sending to an IP address where no one is listening.

If anyone thinks this timeout duration could be problematic for their particular situation, let me know. A possible enhancement to the driver could be to make the timeout duration configurable through Settings.

[Tim99 (Tim)]

Hi @TAustin could you please share a code snippet of how ou are doing a HTTPS call please.
Many hanks TIM

[TAustin]

local https = require "ssl.https"
local responsechunks = {}

local body, code, headers, status = https.request{
      method = req_method,
      url = req_url,
      protocol = "any",
      options =  {"all"},
      verify = "none",
      sink = ltn12.sink.table(responsechunks)
     }
local response = table.concat(responsechunks)

Still testing though…

Credit to @gol

[Tim99 (Tim)]

Ta La! …

[TAustin]

An update to the driver has been pushed out to the shared channel. This update adds https support. Big thanks to @iridris for testing!

A couple other notes:

• A ‘refresh’ capability was added, so the device should be recognizable by webcore now
• I have a forwarding bridge program available for anyone who needs to issue web requests outside of their local LAN. This bridge may have some interesting uses including accessing the SmartThings RESTful API from within an Edge driver! I’m currently expanding its capabilities to include giving devices that can emit http commands access to an Edge driver (currently not possible due to port assignment within the hub). I’ll probably start a new topic on this when it is more complete.

[Alwas]

Web Requestor is working great, and it’s solved many issues. I was meaning to ask, can you put your PayPal more prominently so we can show our appreciation for your work? I can’t find it, maybe put it in the first post please.

[Jason_Meudt]

This bridge may have some interesting uses including accessing the SmartThings RESTful API from within an Edge driver! I’m currently expanding its capabilities to include giving devices that can emit http commands access to an Edge driver (currently not possible due to port assignment within the hub).

As I read this, an Edge driver for the Fully Kiosk Browser jumps to mind… With it’s own REST and Javascript API functions, once the port assignment issue is ‘fixed’ a driver should not be that far away?

bloodtick/SmartThings/blob/master/devicetypes/bloodtick/fully-kiosk-browser-device.src/fully-kiosk-browser-device.groovy

[RELEASE] Fully Kiosk Browser Device and Piston

[TAustin]

Not familiar with Fully Kiosk, but yes, conceivably anything you can connect to on your local LAN will be open to possibilities. And if the port binding issue doesn’t get fixed any time soon, a forwarding bridge is always an option.

[thorsten1]

This Driver is fantastic, thank you very much for your great work.

Is it possible to link a Virtual Device to the result of a GET-Request? With this it would be possible to monitor the state of LAN-Devices, for example: send me a notification if the NAS doesn’t answer a Request on Port 443?

[TAustin]

I’ve been looking at doing that. I can certainly create a text string attribute where I can place whatever was returned from the GET request. But what you can then do with that data is somewhat limited (outside of webCoRE) With the Rules engine, you can do an equality test of the full string in an ‘IF’ construct. Unfortunately there are no sub-string compares available, all you can do is compare against the entire string.

Some things return json, some things return XML, and other things who knows. So I’m not sure I can, or should, come up with some standard format to capture the data in the attribute. It would just be the raw string.

Given what you are looking for, I’d probably fold the http request status itself into that same status field, so you could check for timeouts, failures, refused connections, etc.

[TAustin]

Well I’ve done some experimenting and have gotten some very interesting results!

I now have a test version of Web Requestor that displays the return data. I have observed so far the following:

• In general, displayed text strings are limited to two lines. I think the attribute itself does still contain the entire string, but the mobile app limits what it will display of it

• If the data is XML, SmartThings seems to try and interpret it; the raw XML is not shown, but rather the values that it tries to parse out of it. This tells me that SmartThings may in fact be using XML behind the scenes to encapsulate displayed attribute data. The question remaining is, do they actually change what’s in the attribute field (from raw XML to parsed output), or is it just the way it gets displayed. This will determine how/if it can be for testing in a Rule.
  

  

• If the data is json, it it displaying the raw json as-is. For example, using my bridge program, I sent a SmartThings RESTful API call to get the state value of another device, and the first couple lines of the returned json was displayed - enough to see the state value and part of the timestamp.
  

  

• Now here’s one I really wasn’t expecting: using my bridge program, I did a GET request to www.google.com, and the mobile app actually tries to display the web page!! So html appears to be recognized, and is handled as a special case. And in fact, this is the only case where the output field was expanded beyond two lines. You can even interact with it bit: if you tap on one of the links, it opens google in a browser window.
  

  

I’m sure someone in-the-know from the SmartThings mobile app team could probably shed some light on the above behavior but I thought it was interesting. I’ll be asking to see if there is anything I can do in the capability presentation to alter the way this data is displayed.

I have also changed the way http errors are displayed: I’m now displaying them in the new response field, with a ‘**’ prefix so that Rules would be able to test for things like ‘**timeout’, ‘**refused’, ‘**failed’, etc. This would address your desire to be able to test if there is no answer at a particular port. I haven’t actually tested any IF-THAN Rules yet…

So I’ll be interested in anyone’s comments on the above. I’ll hold off pushing out this update until I get some feedback and see what more I can find out about the various display scenarios.

[orangebucket (Graham)]

The plugins that display device data output HTML so it seems to me you are seeing the usual consequences of outputting raw XML or HTML in a webview.

[gol]

@TAustin
Interesting discovery! To be able to HTML-render the presentations will open for a much nicer UX. Hope ST team will confirm that this is an official supported feature and give us a list with supported HTML tags.

Just an example:

[Tim99 (Tim)]

@TAustin This is brilliant work!
Samsung have a major opportunity here to boost the UX of their platform with this one simple feature.
What Capability do you add to get the Text Field?

[TAustin]

It’s really nothing special at all. The magic happens when you load the attribute with html code…

It’s a custom capability with just a simple text string:

"attributes": {
        "response": {
            "schema": {
                "type": "object",
                "properties": {
                    "value": {
                        "type": "string"
                    }
                },
                "additionalProperties": false,
                "required": [
                    "value"
                ]
            },
            "enumCommands": []
        }
    }

The presentation json is straightforward too:

"detailView": [
        {
            "label": "HTTP Request Response",
            "displayType": "state",
            "state": {
                "label": "{{response.value}}"
            }
        }
    ]

The API doc says this about using label for state display type:

label
string (formattedLabel)
This displays a string. This can be a formatted string with variables

It didn’t elaborate on what they meant by ‘formatted string’ ; who knew it could be html or xml!

[Tim99 (Tim)]

Thanks @TAustin
These text fields could be used to give visual feedback on the status of our smart homes.
E.g All lights are off, Security system is armed, Garage door closed.
These would be welcome on my Smartthings app Favourites page.