Event POST Webhooks for devices

Maybe it’s just me, but after a few weeks reading the developer docs, i’m still not particularly clearer on how SmartThings is put together. I’ve successfully authenticated to the REST API and can read locations, rooms, devices etc, and also send commands to a device as required.

What i need to do is receive an incoming POST payload to a self-hosted URL endpoint from hub-connected devices when they change status. For example, a door sensor sending a JSON structure with the condition of “open” or closed".

Do i really need to create an “app” to do this? It seems a little over-the-top. I’ve created a Webhook automation within the developer workspace and can receive “ping” lifecycle requests, but whenever i try to save it, i receive an unspecified error about the “upstream target” timing out (when i know it is accessible).

1 Like

Hello @Alex_Cstr8,

When you register and save a SmartApp Automation (Webhook), SmartThings sends a PING request to your webhook Url.

Example request body:

{
  "lifecycle": "PING",
  "executionId": "xxxxxxx-xxxx-xxxx-xxxxxx",
  "locale": "en",
  "version": "1.0.0",
  "pingData": {
    "challenge": "xxxxxxx-xxxx-xxxx-xxxxxx"
  }
}

Which must be responded as:

{
  "pingData": {
    "challenge": "xxxxxxx-xxxx-xxxx-xxxxxx"
  }
}

In order to enable real-time events over your devices, you must create a Subscription of your device through the Subscriptions API and when a subscription’s condition gets satisfied, it will emit an EVENT lifecycle request to your webhook Url.

The following JSON example will emit an event whenever the device shows an “open” status:

{
  "sourceType": "DEVICE",
  "device": {
    "deviceId": "device_id",
    "componentId": "main",
    "capability": "contactSensor",
    "attribute": "contact",
    "stateChangeOnly": true,
    "value": "open"
  }
}

For more reference, please read the SmartApp lifecycle and Subscriptions documentations.

Best Regards,
Erick.

1 Like

Thanks @erickv! So would it be accurate to conceptualise a “SmartApp” as a kind of “bridge” between SmartThings and your own web app?

I have the lifecycle set up and have got the various types of calls in place (ping, install, event etc), on top of understanding what you’ve put above.

2 questions:

  1. Is the “install” part of the flow absolutely required? It’s not amazingly clear from the docs what this is useful for exactly. I have no install parameters to add really, other than environmental variables to identify the incoming call.

  2. What is the best approach to dealing with multiple devices? Let’s say i have 10 different SmartThings pieces connected to a hub (e.g. lock, speaker, sensor etc). What if they all require different setups and have differing events? Is it best to have one app per device, or to write in logic which switches/filters based on the device type?

Thanks in advance!

Hi @Alex_Cstr8,

You can think of a SmartApp as an interface where you can monitor and customize the behavior of your devices.

  1. About your first question:

During the INSTALL lifecycle you’ll get your devices’ information needed to create the logic of yourSmartApp’s service.

The most important elements received in an INSTALL lifecycle are:

  • An Authorization Token to interact with the SmartThings API.
  • A Refresh Token, useful to request a new Authorization Token before the previous one expires.
  • The Installed App ID which will relate your SmartApp to subscriptions/schedules created from it.

Note:Check the Weather Color Bulb SmartApp sample which creates schedules to update a device’s state. It also provides an example on how to interact with other web-services to generate the new status of the device.

  1. About your second question:

You can do it by setting up different selectors classified by capability. See the JSON below that corresponds to the CONFIGURATION / PAGE lifecycle:

{
    "configurationData": {
        "page": {
        "pageId": "1",
        "name": "Setup webhook for devices",
        "nextPageId": null,
        "previousPageId": null,
        "complete": true,
        "sections": [
            {
                "name": "Selector for Contact Sensors and Doors",
                "settings": [
                    {
                    "id": "contactDevices",
                    "name": "List of all contacts",
                    "description": "Tap to set",
                    "type": "DEVICE",
                    "required": false,
                    "multiple": true,
                    "capabilities": [ "contactSensor", "doorControl" ],
                    "permissions": [ "r" ]
                    }
                ]
            },
            {
                "name": "Selector for Switches/Lights",
                "settings": [
                    {
                    "id": "switches",
                    "name": "List of all switches / lights",
                    "description": "Tap to set",
                    "type": "DEVICE",
                    "required": false,
                    "multiple": true,
                    "capabilities": [ "switch" ],
                    "permissions": [ "r" ]
                    }
                ]
            }]
        }
    }
}

I hope this information results useful to you,
Erick.

1 Like

Couple of notes for other developers working with this:

  1. The example JSON in the docs is invalid (trailing commas). Make sure to test the examples in a JSON lint validator.

  2. The permissions stated for deviceprofiles are incorrect. It requires a trailing .* the same as the others. If you leave it out, you’ll get a network error screen after agreeing to the permissions on install.

  3. To find the capabilities of each device to get them to show up in the list - as listed in the API - you need to do a GET request to /devices/show/{device_id}.

1 Like

“SmartApp” is a term inherited from the previous version of the platform when all the customer-contributed code was groovy code that ran in the smartthings cloud.

For an explanation of the original term, see the FAQ (The topic title is a clickable link)

I think it’s helpful to understand the original term, because so far 99% of the threads in this forum will be using that meaning.

The new platform operates differently, because now you run your own code elsewhere and then connect via the API.

It appears that when it came to naming things, they didn’t make a terminology distinction between

A) code which is used to format command requests to the controller for a cloud-connected device (the equivalent of a DTH on the original platform) and

B) code which is used to process control data (filters, event subscriptions, user-entered values, time of day, etc) and then request that commands be sent to SmartThings-controlled devices based on that data. (The equivalent of a smartapp on the old platform, except that on the old platform this code ran in the smartthings cloud and now most of it will run on your own server.)

So on the old platform, code that would format the “turn on“ message for a specific model light switch would be in a DTH. Code that would subscribe to the event of “John arrives home” and then request that a selected switch should be turned on when he arrived would be in a smartapp.

Now they appear to have wrapped all of it under the term “smartapp” but with some additional stuff like the “cloud connector“ thrown in and some inconsistencies throughout the documentation in how the various terms are used.

But I did just want to make sure that you understood the original use of the term “smartapp“ because otherwise the rest of this forum is going to be very confusing to you. :sunglasses:

Thanks @JDRoberts! The extra detail is really appreciated. Terminology is crucial for understanding an evosystem of this size and it’s been enormously difficult to get people’s heads round any of it. I work with a team of heavily-tech-savvy devs and if they’re having trouble, i know i’m not going mad.

“How does this device have a capability of ‘configuration’ when we’re supposed to be setting it within the smart app configuration? What on earth is a device handler when we’re handling the device on the server side? Why does the device install have an auth token when we have access to it via a user auth token? Why can’t we install the Hub if it’s listed as an available device in the list? How do we know if a user revokes permissions? Why does this have to involve the phone app when it’s supposed to be an API? Where on earth do we get the capabilities from? Why is the REST API explorer called a Groovy IDE? Why are there country-specific devices which are exactly the same?”.

I tried to explain this to a partner at our firm badly: “so we’re building this app, which talks to a smartapp which you install into the SmartThings phone app. It’s kinda like a Facebook app”. That wasn’t even getting the “cloud” word salad and direct/hub connections. He looked at me totally baffled.

It’s definitely proving painful. Today the Hub had a solid blue light for hours, despite the service status saying there were no incidents. Webhook responses have no feedback to our endpoint to log, and/or contain double-serialized JSON within the “live logging” console, which also has “control events” which don’t seem to be documented anywhere.

And we really, really need a sandbox or emulator to test against. It’s unbelievably frustrating installing and uninstalling the “app” multiple times - not to mention seeing randomly “uninstall” events firing into our Slack debug channel when nobody has uninstalled anything.

We’re getting there, but it’s slow going.

Tagging @jody.albritton

I can’t answer any of the ST-specific platform questions you asked: hopefully others will be able to.

But as to this one, typically that’s because the radio frequency is different in different regions. Everything else about the device is the same if it’s battery-powered. If it’s mains-powered, the plug may also be different. :sunglasses:

Fibaro and Aeotec both make devices of this type.

The same is true of the SmartThings hub models, which are sold with different Zwave frequencies for different regions.

The documentation for the Classic environment was incomplete, and worked more like reading chapters in a book. It did sort of tell you stuff, though. The documentation for the ‘new’ environment is a bit all over the place with links flying everywhere and sending you in circles. You have to fluke finding useful stuff and if feels like those building the Developer Workspace and doing the documentation have different ideas on terminology and the architecture.

An example is the ‘SmartApp’. If you use the DevWS the flow starts with device integrations or automations. Device integrations leads you into Schema connections, direct and cloud connected connections using the API (AWS / webhooks), and hub connected devices using the Groovy IDE. Automations is all about AWS / webhooks apps. SmartApps aren’t mentioned.

Yet in the documentation you can find yourself on a SmartApp page and there they say they are AWS / webhook apps for cloud connected devices or automations. Which is just how it always was in the Classic environment. SmartApps were both automations (webCore, Smart Lighting) and connectors (Alexa, Google). In the ‘new app’ UI these got divided up into SmartApps and Connected Services (ideally SmartApps would be called Automations and the built in automation creator would be called something more distinctive).

So SmartApps are:

  • Classic environment - Automations and connectors (but not Routines).
  • New app - Automations, but not Automations in the sense of the Automation Creator, and not Connectors.
  • New documentation - Automations and Connectors, or unheard of, depending on which page you are on.
  • Developer Workspace - what’s a SmartApp?

As I have said elsewhere. It’s like the ‘new’ environment has been steered in a new direction and not everything has caught up.

2 Likes

That’s one of the reasons I find it frustrating that the primary UI to SmartThings isn’t web based. With a web based UI you are clearly working with a front end to something. Whatever that something is, you know you aren’t installing stuff in your web browser. When you use a mobile app it can create the illusion that the app itself is more than just a front end and you are creating or installing things in the app itself. You see endless comments about how such and such a device handler or app only works or doesn’t work with the classic app or new app. The apps have become conflated with the backend.

I guess these could be rhetorical questions, but I’ll take a crack:

The term ‘SmartApp’ is getting in the way a bit here. Although the term isn’t used much in the documentation or the Developer Workspace, it basically means any app you are using to interact with the SmartThings API. That app will either be acting as a connector to present your devices to SmartThings in a standard fashion, or it will be an automation that is interacting with SmartThings devices wherever they may be.

The Configure capability just advertises that your system (for want of a better word) supports a ‘configure’ command for that device. It is probably intended for sending configuration information to the device hardware, though it is a little vague. In many devices that is a one off thing that you could do at install time (and indeed, if the device has that capability defined, the ‘configure’ method of classic device handlers is called at install time). However it may make sense for users/automations to be able to call a ‘configure’ command on your device from time to time. If it doesn’t make sense then you don’t set the capability.

The classic device model is based on hub connected devices and, as you would expect, the device handler does the usual job of abstracting the device dependent stuff. So if you wanted to turn a switch on, you called the device’s ‘on’ command which resulted in the ‘on()’ method of a device handler being executed and the appropriate commands being sent to the device. The same model was (ab)used to allow bridged or cloud connected devices. So if you wanted to turn a remote switch on, you still called the ‘on’ command, and that still called the ‘on()’ method of the device handler. However that ‘on()’ method would then typically just ask a smartapp servicing the cloud connection to pass the request on. The architecture has now evolved to cut out the device handler. Ideally it should be completely transparent to the user whether your device is hub connected or cloud connected, and so users used to talking about the ‘device handler’ will use it more generically. Just to complicate things, there are ‘placeholder’ device handlers, but they are there to keep older software happy.

The device install and auth token thing is outside my experience so I’ll skip it. Personal Access Tokens are easy, but when it comes to SmartApp tokens and the OAUTH2 flow I glaze over quite quickly and hope for well written examples.

I’ll skip the hub one as I can’t see what you are getting at.

I guess if a user revokes permissions you might just have to find out the hard way by attempting to use them. That would be fairly typical wouldn’t it?

The phone app is just a client too. Regular readers will realise I just don’t get why on earth you need to use a mobile app to usefully configure a cloud based system, but you do. If you logged into a web site and clicked an ‘install this device’ or ‘install this app’ button you probably wouldn’t think twice about it.

How do you mean where do you get the Capabilities from? Do you mean how do you know what they are? They are listed in the documentation (somewhere …).

The Groovy IDE (account.smartthings.com) is so called because it is the integrated development environment for developing devices handlers and apps in the Groovy language. If you aren’t using it for that purpose it doesn’t really look or feel like an IDE. I’m not sure how it is implemented behind the scenes but I don’t see it as a REST API explorer, mainly because it is rooted in the legacy ‘classic’ environment and I’m not sure there is a REST implementation to get that info. I also think of the SmartThings API v1 when I think of the REST API, although REST is used for other things in SmartThings.

I guess if country specific devices really are absolutely identical (rather than having slightly different firmware or internal hardware), it gets called marketing. The prices sure aren’t the same …

1 Like

@JDRoberts @orangebucket legends! Thank you!

If only some of this was in the docs and guides. The “available capabilities” one is a classic i searched for over a good 45mins and found multiple users asking the same; most replied linked to legacy lists and/or sites with collections of outdated data. @erickv eventually gave me the information in a support DM.

All the documentation needs is an edit to say something like “Each manufacturer includes different capabilities within their products. To find out which capabilities your target device supports, make an authenticated GET call to to /devices/{device_id} within the SmartThings REST API. If you list too many or include capabilities the device does not have, it will not show up in your configuration request.”. Hopefully others will be able to find the info in this thread in the future: i spent far too long trying to find the appropriate capability string for a device which has a vibration sensor (quote, from Samjin Multisensor sold in Best Buy) and not finding it anywhere as a listed ST API capability.

Not to say that any of these things are insurmountable, but they make it an unnecessarily frustrating and confusing endeavour.

I find it frustrating that the primary UI to SmartThings isn’t web based.

I totally agree with this. SmartApps are predominantly talking to a webhook backend through the entire process and performing the same “advanced” OAuth flow you’d find reminiscent of most modern web apps. Designing one in a web UI would make it far simpler. It makes absolutely no sense to me to install an “app” inside an “app” - it more like a “plugin” or “external connector module” which doesn’t’ have the flexibility of one API talking to another (and a webhook is really a POST/PUT call to your API).

That app will either be acting as a connector to present your devices to SmartThings in a standard fashion, or it will be an automation that is interacting with SmartThings devices wherever they may be.

See, that terminology - “connector” makes it far simpler to conceptualise. An “automation” could easily be called a “reactor” or “responder”, and a “schedule” a “scheduled task” (a schedule is a calendar list of appointments, an action taken at a specific time is a task - in fact, “action” is even easier to think about).

I guess if a user revokes permissions you might just have to find out the hard way by attempting to use them. That would be fairly typical wouldn’t it?

Not necessarily. You’d check for a 401 response and never assume a 200, but GRANT and/or REVOKE lifecycle stages would be useful, if, say, a device is removed, faulty, or somehow should have conditional access (e.g. door lock - security breach, temporary codes etc).

How do you mean where do you get the Capabilities from? Do you mean how do you know what they are? They are listed in the documentation (somewhere …).

See above - the docs don’t have it anywhere.

it is the integrated development environment for developing devices handlers and apps in the Groovy language.

It’s a web-based console for storing functions and routines really: an IDE has a very specific meaning (think Eclipse or JetBrains stuff) and could really do with a sandbox featuring “dummy” devices to test on. Would be great to have “virtual device” profiles for testing rather than having to spend $$$ on hardware which doesn’t bloody work reliably between the new and classic systems. It would normally be a semantic (read: pedantic) distinction but it speaks to the issue of terminology again - software development is always about creating a “virtual world” or representation of things in your head before they are applied in code, which is what makes it so maddening!

I get that SmartThings is an ecosystem designed to be powered from Samsung’s phones and the Android app, but surely there have to have been conversations about making the platform truly developer-friendly?

I always Google for ‘smartthings capabilities’ which brings up the Capabilities page in the ‘Classic’ documentation, which has a redirect at the top to https://smartthings.developer.samsung.com/docs/api-ref/capabilities.html - I find that quicker than starting within the documentation itself.

There is an advantage to having played with device handlers in the ‘classic’ environment. You get to know how stuff is put together and it provides you with a base of knowledge. If I’d started with the ‘new’ documentation and the Developer Workspace I’d have just given up, same I would have done had I started, as recommended, with the ‘new’ app. I was lucky, in a way, that the new app was so demanding of resources that it took me two years before I had a device that could run it.

The Groovy IDE does have a simulator but it is rumoured to be more trouble than it is worth. I’ve never bothered to use it. There are plenty of device handlers for virtual/simulated devices available. How you are supposed to know they exist is another matter.

2 Likes

I had a look there originally but still couldn’t match up the vendor’s terminology with the SmartThings list. Here’s an example JSON object for the $10 Best Buy sensor from our setup (sensitive info redacted):

    {
  "deviceId": "[redacted]",
  "name": "Multipurpose Sensor",
  "label": "Multipurpose Sensor",
  "locationId": "[redacted]",
  "roomId": "[redacted]",
  "deviceTypeId": "[redacted]",
  "deviceTypeName": "SmartSense Multi Sensor",
  "deviceNetworkType": "ZIGBEE",
  "components": [
    {
      "id": "main",
      "label": "Multipurpose Sensor",
      "capabilities": [
        {
          "id": "temperatureMeasurement",
          "version": 1
        },
        {
          "id": "battery",
          "version": 1
        },
        {
          "id": "contactSensor",
          "version": 1
        },
        {
          "id": "configuration",
          "version": 1
        },
        {
          "id": "threeAxis",
          "version": 1
        },
        {
          "id": "accelerationSensor",
          "version": 1
        },
        {
          "id": "refresh",
          "version": 1
        },
        {
          "id": "sensor",
          "version": 1
        },
        {
          "id": "healthCheck",
          "version": 1
        }
      ]
    }
  ],
  "dth": {
    "deviceTypeId": "[redacted]",
    "deviceTypeName": "SmartSense Multi Sensor",
    "deviceNetworkType": "ZIGBEE",
    "completedSetup": true,
    "networkSecurityLevel": "UNKNOWN",
    "hubId": "[redacted]"
  },
  "type": "DTH"
},

Here’s the maufacturer page: http://www.samjin.com/en/product/iot_view04.php , the ST page: https://www.smartthings.com/products/smartthings-multipurpose-sensor , and the support/tech page: https://support.smartthings.com/hc/en-us/articles/213496343. One of the features is vibration sensor. The product doesn’t list acceleration, configuration, healthcheck, or threeAxis (although it should be implied).

What we need for the config is:

               'battery',
              'configuration',
              'refresh',
              'healthCheck',
              'sensor',
              'contactSensor',
              'temperatureMeasurement',
              'accelerationSensor',
              'threeAxis',

Example problems:

a) The manufacturer isn’t known (Samjin) until you link it to the device.
b) Where does that list of capabilities above come from, if not from the API call?