Struggling with getting devices live

[jomag (Jonatan)]

Hi,
I’m writing a schema integration (just a sandbox test for now). I did follow the WebRTC tutorial and got it working (except that on second try, the invitation did not work for whatever reason).

Then I’ve written my own implementation from scratch to make sure I understand it all.

I create the schema app and a device profile. I’ve created an invitation and the authentication required to accept it. I’ve implemented all webhooks and don’t get any error responses (“interaction results”) anymore.

On my phone, I’ve accepted the invitation and I can see the device in the grid. But it’s disabled, and if I click it I get the error:

“Can’t connect to device. Check device and try again.”

I do get a “command request” webhook when I click it:

{"externalDeviceId":"1000","deviceCookie":{},"commands":[{"component":"main","capability":"st.imageCapture","command":"take","arguments":[]}]}

I’ve made sure to setup the device profile exactly like in WebRTC example, and the states I report are also exactly the same.

Any idea how to continue debugging this? I can’t find any logs or anything else to help me at this point.

[jomag (Jonatan)]

I’ll be glad to post whatever code or request/response captures that could be useful in debugging this. I would also prefer to create a much simpler device profile and device than the WebRTC example, but since that’s the only example I’ve seen working, I did an exact copy of that.

All I want at this point is some basic but statble “hello world” implementation.

[Andrew_Bartlett]

Tagging @nayelyz

[nayelyz (SmartThings Developer Support)]

Hi, @jomag

I haven’t gone through the sample completely, but when you cannot open the device’s detail view, you need to verify:

1. The device appears in the API (devices endpoint)
2. All the properties are set correctly (the main to verify are capabilities, presentationId, profileId)

@jomag

I create the schema app and a device profile.

I think for testing purposes, you can create the schema connector in the Developer Workspace and then “deploy it to test”. This should make it available in your catalog (Add device - Partner devices - Recommendations - My testing devices).
These instructions can help you go through the registration process: Schema Devices and Custom Capabilities

Remember you need to enable the developer mode in your ST app.

Thanks for the tag, @Andrew_Bartlett

[jomag (Jonatan)]

Thanks for the reply, I’ve been away from this for some time, but looking into it now again.

I started by verifying the devices endpoint as you asked me to. The test device I’ve created do appear, and it has capabilities that seems right. It has no presentationId field though! And the profile id is nested like this:

            "profile": {
                "id": "f6103380-bbe3-4ca3-a649-f1ce8c2502ca"
            },

So I need to set the presentationId. For schema apps, as I understand it, the devices are not “registered”, but are returned by the discovery handler. The response does not support any presentationId though (Interaction Types Reference | Developer Documentation | SmartThings). I’m assuming it’s my device profile that I need to configure the presentation ID in then?

[nayelyz (SmartThings Developer Support)]

Hi, @jomag
The presentation isn’t sent in a schema interaction. It must be part of the “metadata” property in the device profile. For example:

{
    "id": "58463b6b-5c8b-40ab-bfc0-...",
    "name": "testVisibleCondition",
    "metadata": {
        "vid": "bed794f9-1b32-3866-bd2b-...",
        "mnmn": "SmartThingsCommunity"
    },
    "migrationStatus": "NOT_MIGRATED",
    "status": "DEVELOPMENT",
    "preferences": [],
    "components": [
        {
            "label": "main",
            "id": "main",
            "capabilities": [
                {
                    "id": "audioMute",
                    "version": 1,
                    "ephemeral": false
                },
                {
                    "id": "smokeDetector",
                    "version": 1,
                    "ephemeral": false
                }
            ],
            "categories": [
                {
                    "name": "Switch",
                    "categoryType": "manufacturer"
                }
            ]
        }
    ],
    "owner": {
        "ownerType": "USER",
        "ownerId": "..."
    }
}

The “vid” is the presentationId, they are synonyms, you’ll see them around.
So, once you create the device presentation, add that value to this section.

[jomag (Jonatan)]

Thank you for the help! You were completely right, I needed to set a proper vid before it could work. I’m implementing a mocked doorbell device now with video, motion detection, image capture and button press. I changed the vid in my device profile to be SmartThings-smartthings-SkyBell_HD_Doorbell, and suddenly I no longer get the “can’t connect to device” error anymore. Instead I see the motion status on the dashboard, and when I open the device I see both motion status and button press status, and both seem to work when I trigger those events in my mock.

I’ve also implemented image capture, and I can see that it works all the way up to where Smart Things successfully fetch the captured image from my webserver. But the image does not appear in the app. A few times the “tap to capture” label in the screenshot below was replaced with what seems to be the URL of the captured image.

And for the video stream, I just get the hostname of the stun service:

I’m kinda stuck on how to progress from here. My guess is that I need to create a custom device presentation instead of relying the predefined presentation for a similar product. But I find the documentation lacking here. And the WebRTC tutorial seems to skip this step.

So here are a whole bunch of question that I would love some answers to:

• Is the content of the screenshot above what’s expected when using vid SmartThings-smartthings-SkyBell_HD_Doorbell?
• What exactly is the difference between a device config and a device presentation?
• What API should I use to create the config/presentation? All the presentation API endpoints are marked as beta. I’ve also noted that there is a deviceConfig subtree in device profile. Would that create the config automatically? I tested, but I got the old “can’t connect to device” problem again, so at least it’s not automatically assigned to the device profile.
• I’ve created a device config using /v1/presentation/deviceconfig. I made sure to write down the ID in the response, and then I successfully assigned it to my device profile. But there seems to be no way to get the list of created configs? So if I lose that ID, there’s no way to get it back, and there will be unused device configs in the ystem.
• What would a basic device config look like, that allows showing captured images? What about streaming video?
• Where can I learn more about how to setup device configs/presentations?

Very grateful for some response on these questions as I’m quite blocked and down to randomly testing stuff!

[nayelyz (SmartThings Developer Support)]

@jomag

I changed the vid in my device profile to be SmartThings-smartthings-SkyBell_HD_Doorbell, and suddenly I no longer get the “can’t connect to device” error anymore. Instead I see the motion status on the dashboard, and when I open the device I see both motion status and button press status, and both seem to work when I trigger those events in my mock.

Hi, @jomag
Let me check some details with the engineering team about the sample you’re basing on and the change made to the presentation.

For now, I’ll answer the questions I can:

@jomag

SmartThings-smartthings-SkyBell_HD_Doorbell

If we query its configuration, it seems so, but I don’t know if this loads a custom UI for the official integration. I’m guessing you got that value from another device installed, right?

@jomag

What exactly is the difference between a device config and a device presentation?

The device config is basically a file where we define the changes for the device’s UI (the changes we can make are limited, for example, we can edit the range of the numeric capabilities, options available for the “mode” capabilities) and list which capabilities we want to appear in the UI.
It is not effective until we use the file to create a device presentation since it is the result of applying the device config’s changes to each capability’s presentation.
For example, your device profile contains the capabilities you can use in the back end, but if one is omitted in the device config to create a presentation, the front end (ST app) won’t display it.

@jomag

What API should I use to create the config/presentation?

//command to use a device config file to create a device presentation
smartthings presentation:device-config:create -i deviceConfig_file.json

//command to generate a device config file when we have a presentationID/vid and mnmn
smartthings presentation:device-config presentationId mnmn -j -o=deviceConfig_file.json

//command to generate the default device config file so you can use it to create the default device presentation.
smartthings presentation:device-config:generate profileId -o=deviceConfig_file.json

The default device presentation means that the file will include all the capabilities in the profile and the configuration defined on their presentation without modifications.

About the subtree deviceConfig, it is used to define the deviceConfig in the same device profile, but I haven’t tested it completely. It’s best to create the presentation independently and assign it to the profile.

@jomag

So if I lose that ID, there’s no way to get it back, and there will be unused device configs in the ystem.

If you use the same device config file to create a device presentation again, you’ll get the same vid/presentationId as before.
Device presentations are global, so any one can use them and only new combinations of configurations will generate a new presentationID.

@jomag

What would a basic device config look like, that allows showing captured images? What about streaming video?

This is what I’m checking with the engineering team, once I have more info, I’ll let you know.

@jomag

Where can I learn more about how to setup device configs/presentations?

In this doc, there are samples of how your device config should look like and how it’s presented in the app:

Device Configurations | Developer Documentation | SmartThings

The API reference describes other properties that can be used on each view - API | Developer Documentation | SmartThings

For your case, do you want to create the device presentation yourself to see if it displays something different?

[jomag (Jonatan)]

Thanks for the reply!

Regarding the ID SmartThings-smartthings-SkyBell_HD_Doorbell, I got it from here: https://vid-selector.glitch.me/.

I’ve now got a Google Nest Cam, just to verify streaming functionality, and I can see that they display streaming video etc in the Smart Things app, but it looks like nothing that can be accomplished with the documented device-config/device-presentation features. You also say that you “don’t know if they loads a custom UI for the official integration”: do I then understand it correct that you can create an interface with device-config, but there’s an alternative, possibly only available to partners?

Thanks for clarifying the relation between device configs, presentations and presentation ID’s.

[nayelyz (SmartThings Developer Support)]

@jomag

I can see that they display streaming video etc in the Smart Things app, but it looks like nothing that can be accomplished with the documented device-config/device-presentation features.

I’m checking this with the engineering team. What I meant before is I don’t know if there’s something extra they use in the background since I haven’t had much contact with this kind of integration for a while.
I’ll keep you posted.

[nayelyz (SmartThings Developer Support)]

Hi, @jomag

Sorry for the delay, we finally got a reply from the engineering team.
So, instead of using a presentation ID like “SmartThings-smartthings-SkyBell_HD_Doorbell”. You can create your own device configuration, but you’ll need to add the property “dpInfo” with the following values:

"dpInfo": [
        {
            "os": "ios",
            "dpUri": "storyboard://HMVSController/HMVSViewController",
            "arguments": []
        },
        {
            "os": "android",
            "dpUri": "plugin://com.samsung.android.plugin.camera",
            "arguments": []
        }
    ]

Please let me know if you have any questions.