The state of ZigBee with attribute reporting

ZigBee attribute reporting is a great feature. It basically allows for “push” messages from devices when their state has changed. This eliminates the need for polling, reduces traffic on the network, and keeps SmartThings instantly in sync.

I’ve been hitting various roadblocks in this endeavor and I thought I’d document them in this thread. Feel free to add your own observations.

First up are the SmartThings devices. In nearly all the device type templates I’ve looked at, there is a bug in the report configuration. For example here’s a snip of the Cree device type:

def configure() {

log.debug "Configuring Reporting and Bindings."
    def configCmds = [	

    //Switch Reporting
    "zcl global send-me-a-report 6 0 0x10 0 3600 {01}", "delay 500",
    "send 0x${device.deviceNetworkId} ${endpointId} 1", "delay 1000",
    
    //Level Control Reporting
    "zcl global send-me-a-report 8 0 0x20 5 3600 {0010}", "delay 200",
    "send 0x${device.deviceNetworkId} ${endpointId} 1", "delay 1500",
    
    "zdo bind 0x${device.deviceNetworkId} ${endpointId} 1 6 {${device.zigbeeId}} {}", "delay 1000",
	"zdo bind 0x${device.deviceNetworkId} ${endpointId} 1 8 {${device.zigbeeId}} {}", "delay 500",
]
return configCmds + refresh() // send refresh cmds as part of config

On the “send” command, the source and destination endpoints are backward. The on/off reporting should not have any payload. Also the level reporting has too many digits. The max interval is way too long. It should be like so:

//Switch Reporting
"zcl global send-me-a-report 6 0 0x10 0 120 {}", "delay 500",
"send 0x${device.deviceNetworkId} 1 ${endpointId}", "delay 1000",

//Level Control Reporting
"zcl global send-me-a-report 8 0 0x20 5 120 {10}", "delay 200",
"send 0x${device.deviceNetworkId} 1 ${endpointId}", "delay 1500",

On the Cree bulbs this is particularly a problem because it uses endpoint 10, and not 1 like a lot of devices. Not a big deal in this case, because the code can be easily fixed.

So once that’s solved, the bulb now reports periodically (in the case of the code above, every 120 seconds). However, the bulb should also be reporting on change; in the case above any change of on/off or a level change > 10 should cause a attribute report to be sent to the coordinator. This doesn’t work. I’ve logged the packets with the Ubiqua analyzer and they all check out okay, so I can assume this is a bug in the Cree firmware.

The second problem with the Cree bulb is that these report configurations are volatile; if the light is powered off it loses it’s memory. This should not be the case. And don’t even get me started about the Cree bulb using the device ID of 0x0100 (“On/Off Light”) instead of 0x0101 (“Dimmable Light”). Ugh.

Next up is my Kwikset 914 deadbolt. I’ve been working on a device type for this and it too has problems with attribute reporting. In this case, the attribute report does not reflect the true state of the lock. What happens is as follows: If I refresh the device, the attribute report is correct. If I turn the lock by hand, I do not get a change report. The next attribute report (on max interval) reports the old state! No wonder I didn’t get a change report. If I then refresh, now I get the correct state, and subsequent attribute reports also report the correct state. It’s as if only a refresh (read attribute) will update the internal variable containing the state.

I think attribute reporting is an important feature. For example it will update the status in SmartThings when a smart bulb is power cycled. If SmartThings doesn’t see a report in the allotted time it can assume the bulb has dropped off the network (manually turned off). The state of devices with manual input (door locks, etc) can stay in sync with the SmartThings state. The alternative to attribute reporting is polling, which creates a lot more network traffic and also creates a delay in the status updates; not to mention that polling devices seems broken in SmartThings at this time.

2 Likes

A lot of our recent changes are regarding the configuration updates to the zigbee DTH. We updated the first batch of DTH 2 weeks back to reflect this.

For zigbee devices, we are documenting these changes and will have the new changes in our docs soon.

For now, you can use the following method in the configure() to set up the DTH:

zigbee.onOffConfig() 

That will setup the switch to auto report every 10 min (heartbeat) even if there is no change. If you are using any other custom cluster, there is support for

zigbee.configSetup(cluster, attributeId, dataType, minReportTime, maxReportTime, reportableChange)
/**
	 * @param cluster - The cluster id of the requested report.
	 * @param attributeId - The attribute id for requested report.
	 * @param dataType - The two byte ZigBee type value for the requested report.
	 * @param minReportTime - Minimum number of seconds between reports.
	 * @param maxReportTime - Maximum number of seconds between reports.
	 * @param reportableChange - OCTET_STRING - Amount of change to trigger a report in curly braces. Empty curly braces if no change needs to be configured.
	 */

For example the onOffConfig() internally does this:

def onOffConfig() {
   configSetup("6", "0", "0x10", "0", "600", "{01}")
}

As I said, we are working hard toward multiple of these changes to make it easier for our developer to integrate the huge list of the devices that will come up in the near future.

3 Likes

Great, thanks for your input! It’s good to see more official support for attribute reporting. BTW I think the 0x10 data type (boolean “discrete”) doesn’t use the payload; see 2.4.7.1.7 and 2.5.2 of the ZCL spec. I realized this when I saw it in the Ubiqua sniffer.

If I could recommend one more API feature, it would be a method that would be called when the device receives an “End Device Announce”. This would allow the device to re-run the configure method (if needed, like it seems to be for Cree), as well as poll for the status (read attribute) to update the state if for example the device was just turned on manually.

Is there something in the works for a polling fix too? It would be great for brain-dead devices like my Kwikset lock (to refresh the lock state). Also, for devices that periodically report (maxReportTime) it would allow a third state like ‘on’, ‘off’, and ‘offline’ when the device doesn’t send out a report when it should have.

Thanks again!

2 Likes

Great stuff @rpress. I would also like to call to your attention and others who may be reading this tread that SmartThings intercepts some ZigBee attribute reports before they get handed to your parse method. For example I have a device that has two thermistors and reports temperature for each probe on two unique endpoints to the SmartThings hub. Well when I add the temperature capability to my device type the reports show up in my parse method with the source end points stripped out. I have no way of knowing what probe the report is for. Here is a thread where I discuss this ZigBee Temperature reporting question for Cluster 0x402. I never got an answer to my question as the SmartThings engineer who was trying to help me got hung up on the bind order of my endpoints. I find it very interesting that you have found several code samples with the binding orders reversed. There have been others who have seen this same thing with other capabilities.

Here is another case of this “source endpoint masking” for the Switch capability: Zigbee Receive Report Attribute from multi-endpoint

I don’t mean to hijack your thread I just want someone to address this issue and I’m just calling it out here as it is related to attribute reporting.

3 Likes

@JohnR thanks for bringing this up again. This is definitely an issue at this point for some devices, and will only get worse with more devices in the future.

Part of the reasoning is the simplification of the messages done early on for v1 hub with the smart shield and other devices. We are updating these code snippets so that we can be ready for the newer devices, like yours, with multi-endpoint-similar-capability devices.

The current plan is to have these updates along with the document update for ZigBee DTH ready for the community in about 2-3 weeks. The process is ongoing and you can see the changes already being done to the ZigBee devices on our github code repo.

2 Likes

I have tested the GE Link bulb, and they correctly report on change and also remember the settings. It’s unfortunate for me because I bought a bunch of Cree bulbs (I really like the quality of the light) but they are certainly inferior in their ZigBee stack. Once the GE bulbs get the firmware update (dropping off the network) I think they will be the best option.

This is exactly what I wanted for this thread, a place where we could discuss attribute reporting issues. And not just with SmartThings but with ZigBee devices in general. Thanks for bringing that up!

1 Like

I’ve tested the Sylvania IQ BR30 bulb and it works great. The attribute reporting works fine and it has great light output as well. I should note these are a bit taller than a regular BR30, so I needed to adjust the depth of the can.

I’ve ordered a Wemo Smart LED bulb (which looks like it’s made by Sylvania) to test out next.

1 Like

wow a $19 LED dimmable ZigBee bulb that supports attribute reporting (wish Hue did and it cost $59). So here is something I have thought about doing in the past. Since these bulbs support attribute reporting you could use them as a presence sensor. So walk into a room switch on the bulb with a standard wall mounted light switch and bang you know someone is in the room. Just one small problem you wont know when someone leaves the room since turning off the light with the power switch wont allow it to send a report. How cool would it be if they added a small cap to hold enough juice to send just one report packet (have the bulb report it was powered off). Anyway $19 is exciting!! Good work @rpress

1 Like

Thanks @JohnR! One way I was thinking of detecting “bulb powered off” is to detect a missing periodic update from the bulb. The bulb will report at “max interval”, and if SmartThings doesn’t see a report in the alloted time frame it will assume the bulb has gone offline. I believe this is how ZigBee Alliance intends that this is used. For this to work we need the polling feature fixed, though…

As an aside I’d bet the bulbs do have enough capacitance to send a dying breath but they just don’t do it now. They need some kind of persistence to detect the on/off cycles used for resetting the bulb, and I doubt they’re using the EEPROM/flash for this.

I like using the max interval, good idea! I have another project where I want to make sure one of my devices reports a reading from its sensor according to the max interval. That would have to be in a SmartApp I just haven’t spent the time to make it so. I have concerns that the report will be hidden from the SmartApp. From what I understand if SmartThings receives a report and the value is the same as the previous value a SmartApp has no way of knowing it received the report. So to work around this I was going to make a counter that my device type would increment whenever a report was received regardless of the value. Now I have two reason to do this!!

I’m liking these bulbs more and more.

This Wemo bulb is a bust. The attribute reports don’t work right, and also it has this weird feature where it turns on instantly but turns off by dimming over 4 seconds. I couldn’t change this “feature” with the “OnOffTransitionTime” attribute either.

I have also tested the OSRAM Gardenspot. The attribute reports seem to work fine. I’ll need to change the DTH to get it updating correctly. Oddly enough this one doesn’t support group multicast (ZigBee Pro) but only group broadcast.

I hacked together some code that does exactly this for the native Hue bulb and the OSRAM bulbs. It completely sucks compared to attribute reporting though

Zigbe mesh doesn’t have enforced sequencing. You have no way of knowing the exact order in which messages will arrive. This tends to make it inappropriate for any kind of microlocation. You might get the message from the far room before you get the message from the near room. There’s just no way of knowing which repeaters will be available and when.

Microlocation is better done with a star or tree topology. You can force zigbee into this, but Not with the smartthings hub.

Plus with bulb-based presence there’s the question of whether you only want location information at night. :wink:

Zuli is doing really interesting work with microlocation. The problem is the person has to carry the identifier device, and not everyone carries their phones everywhere in the house. Zuli’s waiting on one of the smartwatches or fitness bands to come up with an identifier they can use.

https://zuli.io/lifewithzuli/lifestyle/

1 Like

I’ve received the OSRAM LIGHTIFY Tunable White 60 bulb. It works really well, the only slightly odd thing is that it allows you to dim it to zero (like the GE Link) so you can’t tell if it’s off, or just dimmed to zero. Otherwise it reports attributes fine and otherwise seems great. The tunable color temp is nice.

It’s pretty pricey at $30, otherwise I’d say it would be great to replace all my Cree’s with this one.

Does anyone have any ideas for an A19 bulb that is cheaper than $30 but might support attribute reports? I read the Philips bulbs didn’t report attributes in the past, maybe the new ones are better?

1 Like

I bet your quoting one of my earlier post about this. What I was trying to say is the Hue Bulb does not support attribute reporting to a bound address. The bulb can not be bound to a ZigBee address. You can read the attributes but if you configure it to report them since it cant be bound to a destination it won’t automatically report them. You have to poll them. I came across all this about two years ago in a Philips forum. This morning I just went back and verified this. None of my hue bulbs can be bound to an address (all my bulbs are the Gen 1 bulbs).

I’m going to hunt back through my post and update my other post on this. If you get someone to test this please include me in the results I want to keep up to speed on the new Hue bulbs.

@rpress if you have time take a look at the SmartThings custom device type for the ZigBee Hue bulb. It has the Configuration capability but no configure method that would bind the bulbs to the hub. If the new bulbs support binding we need to rewrite this custom device type.

1 Like

I have one of the new A19 color bulbs and was lamenting that the configuration section was absent. I didn’t realize the gen1 bulbs didn’t support attribute reporting. Hopefully the new bulbs do support this. I’ve tried adding the configure() to my device type with reporting and binding, but it looks like Live Logging isn’t working for me right now to tell if it worked.

Attribute reporting is key for me since I want to use this bulb in conjunction with the Hue dimmer remote, so I need the bulb to report back that its state changed to prevent sync issues.

Here’s what I added to the configure section if you can confirm it looks ok. I went with 60 seconds for the on/off to test whether it would report. For the binds, I always get confused which endpoint is which since mostly they’ve been 1 for both devices. The Hue uses endpoint 0B.

def configure(){
	log.debug "Initiating configuration reporting and binding"
    
    [
	zigbee.configSetup("6", "0", "0x10", "0", "60", "{}"), "delay 1000", 
    
    zigbee.configSetup("8", "0", "0x20", "5", "600", "{0010}"), "delay 1500", 
    
    "zdo bind 0x${device.deviceNetworkId} 1 ${endpointId} 6 {${device.zigbeeId}} {}", "delay 1000",
	"zdo bind 0x${device.deviceNetworkId} 1 ${endpointId} 8 {${device.zigbeeId}} {}"
	]
    
}

Update: Got the live log working, but unfortunately I got the following back. Looks like an Unsupported response.

catchall: 0104 0006 0B 01 0100 00 D5B3 00 00 0000 0B 01 0682
catchall: 0104 0008 0B 01 0100 00 D5B3 00 00 0000 0B 01 0682

It looks like your bind endpoint order is wrong. It should be src dest, where src is the bulb and dest is the hub. There should only be two characters (one hex byte) in the payload for level report config.

2 Likes

Thanks for the help! Still getting the same Unsupported response to the report request. That is just silly for such expensive bulbs not to support attribute reporting…

2 Likes