[Tutorial] How to understand Zigbee to develop a device handler

Heya All!

As I’ve been going through the journey of developing Zigbee device handlers (DTHs), I thought it was a good idea to share my journey of understanding the protocol, how SmartThings works with it, and how can you use such knowledge to develop a device handler (without trying to bang your head too much against your desk :stuck_out_tongue: ).

The problem I’ve always came up with is that the documentation is all around the place, missing pieces, etc., and the current/old primer document isn’t enough /not clear enough.

Most of this post will apply to ANY Zigbee driver/handler development, but it is made for SmartThings.


Versioning this post with a changelog, as I keep adding comments/learnings/etc.

Post Version: V1.0 (14 Jan 2021)

Changelog:
N/A


Missing topics in this post as of V1.0:

  • How do I know which attributes are available in a Zigbee cluster without doing a ‘for’ cycle in a device handler?

Now for references to read/consult, you will need this:

1.A - The old SmartThings zigbee primer:
ZigBee Primer — SmartThings Classic Developer Documentation

1.B - The really old primer:
ZigBee Primer — SmartThings Documentation 1.0 documentation (stdavedemo.readthedocs.io)

2.A - The old SmartThings documentation on zigbee:
ZigBee Reference — SmartThings Classic Developer Documentation

3.A - The SmartThings documentation on writing zigbee handlers :
Building ZigBee Device Handlers — SmartThings Classic Developer Documentation

3.B - The really old write a zigbee handler doc
Building ZigBee Device Handlers — SmartThings Documentation 1.0 documentation (stdavedemo.readthedocs.io)

4.A - The Zigbee cluster library (ZCL) v1.5 (2018) documentation:
JN-UG-3115.book (nxp.com)


Get accustomed to the SmartThings IDE first (for the moment as of writing this article in Jan 2021, its still the groovy IDE, the new API is up now but it doesn’t have a GUI yet, its all CLI based, and you have to turn ‘developer mode’ on in your hub, so we’ll stick to the groovy based one for now).

To get started, get a device joined/paired to you hub, normally the device will have a pairing button (or a sequence like turn on/off X times, like Sonoff or Gledopto) where a light (maybe) will start blinking rapidly.

In the SmartThings App, go into the + icon to add a device, add a device, and use the “scan nerby”

Once the device is joined it’ll probably show as ‘Thing’ since it doesn’t know yet what is is (or might have a partial idea), go into the SmartThings IDE, and into my hubs, the hub you’re using, ‘list events’, and find the ‘zbjoin’ event that applies to your device, you will see something like this:

zbjoin: {“dni”:“A123”,“d”:“0012345678ABCDEF”,“capabilities”:“8E”,“endpoints”:[{“simple”:“01 0104 0051 01 09 0000 0004 0003 0006 0010 0005 000A 0001 0002 02 0019 000A”,“application”:“01”,“manufacturer”:“LUMI”,“model”:“lumi.plug”},{“simple”:“02 0104 0009 01 01 000C 02 000C 0004”,“application”:“”,“manufacturer”:“”,“model”:“”},{“simple”:“03 0104 0053 01 01 000C 01 000C”,“application”:“”,“manufacturer”:“”,“model”:“”}]

Now what does this mean?

This is basically the device ‘telling’ the gateway that its joining what Zigbee clusters it has, but they don’t necessarily have to be compliant with the ZCL (Zigbee Cluster Library).

The ZCL (Zigbee Cluster Library) basically is a standardized library for defining what exactly is each cluster, what attributes it has, can they be read/written?, what do they do, and what commands do they accept.

The problem is sometimes the manufacturers don’t always abide by the standard (I’m looking at you Xiaomi :P)


So how do you interpret the previous message?

Lets look at the data we want, which is the endpoints data that show in the ‘simple’ attribute:

[
{“simple”:“01 0104 0051 01 09 0000 0004 0003 0006 0010 0005 000A 0001 0002 02 0019 000A”,“application”:“01”,“manufacturer”:“LUMI”,“model”:“lumi.plug”},
{“simple”:“02 0104 0009 01 01 000C 02 000C 0004”,“application”:“”,“manufacturer”:“”,“model”:“”},
{“simple”:“03 0104 0053 01 01 000C 01 000C”,“application”:“”,“manufacturer”:“”,“model”:“”}
]

We’re going to pay more attention after the first XX XXXX XXXX XX digits,

Here’s my graphical explanation on what exactly is each thing in that ugly Zigbee message:

Now with the previous information (and beautiful paint-based graphic!), you should now have an ‘idea’ on how to interpret the zigbee information that the hub receives, and now you need to understand what to do with it.


Basically you do 4 things with zigbee:
A.- you configure reporting (via the zigbee.configureReporting() method), which ‘tells’ the device how much time it has to pass to report what value in an attribute in a cluster and how much it has to vary.

Example: tell me the temperature every 1 minute, but don’t wait no more than 60 minutes if it fails to send it, and only if it changes 0.1 degrees, which is on endpoint 1, cluster 0002, attribute 0, in hexadecimal format.

B.- you read a value from an attribute in a cluster (via the zigbee.readAttribute() method or the old “st rattr”), which ‘tells’ the device to report the value requested.

Example: tell me the value of endpoint 1, cluster 0006, attribute 0, which is the on/off state. (in this particular case SmartThings has added a native method called zigbee.onOffRefresh())

C.- write a value to an attribute in a cluster (via the zigbee.writeAttribute() method, or the old “st wrattr” one), which tells the device to write the value you’re sending to it.

Example: Enable the device power on on power failure memory in cluster 0x0000, attribute 0x0201, type 0x1 (boolean), with value 1.

(Bonus tip: you can even filter by the manufacturer’s ID code, for example Xiaomi has sometimes a value of 0x115F, so in the writeAttribute() method you can filter it by using [mfgCode:0x115F] as an ‘option’ of it. This mean you can write a generic handler, for example ‘Motion Sensor’, that uses the same logic for X number of vendors, but specifically runs read/write/etc methods for specific vendors which might use different clusters/attributes/types of data.

D.- Send a command (via the zigbee.command() method, or the old “st cmd”)

Example: turn you device on, which means send a 0x01 command to cluster 0x0006.


Are you following me after all this? :stuck_out_tongue:

Now the part that follows is writing your code and logic to ensure the device does the following (basically, there might be more you need to do):

  • Turn on/off (via the on() off() method)
  • Report on reportable values (but don’t abuse how many times to not overload the zigbee network). (for example via the configure() method for example)
  • read values every X amount of time if the value you’re looking for is not reportable (for example via the refresh() method + runEveryXXMinutes(refresh) method).
  • what happens when the device is installed in the hub. (for example via the installed() method)
  • what happens when the device is configured. (for example via the configure() method)
  • what happens when the device configuration is updated. (this happens when you edit settings in the * App, for example via the updated() method)
  • what happens when the hub receives a zigbee message (what you configured to report back, or maybe unsupported messages).
  • how does the device handler translate a machine value to human readable (temperature in hexadecimal to celsius degrees).
  • Create a fingerprint for the device to automatically use your handler as soon as it joins the hub.

Until you consider your device handler ready for ‘production’, don’t fear the “log.debug” method to write the values you get, or messages that go into the ‘live logging’ in the SmartThings IDE. Just remember to comment/disable the log.debug once you’re done! (We don’t want useless debug data consuming logs :slight_smile: )

The (in)famous ‘catchall’ message captures zigbee messages that your device handler may not support and might be of interest to you.


The fingerprinting as mentioned will allow you new custom/unsupported device to start using the handler you developed without manually setting it in the IDE.

This goes in the metadata part of the handler, You will use the profileId which is the second 4 digit number in the ‘zbjoin’ message, some of the ‘server clusters’ which we’ll call inClusters, some of the ‘client clusters’ which we’ll call outClusters, and you will use the data that was already gathered by the zbjoin which is the manufacturer (which as mentioned we can extract from reading cluster 0x0000 attribute ID 0x0004), and the model (from cluster 0x0000 attribute ID 0x0005), and you will define ‘placeholder name’ that will replace the ‘thing’ as soon as it joins your hub, this is an example:

fingerprint profileId: "0104", inClusters: "0000,0400,0003,0006", outClusters: "0019,000A", manufacturer: "LUMI", model: "lumi.plug", deviceJoinName: "Xiaomi Zigbee Smart Outlet"


I’ve written a device handler for the AU/NZ/AR/CN version of the Xiaomi Zigbee Smart Outlet (ZNCZ02LM), that while it might not be perfect and might have some things wrong, I’ve documented it extensively so that it can be used as an example: Xiaomi Smart Outlet Device Handler


I hope this post is useful to someone in the community so that they dont have to go through the same pain as I did :stuck_out_tongue:.

If you like it and/or find the post useful, and If you can of course, get me a coffee so I can keep programming :smiley: : Ko-Fi / Buymeacoffee

Enjoy!

12 Likes

@Chares

Great post… but how can I identify what each cluster is and the attributes available?
I’m trying to modify a device handler to support a newer Zigbee 3.0 model of a device.

Everything is documented in the cluster library. NXP has the docs online.

2 Likes

It might be beyond my pay grade to modify this driver alone, it seems that the bit values for the attributes in the original driver don’t even match up to the values for the standard attributes.

I am offering a sample to anyone who wants to help with this driver, and I can guarantee you won’t be able to get the t1 units anywhere else right now since they’re sold out in china, so this is a good opportunity for someone who wants to get their hands on one early in return for helping me out.

1 Like

Great !!
Just a quick question. Can you please explain what is “application”:“01“ means.
sometimes I getting different numbers like ,“application”:“23” or “application”:“15” . What are those numbers ?

Hi @dotan_shai, It’s been a while since I played with zigbee, from memory, application would mean what it is, light, plug, switch, etc.

Thanks, that is what I thought as well. I tried to find information is the Zigbee Spec but I did not find any.
Can you please send a link to a table that summarize all the applications that exist ?

Thanks again…

Replying my self. I find my answer here:

Page 526:

or in other words:
application: the Application Version read from attribute 0x0001 of the Basic Cluster

1 Like