ST Platform mDNS?

I have been using a home grown module for mdns in my drivers thus far. I noticed that the latest platform libs include an mdns module. Is this available for public use? I tried using it and it doesn’t seem to match the documented API. Here is the file I found in hub v46

-- Copyright (c) 2022 SmartThings.
local mdns_rpc = _envlibrequire("mdns")

--- @class mdns
local mdns = {}

--- @class ServiceInfo
--- @field public name string the informational name of the service record
--- @field public service_type string the service type, which should match the one passed in during the query
--- @field public domain string the domain on which the service is located, which should match the one passed in during the query
local ServiceInfo = {}

--- @class HostInfo
--- @field public name string The hostname on the domain.
--- @field public address string The IP address as a string at which the host can be reached.
--- @field public port integer The port number as an integer for the service on the host
local HostInfo = {}

--- @class RawTxtRecord
--- @field public text string[] an array of raw byte strings. Each element in the array represents a record in its entirety as a single byte string; this means that for key-value type records, the string has the entire <key=value> contents as a single item.
local RawTxtRecord = {}

--- @class ServiceDiscoveryEvent
--- @field private iface_info table
--- @field public service_info ServiceInfo
--- @field public host_info HostInfo
--- @field public txt RawTxtRecord
local ServiceDiscoveryEvent = {}

--- @class ServiceDiscoveryResponse
--- All responses to a service discovery request.
---
--- @see ServiceDiscoveryEvent
---
--- @field public found ServiceDiscoveryEvent[] Information about found hosts for the given service type
local ServiceDiscoveryResponse = {}

--- Perform mdns/DNS-SD discovery for service types on the given domain.
---
--- Service types in a search query are often prefixed with underscores by
--- convention to avoid collisions with existing hostnames.
---
--- For network services, they are often composed of two parts:
--- The unique service name, followed by a dot, and then the underlying
--- transport protocol being used. For example, the Philips Hue bridge
--- advertises itself as the "hue" service, and because it's a REST API it
--- goes over TCP. As such, to search for Hue bridges on the local domain,
--- you should use `_hue._tcp` as the service name, and `local` as the domain.
--- Another example would be Apple's AirPlay service, which is also TCP-backed
--- and advertises itself as "airplay", meaning the search term would use
--- `_airplay._tcp` on the local domain (again noting the underscores).
---
--- Replies to the query should be a deduplicated list of responses; however,
--- if a service supports IPv4 and IPv6 you may receive multiple responses
--- for the same host name, one for each interface. The host_info portion of
--- the response has helper methods `:is_valid_ipv4()` and `:is_valid_ipv6()`
--- to help work with this.
---
--- Philips Hue discover example:
--- ```
---     local discover_responses = mdns.discover("_hue._tcp", "local") or {} -- scan for Hue bridges on the local network
---
---     for idx, found in ipairs(discover_responses.found) do
---         -- sanity check that the answer contains a response to the correct service type, and we only want to process ipv4
---         if answer ~= nil and answer.service_info.name == "_hue._tcp" and answer.host_info:is_valid_ipv4() then
---             -- process answer
---         end
---     end
--- ```
---
--- @param service_type string the service type to search for hosts on
--- @param domain string the domain to search for hosts on
--- @return ServiceDiscoveryResponse|nil the response to the query, or nil if there was an error.
--- @return nil|string error message if any
function mdns.discover(service_type, domain)
  return mdns_rpc.discover(service_type, domain)
end

--- Resolve the IP address needed to interact with a service given the host name, service type,
--- and domain. Note that "resolve" in the context of DNS Service Discovery over mDNS can refer
--- to resolving a PTR record to an SRV record, or refer to resolving an SRV record to an A/AAAA record.
--- What this API does is:
---
---   - Given a hostname and a service type, find all of the A/AAAA records that resolve the `host`
---     argument by performing a browse for the given service type and following the PTR and SRV
---     records to find all of the relevant hostnames.
---
--- If you don't know the specific host name of the host providing the service, the `discover` API
--- will instead perform browse for the given service type and return all of the SRV,PTR,TXT, and
--- A/AAAA record information for all responders for that service on the given domain.
---
--- @see mdns.discover
---
--- @param host string the hostname to resolve
--- @param service_type string the service type to search for hosts on
--- @param domain string the domain to search for hosts on
--- @return HostInfo[]|nil the response to the query, or nil if there was an error.
--- @return nil|string error message if any
function mdns.resolve(host, service_type, domain)
  local browse, err = mdns.discover(service_type, domain)
  if err ~= nil then return browse, err end

  local resolved = {}
  if not (browse and browse.found and #(browse.found) > 0) then return resolved end

  for _, event in ipairs(browse.found) do
    local base_hostname = event.host_info.name:sub(1, -(#("." .. domain) + 1))
    if event.service_info.service_type == service_type and event.service_info.domain == domain and
        (event.host_info.name == host or base_hostname == host) then
      table.insert(resolved, event.host_info)
    end
  end

  return resolved
end

return mdns
1 Like

Hi, @blueyetisoftware!

Yes, I checked with the team.

About this, can you share the firmware version that is currently installed on your Hub, please?

1 Like

I tried it against v45 and I get responses that don’t match. I’m guessing they changed the api for v46. I’ll wait for that version and try it again. Glad they added it. It’s nice to have that available for everyone to use.

Yes, there were some changes from v45 to v46 in this regard. The team provided the file that should match your version:

-- Copyright (c) 2022 SmartThings.
local mdns_rpc = _envlibrequire("mdns")

--- @class mdns
local mdns = {}

--- @class ARecord
--- IPv4 records
--- @field public ipv4 string the ipv4 address of the resource
local ARecord = {}

--- @class AaaaRecord
--- IPv6 records
--- @field public ipv6 string the ipv4 address of the resource
local AaaaRecord = {}

--- @class CnameRecord
--- Canonical Name records
--- @field public cname string a Canonical Name record for a resource that is an alias
local CnameRecord = {}

--- @class MxRecord
--- Mail Exchange records.
--- @field public preference integer the priority of the mail exchange server; used if there is more than one to indicate which a consumer should prefer.
--- @field public exchange string the mail server
local MxRecord = {}

--- @class NsRecord
--- Nameserver record
--- @field public ns string the nameserver
local NsRecord = {}

--- @class SrvRecord
--- Service record. Contains ip and port information for a host providing a service.
--- @field public priority integer the priority of this server providing the service compared to others providing the same service (for users to know which to prefer)
--- @field public weight integer indication of traffic preference; higher weight means a server can handle more traffic. Can be used as a `priority` tie-breaker.
--- @field public srvPort integer the port
--- @field public target string the hostname/IP address.
local SrvRecord = {}

--- @class TxtRecord
--- TXT record. Arbitrary text payloads. Many security add-ons like DKIM are implemented using TXT records.
--- @field public text string the payload
local TxtRecord = {}

--- @class PtrRecord
--- PTR record. Opposite of an A/AAAA Record; it contains a domain name for an IP address record query
--- @field public ptr string the domain name for the IP address
local PtrRecord = {}

--- @class UnimplementedRecord
--- A record type that is not understood. The payload is the raw byte string.
--- @field public value string the bytes
local UnimplementedRecord = {}

--- @class RecordKind
--- A table with a single field; that field can be one of the various resource record types and field existence is used to indicate resource record type.
--- @field public ARecord ARecord (optional)
--- @field public AaaaRecord AaaaRecord (optional)
--- @field public CnameRecord CnameRecord (optional)
--- @field public MxRecord MxRecord (optional)
--- @field public NsRecord NsRecord (optional)
--- @field public SrvRecord SrvRecord (optional)
--- @field public TxtRecord TxtRecord (optional)
--- @field public PtrRecord PtrRecord (optional)
--- @field public UnimplementedRecord UnimplementedRecord (optional)
local RecordKind = {}

--- @class MdnsRecord
--- A DNS Resource Record as described at https://www.rfc-editor.org/rfc/rfc1035.
---
--- @field public name string a domain name to which this resource record pertains
--- @field public class integer class identifier for the RDATA of the record; 1 = IN, 2 = CS, 3 = CH, 4 = HS. See the RFC for more information.
--- @field public ttl integer an unsigned integer that specifies the time to live of the record for caching purposes, in seconds.
--- @field public kind RecordKind existential indicator of the type of Resource Record and its associated payload
local MdnsRecord = {}

--- @class MdnsResponse
--- A response to an mDNS request. Contains three lists of DNS Resource Records.
--- More information on the types of Resource Records in responses can be found
--- here: https://www.rfc-editor.org/rfc/rfc1035
---
--- @see MdnsRecord
---
--- @field public answers MdnsRecord[] the primary answer(s) to the mDNS question issued
--- @field public additional MdnsRecord[] additional information about the queried resource that was not directly asked
--- @field public nameservers MdnsRecord[] Known as "authoritative" in the DNS specification. Records that refer to authoritative nameservers related to the query.
local MdnsResponse = {}

--- Perform mdns/DNS-SD discovery for service types on the given domain.
---
--- Answers to this can contain a variety of records due to the multicast
--- nature of mDNS, but the answers that will contain the information related
--- to the search query will typically be `PTR` records
---
--- Service types in an mDNS search query are often prefixed
--- with underscores by convention to avoid collisions with existing DNS
--- hostnames. For network services, they are often composed of two parts:
--- The unique service name, followed by a dot, and then the underlying
--- transport protocol being used. For example, the Philips Hue bridge
--- advertises itself as the "hue" service, and because it's a REST API it 
--- goes over TCP. As such, to search for Hue bridges on the local domain,
--- you sould use `_hue._tcp` as the service name, and `local` as the domain.
--- Another example would be Apple's AirPlay services, which is also TCP-backed
--- and advertises itself as "airplay", meaning the search term would use
--- `_airplay._tcp` on the local domain (again noting the underscores).
---
--- Philips Hue discover example:
--- ```
---     local discover_responses = mdns.discover("_hue._tcp", "local") or {} -- scan for Hue bridges on the local network
---     
---     for idx, answer in ipairs(discovery_responses.answers) do
---         if answer.kind.PtrRecord ~= nil and answer.kind.PtrRecord.ptr:find("_hue._tcp") then -- this answer contains a response to the search query
---             -- process answer
---         end
---     end
--- ```
---
--- @param service_type string the service type to search for hosts on
--- @param domain string the domain to search for hosts on
--- @return MdnsResponse the response to the query. In general, the most useful information will be in the form of PtrRecord or SrvRecord in the answers or additional sections.
function mdns.discover(service_type, domain)
    return mdns_rpc.discover(service_type, domain)
end

--- Perform mdns/DNS-SD resolution for hosts of the given service type,
--- on the given domain.
---
--- Answers to this can contain a variety of records due to the multicast
--- nature of mDNS, but the answers that will contain the information related
--- to the search query will typically be SRV records in the `answers` field,
--- or A records in the `additional` field on the response payload.
---
--- Service types in an mDNS search query are often prefixed
--- with underscores by convention to avoid collisions with existing DNS
--- hostnames. For network services, they are often composed of two parts:
--- The unique service name, followed by a dot, and then the underlying
--- transport protocol being used. For example, the Philips Hue bridge
--- advertises itself as the "hue" service, and because it's a REST API it 
--- goes over TCP. As such, to search for Hue bridges on the local domain,
--- you sould use `_hue._tcp` as the service name, and `local` as the domain.
--- Another example would be Apple's AirPlay services, which is also TCP-backed
--- and advertises itself as "airplay", meaning the search term would use
--- `_airplay._tcp` on the local domain (again noting the underscores).
--- 
--- The discover call is used after scanning to find the hostname(s) 
--- of the services you are scanning for, to resolve their IP addresses:
---
--- Philips Hue example:
--- ```
---     local discover_responses = mdns.discover("_hue._tcp", "local") or {} -- scan for Hue bridges on the local network
---     for _, answer in ipairs(discovery_responses.answers) do
---         if answer.kind.PtrRecord ~= nil and answer.kind.PtrRecord.ptr:find("_hue._tcp") then -- this answer contains a response to the search query
---             local host = answer.kind.PtrRecord.ptr:match("[^%.]+") -- Extract substring before first `.` to get hostname
---             local resolve_resp = mdns.resolve(host, "_hue._tcp", "local") or {}
---             for _, response in ipairs(resolve_resp.additional or {}) do -- here we're looking up the ARecords in the additional responses to get the IPv4 address
---                 if response.kind.ARecord ~= nil then
---                     log.info(string.format("Found Hue bridge with IP address %s", response.kind.ARecord.ipv4))
---                 end
---             end
---         end
---     end
--- ```
---
--- @param host string the host to do lookup on
--- @param service_type string the service type to search for hosts on
--- @param domain string the domain to search for hosts on
--- @return MdnsResponse the response to the query. In general, the most useful information will be in the A Records, which can be in both the answers and additional sections.
function mdns.resolve(host, service_type, domain)
    return mdns_rpc.resolve(host, service_type, domain)
end

return mdns

That looks correct. Thanks for following up. Things are growing nicely on Edge :+1:

1 Like

@nayelyz Did they suggest anything regarding using this for both v45 and v46 since most users are still on v45? Are we best off just waiting until v46 is officially rolled out, or is there some sort of wrapper that makes v45 compatible as well?

Hi, @blueyetisoftware.

Sorry for the delay, I already asked the team about this. I’ll keep you posted.

1 Like

Hi, @blueyetisoftware.
Following up, the team mentioned the following:

The functions take the same inputs across the API boundary, it’s just the structure of the output that’s different. So, you can support both by just having two different output processing code paths.

1 Like

Perfect. I’ll give that a try

1 Like

@blueyetisoftware @nayelyz

Question about how the built-in mDNS function works. Is the hub always listening on the mDNS multicast address and maintaining an internal table of discovered devices? Or, is it doing a ‘live’ listen on the multicast at each request?

I want to know, if I call it from my driver, is it returning only a list of devices broadcasting at that particular time period, or is it pulling from a maintained table of devices that may have been discovered previously?

It broadcasts right when you call it. Seems to work pretty well and handles all of the spawning and waiting internally. You just call it like a function and get back the data. Trickiest part was dealing with the different data coming from v45 vs v46. I ended up wrapping the v45 data and making it look like v46 data.

1 Like

I was hoping it would maintain a table. So this means you still have to worry about not catching a device’s broadcast message at the right time. Meaning multiple calls may be needed.

Where are you finding the documentation for the API? Does it have a parameter to tell it how long to do the scan for?

I just pulled them from the ST platform repo to see their API. Both are captured in this thread.

Thanks.

You’ve now used both my library and this one - what would you say are the advantages of the API?

Here is the code I used to wrap the v45 response, so I could use just v46

local collection = require 'collection'

local mdns45 = {}

function mdns45.fwd_compat_v45(response_v45)
    local found = {}

    local srv_records = collection(response_v45.additional):filter(function(_, record)
        return record.kind.SrvRecord ~= nil and record.kind.SrvRecord.target ~= nil
    end):remap(function(_, record)
        return record.name, record
    end)

    local a_records = srv_records and collection(response_v45.additional):filter(function(_, record)
        return record.kind.ARecord ~= nil and record.kind.ARecord.ipv4 ~= nil
    end):remap(function(_, record)
        return record.name, record
    end)

    local txt_records = collection(response_v45.additional):filter(function(_, record)
        return record.kind.TxtRecord ~= nil and record.kind.TxtRecord.text ~= nil
    end):remap(function(_, record)
        return record.name, record
    end)

    collection(response_v45.answers):filter(function(_, answer)
        return answer.kind.PtrRecord ~= nil and answer.kind.PtrRecord.ptr ~= nil
    end):each(function(_, answer)
        local ptr = answer.kind.PtrRecord.ptr
        local srv = srv_records[ptr]
        local a = srv and a_records[srv.kind.SrvRecord.target]
        local txt = txt_records[ptr]

        local host, service_type, domain = ptr:match('^([^%.]+)%.(.+)%.([^%.]+)$')

        table.insert(found, {
            service_info = {
                name = host,
                service_type = service_type,
                domain = domain
            },
            host_info = {
                name = host and service_type and string.format('%s.%s', host, service_type),
                address = a and a.kind.ARecord.ipv4,
                port = srv and tonumber(srv.kind.SrvRecord.srv_port),
            },
            txt = txt.kind.TxtRecord.text,
        })
    end)

    return { found = found }
end

return mdns45

In use…

local mdns = require 'st.mdns'
local mdns45 = require 'mdns45'

-- scan for devices on the local network
local discovery_responses = mdns.discover('_hue._tcp', 'local') or {}

-- if response in v45 format, convert to v46
if discovery_responses.answers ~= nil then
    discovery_responses = mdns45.fwd_compat_v45(discovery_responses)
end
1 Like

I used yours, and then eventually landed on my own implementation that I used for my devices. I migrated to this to see how it was different. All 3 are pretty similar. The biggest difference for me was just being able to remove a shared dependency that I had to package into all of my drivers. Getting it from the platform shrinks my footprint and simplifies my local code. I also liked being able to call the module as a simple function rather than managing the socket myself.

There isn’t a big difference between them. My hope is that the platform one will grow to be more fully featured down the road, including a history of devices as you mentioned.

OK, thanks! I’m just trying to decide if it’s worth it right now to switch over. I probably will eventually for the reasons you stated.

I ran up against a really weird one with a Google device that wasn’t formatting the TXT record correctly with key-value pairs and I was wondering if there was something I was missing in my code. But I’ve checked and double-checked the spec. I’ll be curious to see what this library does with it…

I didn’t get any new “features” out of the gate. It was just a technical debt thing. I haven’t pushed the converted version out to any users. Based on the other modules in the platform, I think it is also doing some IP and port validation, which I also had on my own. They also remove duplicate entires which was nice.

@nayelyz

I got a chance to try the new built-in mDNS function and I wanted to pass along two feedback items to the engineering team:

  1. I see that it is only scanning the multicast address for about 1.2 seconds which is not sufficiently long. Some IOT devices - especially battery-powered ones - only broadcast mDNS messages about once every 30 seconds. The chances of catching these kinds of devices with this library are pretty small unless you put it into a loop. I think a better approach would be to add a scan duration parameter to the discover method to allow developers to set the appropriate time to listen.

  2. The return format of the TXT records is unnecessarily inconvenient. 98% of modern mDNS messages should have a <key>=<value> format in the TXT record as called out in the latest specification. I realize that some manufacturers still put plain text strings in these records. Rather than return a table of byte values, why not just return a table of strings? You can leave it to the programmer to parse the key/value pairs, which is going to be the vast majority of cases.

Other than that, I’m glad to see the capability included, so thanks to the team.

Thanks.

This is what I did which worked well for discovery

while should_continue() do
    local discovery_responses = mdns.discover('_hue._tcp', 'local') or {}

    -- if response in v45 format, convert to v46
    if discovery_responses.answers ~= nil then
        discovery_responses = mdns45.fwd_compat_v45(discovery_responses)
    end

    -- do something with the results

    -- yield so should_continue can be evaluated
    socket.sleep(0.2)
end

I’m seeing an array of strings. Are you making a distinction between a binary string vs utf-8 string?