Protocol Buffers or raw sockets examples?

Developer Programs Writing Edge Drivers
1

I’m looking into building an Edge driver to integrate with ESPHome. If you haven’t heard of it, ESPHome is a very flexible and powerful firmware framework for ESP8266 and ESP32 devices. I have my own personal motivations for building such an integration, but I also think that if it’s done well, this would be a huge benefit for the SmartThings and home automation community as a whole because it opens up so much capability for DIY sensing and actuating devices.

ESPHome’s native API is based on Protocol Buffers: Native API Component — ESPHome

The SmartThings Edge documentation makes a brief mention of Protocol Buffers [Get Started with SmartThings Edge | SmartThings Developers] but no details or examples that I could find.

In our initial proof-of-concept experimentation, we were unable to connect to the ESPHome device from a ST Edge socket. Currently the suspicion is that in order to connect, it needs to be over a raw socket but cosock and LuaSockets libraries only expose TCP sockets (as far as I could tell).

I found a pure-Lua implementation of protocol buffers here [GitHub - Neopallium/lua-pb: Lua Protocol Buffers] but without the ability to pass the encoded message over a raw socket, it doesn’t seem too helpful.

So, to my question: Are there any examples or sample code for actually implementing Protocol Buffers on ST edge? Is there any reference documentation on the cosock.socket APIs mentioned here [Socket — SmartThings Edge Device Drivers documentation]?

Thanks in advance!
Nate

2

Tagging @nayelyz @TAustin

3

I’ve never used them so my ability to help is limited. However, my understanding is that protocol buffers can use any network protocol, be it TCP, UDP, HTTP, etc. Aren’t your devices using TCP?

I’m happy to try and help any way I can.

EDIT: I’ve looked at the link you provided. This is stated:

The ESPHome native API is based on a custom TCP protocol using protocol buffers.

There is a linked client example written in Python. From the code, the socket connection looks to be a standard TCP socket:

async def _connect_socket_connect(self, addr: hr.AddrInfo) -> None:
        """Step 2 in connect process: connect the socket."""
        self._socket = socket.socket(
            family=addr.family, type=addr.type, proto=addr.proto
        )
        self._socket.setblocking(False)
        self._socket.setsockopt(socket.IPPROTO_TCP, socket.TCP_NODELAY, 1)
        # Try to reduce the pressure on esphome device as it measures
        # ram in bytes and we measure ram in megabytes.
        try:
            self._socket.setsockopt(socket.SOL_SOCKET, socket.SO_RCVBUF, BUFFER_SIZE)
        except OSError as err:
            _LOGGER.warning(
                "%s: Failed to set socket receive buffer size: %s",
                self.log_name,
                err,
            )

There are some lesser-used socket options that Edge doesn’t make available to drivers, and it’s possible this may be an inhibitor, but other than that, I don’t see anything else obvious that should prevent you from implementing this in an Edge driver using the cosock library.

1 Like
4

Is there an API reference for this library? All I could really find was a brief mention of it here.

5

Hi @heythisisnate after talking with the team and they commented the next, all the socket calls on aioesphomeapi use TCP sockets so raw sockets aren’t required, Neopallium doesn’t support Protobuf 3 with is required for ESPHome API and this library messes with the package loader.

Unfortunately, we don’t have Protocol Buffers example implementations currently

6

Yea, they don’t give us much to go on do they?! Cosock supports much of the basic Lua socket support documented here.

You can also look at the cosock repository on github. No API doc last I looked, but they do have a number of examples that might help.

7

Well that’s disappointing. I couldn’t find any other pure-lua implementation. If you’ve come across anything I’d appreciate.

Thanks, this is helpful.

We’ll keep going at it and see if we can figure it out. Thank you for the insights.