[SmartThings Edge] Issue with HTTP requests in LAN drivers

That’s some really good detective work. And a very odd result. It would be enlightening to capture a trace of the TCP packets with tcpdump or wireshark to see what was actually being received from the device. It seems as you say to be in the underlying socket library code and maybe not in the Lua library itself.

Have you tried that alternate http library I pointed you to? Same result?

I had also found an anomaly with the Lua http library in that it sends the headers and body in two separate socket sends. Technnically, there should be nothing wrong with that, but this was enough to change the timing on the device end, which caused me all sorts of grief. And was the reason why I wasn’t getting the same result when using curl, browser, or Python code.

I hope a solution is found, but there is always a possibility that there is something in the TCP layer coming from the device itself that’s causing this.

Not sure on this since it doesn’t occur until using cosock.asyncify. It works fine when using require. It also works when making the calls locally off the hub, so I don’t think it is the device (Bond Bridge).

Not yet. Planning to try today. I may also dive into the platform tcp lua code.

Hi, @TAustin, @blueyetisoftware.
Thank you for sharing that info, I already mentioned this to the team. Once they provide their feedback, I’ll come back here.

3 Likes

As a follow-up, the team mentioned it is a bug in cosock and raised this issue in the repo:

2 Likes

Awesome. Thanks for having them add it. This was a head scratcher for a while.

Looks like the fix is also already up. That was quick.

Is there any way to check the firmware version in the driver so that we can use asyncify on versions that have this fix, but require prior to that.

1 Like

Is there possibly a workaround to be had by adjusting the timeouts? My driver is not able to use cosock until this rolls out to the hub.

Considering: the different statuses you saw; where the prefix was ending up; noting the effect of logging; and bearing in mind no one else seems to have noticed what is actually a rather critical bug; I am not seeing timeouts as the cause of the partial reads. To me it looks more like instead of sending

HTTP/1.1 200 OK\n

and indeed sending it with the rest of the headers and even the body, the server has actually split it into teeny, tiny chunks and sent something more like

HTTP/1.1 (with trailing space)
200
OK (with leading space)
\n

The spaces could also have been separate, I just didn’t see evidence for that.

I’d like to be missing a fundamental point here.

Sorry but no, the team mentioned this info is not available for the driver.

I agree on how the packets seem to be chunked. My question was spurred by @robert.masen 's comment on the pull request

I belive the passthrough for receives here is incorrectly appending the prefix argument across timeouts instead of just applying it to the whole message. I would expect that we would want to only perform that concatenation once

I’m going to end up packaging my own version of socket.http for now to get the driver running.

Ah I see. He did rather change the emphasis from the small chunks which illustrated the problem, to the timeouts which I guess could be more prevalent in practice.

It is a nice and timely catch though.

I prefer your workaround to the original code.

1 Like

Here is the workaround for anyone else that may see this. Works with asyncify on v42 and v43.

local _, code, _, status = http.request {
    method = method,
    url = url,
    headers = headers,
    source = source,
    sink = sink,

    -- modify receive call for prefix bug in ST tcp implementation
    -- allows code prior to a fix to work with cosock asyncify
    -- https://github.com/cosock/cosock/issues/26
    create = function()
        local sock = socket.tcp()
        function sock:receive(pattern, prefix)
            local data = socket.tcp.receive(self, pattern)
            if prefix ~= nil and data ~= nil then
                return prefix .. data
            else
                return data
            end
        end

        return sock
    end
}
2 Likes