Multi-threading and http.request or xml parsing concurrency issues

Developer Programs Writing Edge Drivers
#1

Is every http.request() call off of the http table able to run concurrently off the same instance of the http table?

When I have multiple calls happen simultaneously, it seems like the http.request calls block till some issue gets resolved and then proceed a while later (seconds, not micro-seconds).

I am using code that looks something like this…

local cosock = require "cosock"
local socket = cosock.socket
local http = cosock.asyncify "socket.http"

local xml2lua = require "xml2lua"
local xml_handler = require "xmlhandler.tree"


local url = 'http://'..ip..':'..config.PORT..'/xml/description.xml'
local _, status = http.request({
      url=url,
      sink=ltn12.sink.table(res)
})

local xmlres = xml_handler:new()
local xml_parser = xml2lua.parser(xmlres)
xml_parser:parse(table.concat(res))

Not sure if the http requests or the xml parsing is causing issues when multiple threads call this function or similar ones simultaneously.

#2

Hi, @schwark!

I already asked the engineering team about this. I’ll keep you posted.

#3

Following up on this, @schwark. The team mentioned the following:

  1. As there is only 1 OS thread available, the driver can support multiple calls concurrently but not simultaneously.
    • This means that each request will make as much progress as possible until it needs to wait on a "receive" at which point it yields control of the program to some other co-routine.
  2. “Is every http.request() call off of the http table able to run concurrently off the same instance of the http table?”
    • There aren’t any instances of an http table. http is a module so request is just a function attached to that module, this means two calls to request do not interact with one another.
  3. It seems part of the problem is that after the request completes, it may be taking some time to make it through xml_parser:parse which can block other requests from moving forward.
#4

yes, it may be the xml parser… All kinds of weird stuff happening with xml parsing - some of the entities are being partially decoded/encoded leading to unparseable xml - and it is unpredictably so - running it ten times can lead to almost ten different versions of parsed xml.