Bug in st.json for explicit JSON null values

Developer Programs Writing Edge Drivers
1

ST is recommending that we use st.json instead of dkjson as our encoder and decoder in Edge drivers. I have noticed that st.json does not support the constant json.null for explicit null values. It also doesn’t support defining a null placeholder in the encode/decode functions. This explicit null is used by some devices to indicate a field has been deleted. If we need this feature, should we use dkjson? Can this be added to st.json?

Here is an example that works in dkjson

json.decode(msg.data, 1, json.null) -- replace explicit nulls with json.null
json.decode(msg.data, 1, 'NULL') -- replaces explicit nulls with 'NULL' as a placeholder

Dkjson provides functions like this

`json.decode (string [, position [, null]])`
--------------------------------------------

Decode `string` starting at `position` or at 1 if `position` was
omitted.

`null` is an optional value to be returned for null values. The
default is `nil`, but you could set it to `json.null` or any other
value.

json.null is defined like so and serializes as lowercase ‘null’

json.null = setmetatable ({}, {
  __tojson = function () return "null" end
})
2

To clarify, the st.json implementation does kind of support explicit nulls. The result is that when you decode them, you get a userdata with nil in it. For example:

-- decode this
'{ "foo": null, "bar": "val" }'

-- get this
{
  foo = userdata, -- userdata of nil
  bar = 'val'
}

which you would need to distinguish from just this which is non-explicit. I ended up doing type() checking to make the distinction.

-- decode this
'{ "bar": "val" }'

-- get this
{
  bar = 'val'
}

The dkjson version has that placeholder capability to allow you to index it easily. So the “bug” in this case would be that st.json is not a drop in replacement for dkjson.

3

Hi, @blueyetisoftware
I already asked the engineering team about this. Once I get more info, I’ll let you know.

1 Like
4

Hi, @blueyetisoftware
The team confirmed there’s a bug but also, it is be important for them to identify correctly what you expect from this library, would it be to have a better way to encode and compare against null JSON values with a usable constant, or for the st.json API to allow for overriding the null placeholder?

1 Like
5

The important part is to be able to compare against null with a constant for both encode/decode. I don’t care about overriding it myself. I only went down that road when the constant didn’t exist. I think the override would only be useful if the json output was being sent to another library or device that utilizes a different placeholder constant.

6

Recently, I was expecting anything different of nil and I was receiving userdata of nil.

So, it broke my logic because userdata of nil is different of nil.

Something like:

if (device.preferences.someNullableString ~= nil) then
  ...
end

or simply:

if (device.preferences.someNullableString) then
  ...
end
7

Yep. You would have to change it to this if you were expecting table objects:

if (type(device.preferences.someNullableValue) == 'table') then
  ...
end

In your case, if you want ‘nil’ for an explicit ‘null’, you would want the override functionality so you could tell the library to encode all nulls as nil instead of the null constant that I am asking for.

@nayelyz This would be another case for allowing the user to override the desired value. Most users may actually prefer nil as the default.

1 Like
8

In my case, I was expecting string.

I had to avoid type userdata ( type(value) ~= “userdata” )

9

Hi, @blueyetisoftware, @w35l3y
I added the other comments to the report created by the team, thank you for sharing.

2 Likes