[ST Edge] Persistent Store

@nayelyz,

Apart from this, which has also happened to me.
I have seen that only when you make a change in some value of the preferences, the lifecycle infoChanged is executed.
This is fine to manage if you have to take any action, such as stopping a timer or resuming it.
But I have seen that infoChnged is executed approximately every 1 hour and if at this moment a timer is being executed, for example to progressively change the level of a light bulb, the function stops the timer if you do not have a function implemented to check if there has been any change in preferences before stopping timer.

To make sure and implement or not this function of checking changes in preferences:
This periodic 1 hour run of infoChanged, is it programmed by default as the heath poll every 30 sec or is it an error?



Meanwhile, if your timer is set based on the changes to the preferences, I suggest you add a condition using the persistent store to verify if they actually changed. Eg.

for id, value in pairs(device.preferences) do
      local oldPreferenceValue = device:get_field(id)
      print("oldPreferenceValue " .. oldPreferenceValue)
      local newParameterValue = device.preferences[id]
      print("newParameterValue " .. newParameterValue)
      if oldPreferenceValue ~= newParameterValue then
        device:set_field(id, newParameterValue, {persist = true})
        print("change device parameter")
      end
end
1 Like

I implemented it in a similar way, but I like this way better.
I see who knows!! :grinning:

Thank you.

This is also great for debugging.

2021-09-22T11:56:45.084503931+00:00 PRINT Zigbee Level ColorTemp Bulb v2  device.preferences[infoChanged]=      version
2021-09-22T11:56:45.090593931+00:00 PRINT Zigbee Level ColorTemp Bulb v2  oldPreferenceValue
2021-09-22T11:56:45.101922598+00:00 PRINT Zigbee Level ColorTemp Bulb v2  newParameterValue
2021-09-22T11:56:45.148260931+00:00 PRINT Zigbee Level ColorTemp Bulb v2  device.preferences[infoChanged]=      progOn  Yes
2021-09-22T11:56:45.161653265+00:00 PRINT Zigbee Level ColorTemp Bulb v2  oldPreferenceValue    No
2021-09-22T11:56:45.191974931+00:00 PRINT Zigbee Level ColorTemp Bulb v2  newParameterValue     Yes
2021-09-22T11:56:45.201521598+00:00 PRINT Zigbee Level ColorTemp Bulb v2  change device parameter old, new:     No      Yes
2021-09-22T11:56:45.207693598+00:00 PRINT Zigbee Level ColorTemp Bulb v2  device.preferences[infoChanged]=      offTimeMax      1.0
2021-09-22T11:56:45.214041265+00:00 PRINT Zigbee Level ColorTemp Bulb v2  oldPreferenceValue    1.0
2021-09-22T11:56:45.225017931+00:00 PRINT Zigbee Level ColorTemp Bulb v2  newParameterValue     1.0
2021-09-22T11:56:45.248109598+00:00 PRINT Zigbee Level ColorTemp Bulb v2  device.preferences[infoChanged]=      onTimeMax       15.0
2021-09-22T11:56:45.255618265+00:00 PRINT Zigbee Level ColorTemp Bulb v2  oldPreferenceValue    15.0
2021-09-22T11:56:45.261546931+00:00 PRINT Zigbee Level ColorTemp Bulb v2  newParameterValue     15.0
2021-09-22T11:56:45.267847931+00:00 PRINT Zigbee Level ColorTemp Bulb v2  device.preferences[infoChanged]=      progOff No
2021-09-22T11:56:45.274648598+00:00 PRINT Zigbee Level ColorTemp Bulb v2  oldPreferenceValue    No
2021-09-22T11:56:45.280817598+00:00 PRINT Zigbee Level ColorTemp Bulb v2  newParameterValue     No
2021-09-22T11:56:45.287111265+00:00 PRINT Zigbee Level ColorTemp Bulb v2  device.preferences[infoChanged]=      onLevelEnd      70
2021-09-22T11:56:45.294149598+00:00 PRINT Zigbee Level ColorTemp Bulb v2  oldPreferenceValue    70
2021-09-22T11:56:45.306681265+00:00 PRINT Zigbee Level ColorTemp Bulb v2  newParameterValue     70
2021-09-22T11:56:45.337803598+00:00 PRINT Zigbee Level ColorTemp Bulb v2  device.preferences[infoChanged]=      onLevelStart    20
2021-09-22T11:56:45.346047931+00:00 PRINT Zigbee Level ColorTemp Bulb v2  oldPreferenceValue    20
2021-09-22T11:56:45.353001598+00:00 PRINT Zigbee Level ColorTemp Bulb v2  newParameterValue     20
2021-09-22T11:56:45.359325265+00:00 PRINT Zigbee Level ColorTemp Bulb v2  device.preferences[infoChanged]=      ifPushSwitch    Off
2021-09-22T11:56:45.366870598+00:00 PRINT Zigbee Level ColorTemp Bulb v2  oldPreferenceValue    Off
2021-09-22T11:56:45.374449931+00:00 PRINT Zigbee Level ColorTemp Bulb v2  newParameterValue     Off

I changed code a bit because it gave error when concatenating strings, when some preference has null value.

Thank you!!

--- Update preferences ---
local function do_Preferences (self, device)
  for id, value in pairs(device.preferences) do
    print("device.preferences[infoChanged]=", id, device.preferences[id])
    local oldPreferenceValue = device:get_field(id)
    print("oldPreferenceValue ", oldPreferenceValue)
    local newParameterValue = device.preferences[id]
    print("newParameterValue ", newParameterValue)
    if oldPreferenceValue ~= newParameterValue then
      device:set_field(id, newParameterValue, {persist = true})
      print("change device parameter old, new:",oldPreferenceValue, newParameterValue)
     
    --device.thread:cancel_timer()
    for timer in pairs(device.thread.timers) do
      device.thread:cancel_timer(timer)
    end  
   end
 end
end

@nayelyz I’d like to get some guidance from SmartThings on the use of the {persist = true} option with the device:set_field() method. I read in the docs somewhere that you have to be careful how much you use this because it can literally shorten the life of your hub’s flash memory if you update too often. But how often is too often?

I’ve been very careful to keep its use in my drivers to a minimum, but I have one that I’m working on now that could potentially be used frequently, depending on user automations, which is out of my control. I’m concerned about it and would like to warn users about it as well, but maybe I’m being paranoid…

Hey @TAustin, we appreciate the care. So long as the key space and values that are changing aren’t extremely large, frequent writes to the store from lua are probably not a major concern. Behind the scenes, the store in platform only persists to non-volatile storage in batches at a time interval (or on graceful shutdown events). Currently, this interval is hourly, though that is subject to change at any point. Writes to the store will be “persisted” (just not to non-volatile) in the case of driver restarts/updates even if not written all the way through to flash.

4 Likes

@posborne
Is there a size limit on the set_field value of {persist = true}?

Thank you Paul. I guess I’ll put my fears aside! The fact that the memory writes are not actually happening in real time makes all the difference. But it does raise a different point that because they aren’t, the values can’t be assume to be up-to-date in the event of a driver crash.

Right now there isn’t an explicit limit, though we are discussing limits that will likely be added in the near future. Do you have a particular use case you are concerned might exceed a reasonable limit (let’s say if we allowed each driver to store 64K of serialized data as an example).

1 Like

The data will be persisted across driver crashes just fine as the “store” operation leaves the sandbox, it just isn’t serialized all the way to flash. Most all of the system calls go out over IPC (basically a syscall layer) to another process that manages details like serialization to flash, etc.

1 Like

for example I can imagine that my nano leaf settings including a long scene list in json format can exceed 64kb limit

I was going to say that Bill Gates said 64KB should be enough for anybody, but I guess that was 640K and he might not have actually said that. That doesn’t sound like an unreasonable use case; we’ll document whatever limit is decided and the behavior that can be expected if it is exceeded. We don’t want to needlessly prevent any reasonable (in our opinion) use case but need to balance that with other concerns given that we target a range of variously constrained targets.

In the case of a long scene list, I think the workaround if you were pushing a limit would be to filter that down to the subset of the data that is needed rather than storing the whole blob.

so you propose to make this driver developer problem?

what you can try first instead

  • use binary storage, protocol buffers or archives will fit JSON to 64kb

  • use permissions, just let the user allow “unsafe” 640kb storage