Event Data Limits

As part of ongoing efforts to balance openness with performance, we are implementing a limit to how much data can be stored by an event. The change applies to three parts of an event: descriptionText, value, and data. For descriptionText and value fields, when the limit is reached, the message will be truncated with an ellipsis. When the data field exceeds the limit, in order to avoid saving invalid data such as malformed JSON, the value will be set to null.

This change will be implemented today. Users impacted by this change may lose data from trace events which should only impact live logging.

Regards,
The SmartThings Community Team

1 Like

So what is the quantitative limit? A certain number of characters?

5 Likes

I think this is like the Google puzzle. If you come up with the correct number(s), you get hired :grin:

5 Likes

Let me see …

Description text: 36 bushels
Value fields: 16 gallons
Data field: 14 hectares

… am I hired? :stuck_out_tongue_winking_eye:

Always a guessing game and a lot of speculation that come with/follow these ST announcements :unamused:

2 Likes

Sounds like it’s more of a “you’ll know the limit once you’ve hit it” kind of situation.

2 Likes

I’m betting it is some derivative of π.

1 Like

PM @ady642 to see if you’re close :smiley:

Yeah, I thought about that, LOL …

So what does this mean in layman terms? AKA: someone who has a fairly high number of devices, with a bunch of webCoRE pistons, but doesn’t speak computer🤪?

:thinking: I actually have no idea what I just read…:roll_eyes:

It probably won’t impact 99(.9)% of users. I’m guessing there are a few apps/users who are using it to store and pass large amounts of data between apps which wasn’t the intended design for events. From what I understand events are cached on a high speed system to keep in sync but that means it can only process limited amounts of data without impacting the system performance.

Maybe the staff can identify if there are specific apps which would be impacted or outline the limits so that developers can be extra cautious to build in safety checks.

2 Likes

You are not likely to encounter this.

When events are stored, there are often data, descriptionText, and value fields associated with that event. Usually these are very small bits of information but occasionally a custom device or SmartApp will store a bunch of information in these fields.

Here are some examples of how I believe this will look.

This is a typical lock event with a small amount of information in the data and description fields:

If the amount of information in the data field is too large, null will be stored:

For the description field, the information will be truncated with ellipsis similar to this:

A typical value for a lock event may look like this:

If the information in the value field is too large, it will also be truncated with ellipsis similar to this:

3 Likes

I am a little confused about the reference to “trace events which should only impact live logging”. are there limits on log.trace as part of this?
Also, where can we find documentation of what these new limits are?

2 Likes

Answer is 42

2 Likes

over 3 weeks with no response…
@Brad_ST /@SmartThings can you provide some details on what this change impacts and what the actual size limit(s) are so we can assist people that might encounter seemingly random failures?

1 Like

I believe I may have encountered this, and it has sadly broken my RainMachine integration :frowning:

My integration uses hubAction calls to call out to my Rainmachine via LAN. It then parses those messages with the parse(evt) method. One of the calls is just over 20KB - a good bit of data but not large by most measurements.

It blows up when I do parseLanMessage(evt.description)

error java.lang.RuntimeException: bad character in base64 value @line 250 (parse)

In this case, it’s the call to get all programs. I guess I can try refactoring to try and call each zone individually, but that’s going to suck pretty hard.

Is there any other solution to this scenario?

1 Like

I ran into this today, the limit for a value stored in an event is 9,999 characters. Haven’t found a way around it, but wanted to at least post the number here for future searchers.