65 devices offline since April 29th — Firmware 0.60.13 — SmartThings Station Hub

General Discussion
1

Hi all,

I’m experiencing a large-scale device dropout that started on April 29th at around 18:08 UTC and have not been able to recover since.

Setup:

  • 5 SmartThings Stations in a Hub Group (4 Stations + 1 primary)

  • Hub firmware: 000.060.00013

  • Mix of Matter (39 devices) and Zigbee (22 devices) affected

  • Also 3 OCF and 1 LAN device offline

What happened:

  • On April 29th at exactly 18:08 UTC, 62 out of 65 devices dropped offline simultaneously — both Matter and Zigbee at the same timestamp, which strongly suggests a hub-side event rather than a device or RF issue

  • A smaller second wave dropped on May 2nd at 12:33 UTC

What I’ve already tried:

  • Full hub reboot (multiple times) — no effect

  • Power cycling all mains-powered devices — no effect

  • Changing the primary hub within the Hub Group — no effect

  • Verified Zigbee channel (24) — no interference issue

Key observations:

  • All offline devices show as connected to the primary hub (expected in a Hub Group)

  • The simultaneous dropout of both Matter and Zigbee at the exact same second points to a firmware or cloud event

  • Devices that stayed online are all Virtual switches, MQTT, Mobile, and Hub types — only radio-connected devices dropped

Ask:

  • Has anyone with firmware 0.60.13 on a SmartThings Station seen the same pattern?

  • Is there any recovery path short of deleting and re-pairing 60+ devices?

  • Can SmartThings staff check hub logs for my hub around April 29th 18:08 UTC?

Thanks

2

Just curious, did you try turning off some of the hubs in the group.

For example, try running with only the primary?

3

Hi @nuno7

Could you please confirm if you were able to submit the hub logs at the time of the issue?

4

When you say ‘offline’, is that just as reported by the mobile app, or have local automations stopped working as well? Are the hub LEDs still showing you connected to the backend or are they showing you disconnected?

Are you actually able to re-pair devices?

5

I requested the logs “dump hub logs”, but haven’t received anything yet.

6

I turned them all off, and turned back on one at a time. After that I also changed the primary to a different one.

7

The hubs are connected, the automations that rely on the “offline” devices don’t work. I tried re-pairing one device and it worked, it showed up as a new device. Doing this 64x more times is the problem, and some of them are not easily accessible (relays inside ceiling fancoils)

8

I’m on 16.12 and having same problems, it’s really bad at the minute, it’s been about the third time in the last week, before that it was all vey stable, sometimes using webapp seems to help more than the app to get them to pick back up, gone from 3 offline to 15 now, all zigbee

9

Hi,

Thank you for the logs you’ve shared so far.

Could you please confirm if the logs you submitted were captured only from your primary hub, or if you also included logs from any additional hubs?

10

hey,

same kind of issue looks like hub side problem not devices, because all dropped same time (matter + zigbee both) so not normal.

i think before re-pair 60+ devices try few things:
maybe power off hub for like 15-20 min (not simple reboot) then start again
also try without hub group (single hub) just for testing if group causing issue

firmware 0.60.13 maybe bug or cloud side event at that time

better if support team can check logs from april 29 18:08 UTC, because too exact timing

11

Hi @JoseRizal @uncledaveghr
Could you please enable impersonation access so our engineering team can investigate further?

  1. Confirm the email account registered in the forum is the same one you use for SmartThings. If not, please share it with me over DM.
  2. Enable support access to your account:
  1. Go to the SmartThings Web (my.smartthings.com)
  2. Log in to your Samsung Account
  3. Select Menu (⋮) and choose Settings
  4. Toggle on Account Data Access
  5. Select the time period and confirm - In this step, please select “Until turned off”, once the team finishes, we’ll let you know so you can disable it again.

More info about this access here: https://support.smartthings.com/hc/en-us/articles/36170233944852-Enabling-Account-Data-Access-for-Support

12

Hi everyone,

The engineering team shared the following information after completing their analysis:
Some Zigbee devices may be able to rejoin the network by using the “Scan Nearby” option in the app. If that doesn’t work, the devices will need to be manually factory reset and then rejoined to the hub.

Regarding the issue, it was caused by a rare race condition where both hubs were acting as the primary hub at the same time. As a result, the hub that should have remained secondary made unintended changes to the Zigbee and Matter networks, which caused the devices to stop functioning properly.

This is a rare scenario, and a mitigation is already planned for an upcoming release.

13

just the primary hub, but I allowed impersonation to support.

14

Hi, @nuno7
Can you clarify what you mean by this statement, please?
Due to the issue, the devices must be rejoined, but in the case of Zigbee devices, there’s no need to delete the device from the SmartThings platform first which allows you to keep the automations and other configurations.

  • Some Zigbee devices may be able to rejoin the network by using the “Scan Nearby”
  • If not, they need a factory reset and then perform the Scan Nearby
    This should make the device get connected with the existing record of the device in the platform.

In the case of Matter devices, this approach doesn’t work, so you need to delete them and reconnect them.