Some switches do not report when they are toggled physically and in other cases if you polled the switch and it sent back a result then that looked like it was toggled physically, and in still other cases a repeater might end up forwarding the toggled message in such a way that it looked like a network command. Plus the issue of 3 way switches where physically toggling the accessory switch results in a network command to the master switch.
So you put all that together and it just ended up being something that couldn’t be relied on.