It’s not called direct association in Zigbee, it’s just called binding, or sometimes endpoint binding, and it’s possible only if the switch was designed for that purpose. You’ll see some people doing this, for example, with some of the handheld remotes like the Phillips hue dimmer switch or the Ecosmart remote. It’s most often implemented for zigbee 3.0 devices, since they don’t require a hub at all when not used with smartthings.
What’s the brand and model of the switch?
Also tagging @johnconstantelo in case he has any more to add.
And here’s a very very old thread back from 2015 which talks more about it in a smartthings context, but again, very old.