As long as you’re going to use a smart bulb outside anyway, why not use a battery powered device for the switch? Then you don’t have to worry about the wiring.
You can get one that fits over the existing switch, or goes next to it, or really goes anywhere at all you want.
The switch will talk to the hub and the hub will talk to the bulb and everything should work fine.
See the FAQ. Some of these are mains powered, some of them are battery powered, some are wall-mounted, some are handheld, so read the product description carefully to make sure it’s a match to what you want. This is a clickable link.