Are you in the US or the UK?
I understand that you don’t want to use the smart switch cover, but they are the simplest method at this point. The switch is right where it always was, it works like it always did, but you also have the ability to control the color bulbs with automations. The difference that the switch does look different then a traditional switch.
There are two smart switch covers available in the US and they do look quite different from each other.
https://www.amazon.com/SYLVANIA-LIGHTIFY-Osram-Products-required/dp/B0196M620Y
https://www.amazon.com/GoControl-WA00Z-1-Z-Wave-Scene-Controller-Switch/dp/B01BKWG9XS/
Alternatively you can blank off the existing switch and use any of the other options in the buttons FAQ. There’s quite a variety there. 