If I understand your requirements correctly, you can use any handheld button device that works with SmartThings. You will get your Alexa integration through SmartThings. The button device will send the instructions to the smartthings hub, and the Hub will send the instruction to the bulbs. So you should have quite a few options to choose from.
See the following FAQ (this is a clickable link). Some of the devices are battery-operated, some are mains powered, so read the descriptions carefully.
If you have any questions about a particular device, ask those in the discussion thread for that device, not in the FAQ, which is intended to just be a one post per device listing of device models. Or come back here and ask your question here. 