There are ways to use wall switches with smart bulbs. You just have to make sure that the switch is not actually cutting current to the bulb. That then solves the problem you describe, because the bulb will always respond to a command from the app as well, you won’t have to turn on the switch to be able to turn on the bulb. But whenever you use the switch, the light will respond to that as well. It works fine, you just have to select the correct devices. 
See the following FAQ (this is a clickable link) it should give you some ideas.