Unfortunately, no. When you cut power, which is what you were doing when you use the manual switch, the smart bulbs do not have a chance to send their status to the hub so the hub thinks they are still on. There’s nothing you can do about that with the setup as you have it now.
Note also that your individual bulbs do not have a status of “off,“ they have a status of “off-line“ because the hub cannot reach the radio. This can cause other problems on your network.
So when you cut power with the manual switch, for a brief time the hub will think the lights are still on when they are actually off. Then when they don’t complete their next check in, the hub will mark them as off-line, but again, not actually off. The status update is lost because of the power cut.
In addition, you should be aware that most smart bulbs, with the exception of Sengled, are designed to always be on power so that the radio inside the bulb will always be able to hear the next network command. it’s OK if there’s an occasional power outage, maybe two or three times a year, but if you are regularly cutting the power and then restoring it with a regular switch each time the power comes back on there is a surge of “inrush current“ which overtime will likely damage the radio inside the bulb, making what are already expensive bulbs that much more expensive as they will have a shorter lifespan.
So obviously it’s up to you, but best practices are to replace the dumb switch with a switch that will not cut the power to the bulbs, but instead send a radio message to either the hub or the bulb itself so that the bulb will turn itself off. In this way there’s no inrush current damage and the status in the app will stay in sync.
If you’d like suggestions on these kinds of devices, see the community FAQ on this topic. Although it says “hue“ in the topic title, it applies to other brands of smart bulbs as well.