This is currently an issue with all edge drivers. 
Very annoying! It has been reported for over a year by Beta users, but the only official response is that they’re aware of it.
The unofficial workaround is to create a virtual device for each component and use routines to keep them in sync. Then access the virtual devices in the voice assistant and other thirdparty integrations. It’s a lot of work, but it does solve the problem.
There are several community-created edge drivers for virtual devices. You can find them on the quick browse lists in the community-created wiki. Look for the list of Virtual Devices.
Here is one of several discussion threads on the topic.
[ST Edge] Issues with multi-component devices and third-party integrations