Do you have zigbee repeaters in the rooms that these fans are located? It’s pretty much a must. The radios in the controllers are extremely weak. I have a fan 20 feet from my hub and it requires a repeater. Others have needed repeaters for fans that were mere feet from the hub. Lots of discussion about this in the main thread for the Hampton Bay controller.
As for hard wiring the wall remote. In the main thread for the Hampton bay device is a post where someone used a small power supply that fit within the switch box. He purchased the supply from Amazon. I don’t remember the specifics but would imagine you would have to solder some leads to the remote to attach to the supply. As long as the supply was in-wall rated, I would think it would be ok. DISCLAIMER: I’m not an electrician or knowledgeable on current building codes so I could be completely wrong about this.