Using Google Home (assistant) speaker for notifications/alarms?

I got this working but I have long delays before any sound/text to speech is played. About 6-10 seconds before I get any sound using Big Talker.

What kind of delays are you experiencing with this?