Unfortunately this is an old issue. I’ve put in support requests to have it fixed and posted about it in several threads. Here are some of them. Basically you have to clear the incident for the valve to close again. you don’t need to remove the smart app.
New leaks should close the valve even if you didn’t clear the incident.