Hey David!
Do you mind providing a task URL with an example so I can take a look at what you mean here?
If I think I understand what you mean, there is no way -- However, if you want to prevent it from the get-go, in your 'Email Received' trigger, you can set your follow up behaviour to 'Ignore'