Absolutely have every right to be upset! He lied about how bad their relationship was when clearly he still had feelings for her.. is reassuring her that you will never be his wife.. tried to make you out to be the bad person in this situation instead of admitting he was wrong and apologizing. Since he sees nothing wrong with it, it’s definitely going to continue. You won’t ever be able to find the conversations again because now he will get better at hiding them and even if he is careless enough to let you find them again, you choosing to stay with him after knowing about him doing it is permissive because you’re accepting it. I’m glad you followed your instinct and checked his phone but that leaves you in a position to do something about what was revealed to you. You wouldn’t have seen it if it wasn’t meant for something to change.
Of course there are feelings, on both sides. Also the fact that he’s gaslit you and tried to turn the situation around on you is also very telling. No happily married woman would entertain conversations with her ex unless she felt that chapter of her life wasn’t closed. How much do you want to bet her husband doesn’t know she’s having cosy convos with her ex? 🙃 How does it make you feel knowing that you are just a placeholder? That he will never marry you? That you’re just being used for free sex, a clean home and childminder? Because I would have lost my shit and kicked him out! You don’t need to get past anything or forget about it. You are justifiable in being upset, any normal person would be!