Have you tried family counseling?
This sounds like a much deeper issue. Maybe therapy will help.
You should be taking your kids with you if they have such a big problem with it. You want to go out on a nice date with your husband? Tell the kids they're coming with you since it's such a problem that they have to tell their dad. You need to run to the store? You need to go out to get gas real quick? Pack them kids up every single time. It seems like that is the main problem that you're discussing here.
Honestly, I’m with Kendra on this one. I think getting some family therapy for you and your kids might help if that’s accessible for you but if you can’t do that, I’d send them over.
I would get family counciling. You say he was your abuser and seem like he is using the kids to keep tabs on you. Idk what kind of abuser he was but he could be manipluating them to get the information he wants. So family therapy with the kids may be beneficial. But IDK.
Let them go live with their father. I wouldn’t deal with it.