BD being hard work

My LOs father has always been extremely difficult. Everything is his way or no way. I have concerns regarding safeguarding which I have been told to monitor and record myself. There’s always been police reports because of abuse towards me. He picks and chooses when he sees her all the time. Expects me to just drop everything straight away and makes me out to be hard work if I can’t make it happen exactly how he wants. He has her very limited time already, which is something that I feel is in our daughter’s best interest for many reasons. Because I have asked to spend Mother’s Day with her, he’s now got a bee in his bonnet and is making my life hard. This week, he is demanding her on a different day and to literally have her from first thing in the morning till bedtime. Normally he would have her for 2 hours during the week. I’ve told him that I can’t make this work. He doesn’t have a car so relies on me to drop her off and pick her up. I’ve said that I will drop her off on my lunch break at work (she’s with my mum whilst I work) and pick her up when I finish. This isn’t good enough for him. Even though it will work out 3+ more hours than he usually sees her for. He wants me to drop her off before I start work. 1. I’m not comfortable with her being with him for the whole day. And 2. I literally don’t have the time to get her there before I start. He’s now saying that he’s going to find a way to pick her up. So that he can have her before I start. What do I do?!😫 all I’m getting told is that he is “kind” to me so why have I got to be hard work.