Bf with nipple shields

Lately my LO (11 weeks) has gotten really frustrated when breastfeeding. We use nipple shields at every feed since the beginning due to latching issues but lately I’ve noticed that the shield is getting more and more in the way. He would jump on and off it and knock it off each time. He then gets frustrated and as do I. Really keen to wean him off the nipple shield but he cries and gets frustrated each time I try without the shield but then with the shield he just knocks it off. Only position where it doesn’t get knocked off so easily is if I literally dangle my breasts into his mouth. Anyone else had this issue and have it resolved? Really starting to get to me and I also worry about his milk intake so never know how much to top him up.