Nipple dummy confusion

I gave our EBF baby a dummy a few days ago, only using it to settle him for his naps, or soothe when he was upset in the car, so he didn’t have if much at all, however I’ve now noticed he’s started to not latch properly and suck on my nipple with the same depth he would a dummy. I’m going to stop using the dummy in hopes he forgets and starts latching correctly. Has anyone else had this and managed to correct it?? Thanks!