Removing comfort bottle

My 15m old will only take a bottle of milk before naps and bedtime. Any other time of day he’s drinking water, juice, flavored water, etc from a sippy cup or a cup with a straw. If he wakes in the middle of the night, he is either thirsty or needs changed and will go right back to sleep with a bit of water or milk from a bottle. Him wanting his bottle is also a sign he’s ready to go take a nap. All of this makes me feel like it’s a comfort thing although sometimes he will get upset if I give him anything but milk in the bottle. How do I remove the bottle completely? Should I start by switching to only water at night then remove the bottle? Or do I switch to some kind of cup first? Do I trade the bottle out for a pacifier? (He doesn’t currently use one at all, hasn’t since he was very little)