Your supply may decrease a bit to match demand from your baby but as long as they are still feeding it shouldn’t fully dry up
I would definitely recommend the haakaa too, like Alexandra said. I used the Lansinoh version and it helped a lot. I had a similar experience and ended up with a clogged duct (not fun!) I would pump to just relieve yourself if you can. But don't pump for a full session, then you shouldn't dry up and you'll slowly meet the demand of your baby. I would also recommend trying to find a breastfeeding support group in your area. Some hospitals or birth centers will offer them for free. Hope this helps ☺️ and good luck!
My daughter is 9 weeks and has been sleeping 9 hours straight since she was 5 weeks old. I pump once at like 4:30 am and then just nurse her all day and my milk is still there and we have no problems! She also eats a lot during the day because she sleeps so much without eating. She nurses like every 1.5-2 hours which is quite draining but it works for us and she gets her sleep at night. You may find now that because your son is sleeping longer stretches he might eat a little more during the day which will help your supply too. It’s good he is getting longer stretches of sleep for the both of you!
Not the answer to your main question, but I highly recommend using a haakaa when you wake up in the middle of the night! They are quick and easy relief for the fullness. Just put it on whichever side you aren't feeding from while your son is feeding from the other 🤗