Do you think it could help your relationship by having him start to work a bit? Maybe you could only work 1 job and he work 1 job on different schedules so you can alternate taking care of your baby? If you're sure you're ready to leave, then definitely follow your heart. But if you're looking for ways to make it work, you may have a better relationship if you're not having the full financial burden
Best to leave, been through it myself, ie a single mum and things in general are so much better now, not necessarily easier, but better. I hope you find the right choice for you! Here if you want a chat