Would you recommend going into business with family?

Not spouse like wider family like siblings or cousins? I have had a business idea for years!! I am ready to start working on it and my brother suggested we come together as a family and start it. As in all of us siblings. I’m hesitant I like to keep things separate and this business is very important to me and could be what my family will inherit (my children) I want to do this for my nuclear family and don’t really want any interference from my siblings. Am I being stupid to reject their input and help? Or is this a wise move?