I would just explain exactly that! It’s not the mum ‘work’ part that’s hard, it’s more the isolation etc part of it all. I recommend seeing if there are any playgroups around, or if you make any mum friends on the app - make the effort to see them. It gets lonely but it’s also nice when you make friends and get out there!
My partner used to say the same/similar things. He had 4 weeks off (2 before birth and 2 after) when I had our second baby with our first being 23 months. For those 4 weeks he got to be the default parent and the comments quickly stopped. I suppose from the outside looking in it does seem ‘easy’, but once they actually experience it it’s a hell of a lot different!
Hey fellow mama, following cause I'm in the exact same position. My husband doesn't seem to understand how emotionally and mentally draining it can be. Being a stay at home mum can just get really repetitive and boring and can really bring your mood down. It definitely doesn't help when they make comments that it's easy to be a stay at home mum. My husband made similar comments which really brought me down. Sending my love your way and hopefully you get some good advice. I would potentially recommend couples counseling but I know that's not an option for everyone, especially living off one income