From what I remember of reading the WN (granted, I skimmed because a lot of it is empty-headed boob fetish shit, and that got boring real fast), he never gets to have proper sex. He doesn't get to initiate things; they shove their tits into his face until he's pacified, and that's it. One time, he's mind controlled by someone and turns into a mindless sex beast, and Nina straight up uses her tits to control him long enough to douse him in sleeping potion, and then says something along the lines of "no matter how strong you are, you will never win against my breasts." He's not their lover, he's their tool—a pet that they make do tricks by waving treats in front of its nose.
So the downsides are "these thots don't respect you at all, toy with you to keep you in line, and use you as their ride straight to the top." They're gold diggers who will use cult-like brainwashing tactics and even straight-up mind control to manipulate you into doing what they want. Granted, a lot of what they do works on Shin because he has almost no self-control over his libido and gets seduced ridiculously easily, but it doesn't change the fact that these women will do this shit without a hint of remorse if they can get away with it.
