@Halesy
Yeah well that's how he is from the start. He sleeps around, as expected of a yakuza.
And if you read the side story, Yoshino knows it too. She even gave him presents to give to his side women lol.
If you want the same type of humor with a more decent ML, I recommend you read its sister manga from the same author.. Haru no Noroi.