Those guys are probably paid to cause trouble for that in so it goes bankrupt because some idiot noble or other wealthy merchant wants that place for his own business or so.
That is the "adult affairs" those guys mean.
Though the MC will probably beat up the ones pulling the strings with the reasoning that they tried to involve his girls for interfering in their plans.
Truly, a face that screams: "I am a huge douchebag and I deserve to be shot in the head. In any other manga, I'd be a cannon fodder for the protagonist to destroy."
Wow.