I don't know why Naofumi doesn't get mad when all sorts of guys hit on her. I ain't talking jealous, but you'd think he'd not want to risk someone turning her against him too as I don't think this is when the queen took over yet (???)
Hitting on her once is one thing, actively trying to flirt with her enough to annoy either of them or disturb his plans is another and seriously harassing her to get her is a third.