You have got to find the absurdity of it all funny at least and remember agawa pushed gotou away after the threesome he is just doing what she told him to do Which is not to try and monopolize her.
And to be fair, she was kinda right. Gotou had sex with the sister despite Agawa asking him not too, and then he confesses he is in love with her? Then multiple chapters later, he claims to be in love with everyone, but Shige calls him out on his bs. He just likes to have sex with women (same as Agawa with men), but he wants to pretend he does it because he is in love, when he is just horny (unlike Agawa who doesn't pretend she loves the guys).