I think the ML had a past relationship with a man who wasn't gay and got dumped in the end for being a man. So that traumatized him into deciding to not date straight or bi men.
So, I'm glad Nogi was able to convey his feelings, but I think the fact Tsumu is literally in the same room as them is a good enough reason to not have sex at this particular moment...