I don't know who's dumber: the mangaka who write love triangles like this one or the people who lust after shorthairs like the one in this manga.
The author, in the previous chapter, just had the MC tell the shorthair that she looks like a man--which she does: she could easily pass as male, and there tons of effeminate/soft male characters out there that look a lot like her (minus obvious traits) to prove it. Later that day (this chapter), all it took was the shorthair walking in on him wanking and then flicking the bean in front of him to get him to forget all of that--including what he's been doing with Futaba. (And despite the events of this chapter, he just couldn't figure out that Futaba was on top of him orgasming--somehow. She was just unwell, that's all. Never mind how the shorthair started masturbating in front of him and he got the picture immediately.)
Then there are the people lusting after the shorthair. Compare her to Futaba, and you can see why the author--through the MC--says that she looks like a man. The author explicitly says this while repeatedly presenting this masculinized girl's body to the audience--as if she's somehow comparable to a girl that actually looks like one, an implicit comparison also by the author's design--and there are still people eating the sights up. The author knows what he's selling and what's going on; the only people completely in the dark, blinded by their twisted libidos, are the ones lusting after mannish girls.
To top it all off: "Why am I starting to panic?"