Except some of them are NOT sexually frustrated. Japan is actually pretty conservative-minded and traditional compared to most of the world so it seems like that to the Western countries. Fact of the matter is the guys embody traditional beliefs that women need to be mature and composed at all times.
And if we're really honest, reading a story about a guy who's into gals like regular dudes would be a one-shot in itself (or a very short and boring read). Same with reading about a guy who's indifferent with gals. Not much progress there. Stories where the protagonist evolves from being this rigid ball of whatever to getting together with a gal is much more interesting since we can actually see the evolution happen over time. Only reason people get annoyed is because they are veteran readers who pretty much recognize every cliche and trope written, making reading manga a bit disillusioned for them. (Never heard of readers new to reading manga being affected by this anyway)