Because it taps into idol and idol-adjacent culture. Idol like girls, whether in media or IRL, like in schools or at work, are seen as out of reach. Nobody actually wants to be the 'ugly old man', but if one can get in with the girl, then there's hope for everyone else. It also references the common Japanese power dynamic of older male holding all the power in society, so it makes it feel more plausible.
You know what's really yucky in comparison? American obsessions with 'teen' genre or even 'jailbait'. You might say jp has a loli-obsession problem too, but its mostly strictly confined to fiction, barring a JS/JC idol problem. Even then, in almost all cases, it's presented as something 'immoral'. The men in such stories are almost always scum.
But in the USA what I notice is similar genres always being presented as something that's not really morally wrong, and in fact it highlights a frame of mind that tries to rationalize or find loopholes to allow it. You have terms like 'jailbait', 'fresh out of school', 'first-timer', 'amateur teen', etc. It's unfortunate that modern jp industry especially that operating out of places like Fantia are starting to follow such trends, but it's always been there in the USA.