Personally I think this series is flawed but by no means "garbage". It has strong points but it's by no means "excellent". Props for the author for trying something different, with a homeless and older protag, and for not boring us down with excessive usage of game mechanics.
Regarding the age discussion: for all intents and purposes of the story he's a 25yo with world knowledge of a 56yo. And even if they were an actual romantic couple, once you're past your 17s age doesn't matter that much, you're simply adult. I've been in both sides of this sort of relationship, it isn't that different from a same age relationship.
@Christopher To Love Ru is fairly memorable and Momo is the best girl.
The biggest flaw of your argument is to smash quality (subjective) and popularity (objective) together.