It's admittedly pretty silly to be 'in love' with an idol, especially in the context of modern Japan, where they're usually contractually forbidden from having a boyfriend and actively encourage their fans to 'fall in love with them' or whatever. Then again, he's actually met Nanoka in person - and she was naked, and her body was pressed up against him in a hot spring, and she helped him and didn't seem to mind that she was naked with him.
And I don't think it's a bad thing at all for him to reject a girl for whom he doesn't have feelings. He was straightforward about it and gave her a clear no - that's not a bad trait to have -- although I think it would have been reasonable to at least take her on a few dates to see if those feelings developed.