Okay, he literally had one of those generic “genius prodigy at fantasy high school” harem experiences? Yeah, I’d be pretty pissed if that type of person said they were a “loner” too. Also, I don’t know if I’m happy or sad we skipped over that (okay, I'm happy. This is better than that generic harem story).
And it’s a little late, but going through the comments there worry of this becoming a harem manga. Don’t worry, the chapter makes it pretty obvious it was just used as a joke on that generic setting. I don’t expect any of these girls to show up again any time soon (could be wrong though).