get some reading comprehension
well that is not exactly what that means, the translator probably just couldn't completely understand this part or just didn't want to bother explaining, or couldn't find a better way to translate it. what he is saying is basically that he can put up a front to girls that do not remind him of the girls from his childhood, but when girls act too much like those girls from his childhood, he gets like that. (it also mean in a way that the girls from his childhood 'liked him' but he interpreted it as them bullying him, which every girl that acts like them reminds him of them so he can't put up that mask.) And as we know he if very comfortable in front of the mc, that also means that he is not putting a front when he is with the mc, which probably means he doesn't know the mc is a girl.
it takes a bit of understanding of how the japanese language works, in order to understand translations that the translators don't understand or care to explain through tl/notes.
it also takes a bit of understanding of how the author or mangaka is thinking. Good authors have a reason for every detail in their work, and that detail is sometimes lost in translation or in the understanding of the reader. as i already (kinda) said one way my claim can be strengthened is by checking in the raws, if he ever uses language that indicates that he doesn't know the gender of the mc. like i have said the (-san -chan -kun etc.) in the end of the names indicate most of those details in cases of gender and titles, or what that person thinks of the person they are referring to.