The way he responded to that question suggests that he is well aware of how people perceive him and his mother's relationship as "unusual".
Lets test it out this way, no one would say anything if a mother is going shopping with an 8 year old and picking his clothes. However, when the child gets to a certain age, then people start having thoughts like "so nice, the kid is spending time with his mother" or "he still does that, doesn't he want to spend time with his friends?" I say certain age as it is relative to the person making the judgment.