I wonder if it's so common in Japan in reality that people would immediately assume so, instead of thinking she's just walking with some older cousin, a new step-father, or something, if they know it's not her (biological) father. I'd think not, and it's more like classmates talking shit or even a mild form of bullying.