The way I understand it, it might have more to do with his presence period. Him and his mom had been jumping around people houses and these other people were doing a favor for them. They didn't want to see a mom and kid on the street because they would feel bad, but they aren't necessarily happy with them there either. It's an appearance thing. If he shows himself it reminds them of this situation they aren't pleased by, even if he's behaving well. Behaving well could've made them feel guilty for not liking him being there, and who likes feeling guilty? By doing nothing, essentially being invisible, he allowed the host families to pretend that everything is normal.
That's just how I read the situation.