I sorta wonder if they lock the twins up when there's no tutor.
The servants didn't stop them from approaching their mom, and we've never seen her be nice to them, but they clearly didn't expect her to hit them.
Makes me think they don't have free reign of the house when there isn't a tutor.