It's either that Reynold's a souless asshole who has no empathy for her life as an orphan before she got brought in, or that he's a dense fuck who doesn't know about the real world off his bubble.
Any basis for his hate for Penelope can be both of these, and it just makes him come off as toxic...