At first I thought its was Lillian's mom, but on looking at it again, considering the fantasy started from looking at where Lillian was lying, I think he's actually fantasizing about her. I think he's longing for her to call him by his real name and not as a brother. That also explains why the head maid is rather grumpy. She knows who he really is and that he's pretending so she's displeased about the situation as it probably all seems inappropriate to her. Considering societal hierarchy, she probably doesn't like the idea of a girl of lower birth being in a close relationship with her master especially when they aren't actually related.