Umm...Excuse me, head maid? If he comes to her room and pulls her out of the bed in the middle of the night, what else she is she supposed to do? He's the fucking Duke.
Speaking of which, being duke pretty much gives him licence to shoot his shot with just about anyone in his domain. If, in his mind, he's basically in love with/ pre-engaged to his maid, how is chasing her away from him going to make your life easier?