The fact that she doesn't reject him right there, she either returns to the Main house, or leave the Annex for disobeying him. If she refuses, she has nowhere to go. Honestly she should leave, but it's understandable that she would see both as bad options.