My god, I just realized why she doesn't recognize the guy without his mask! It's because she avoids looking directly at his face so that he can't get a good look at her face because she's trying to hide her identity! That's why the guards were able to recognize him even without the mask (after getting a close look ) without him having to say who he was but she hasn't!
Edit: Heck, if I remember correctly, he thinks that she does recognize him! My memory is a bit fuzzy but I recall him at one point being confused why she was being so distant with him after they talked casually back in the garden, so while it's still kinda weird the dude wears a mask all the time, it would mean that this story isn't trying to make us think that his mask is some kind of master disguise.