This shit is dumb. As an SS rank adventurer, he should've been able to detect his presence. Even so, he could also have just looked around the room before removing his mask. And on top of all that crap, he's been shown to have used memory manipulation magic beforehand. So what's stopping him now?