If I had a nickel for every time an Isekai MC named Alan/Allen has proposed to his one love interest instead of having a harem, I'd have two nickles. Which isn't a lot, but it's weird that it happened twice.
Here's the thing. Anna doesn't know how they know about Allen. So she fears that if they know about him then maybe they have the ways of finding out what happend to him.