They are probably mistaken about it or simply lack information, it's likely that garland does not explain the limitation to his power to everyone, there will be more explanations later.
Garland's goal of destroying all the towers except for his leads us to believe that he might know more than...