Everyone is calling him dumb for not realizing they're the same person, but he still notices when something is off just, in typical rom com manga fashion, he's got too much self doubt to let any conclusion stick. Though, to be fair, anything seems more reasonable than your ten years younger...