To think, this whole manga could have been cut short if he told the old man, "Oh, I didn't notice you also had the same keychain!" thereby making it clear to her that he wasn't talking to the old man without coming across as rude. Then follow it up with something like, "it's so crazy that we all have the same keychain," to her, then answering the old man's questions in a respectful manner.