The density of the MC kinda makes sense, he has a very different perspectivce of his younger brother than we do.
He's still kinda coming to terms with what type of person his brother became and the significance he had in the tower. He can see other people through his brothers perspective through the pocketwatch, but the irony is that he never sees his brother.
It's one thing to recognise a figure from your brothers memory but another to think it's your brother was that important to the other person.