@dramou2010 yes, its retarded, but don't forget the last time he died. He totally forgot, not just about his life before but the fact, that he has the reincarnation spell. In that case, yes, he can remember (right now) to his past life, but can slowly and gradually forget about it.
Since its stated, that he could forget, I can forget to the author too, which is really rare, considering in other/most isekai(like) manga the MC just act as a child, and other than they can add the "isekai" tag, it doesn't really changes the story from a normal fantasy.