But, there's a tragedy tag in this manga, and we still don't know why Uncle left the Isekai (or maybe I'm forgetting something). One sure thing is what was happened in the Isekai is too hard for him, to the point that he wiped his memory.
This is both the best and worst thing I've ever read. It's so well done, but also crushingly depressing. It's difficult to tell if the uncle is genuinely happy with his life or entirely broken.