I could see a working concept here, but with some major changes regarding the character...
After the training period, he should've gained a zen-like confidence/calmness/mentality, which would be the cause of his mind growing to it's "fullest potential". Also, I would've made the MC to meditate during his time in the timeless zone.
Secondly. Change this bullshit about not killing people. He should've sliced that guy in half, regarding his skills and time put into training. Just straight out make him take part in a tournament where killing is okay, and rather common even.. (And make him kill people in duels)
Just mainly, lose the whole bullshit of him constantly doubting his abilities, and over-estimating his opponents skills.. The guy is a fucking grandmaster by now. By the logic of everything, it's very, very unlikely that there exists more skillful swordman than himself.
All things considered, this manga is a wasted potential in my opinion. The main character just ruins this story.
The running joke, of him being surprised of his own strength, while in every other moment he's just an insecure monster... -was NOT funny even on the first time.
This unfunny, irritating trope reminds very much, of this running seasons anime (2021-winter):
"Suppose a Kid from the Last Dungeon Boonies moved to a starter town? "
I hate the main character in that story (Lloyd) for similar reasons (while other things considered, I think it's decent anime, with other likeable characters.)