I can accept that he lost his powers as he returned, sure, but did he also lose all that fighting experience? Surely he knows how to throw a punch and duck and weave?
It's one thing when one's fighting skills are honed by experience the hard way, it's another when that experience is carried
by the accelerated-growth type cheat skill that he lost as he returned, I guess. Him slowly losing his memories of his time as a hero probably had something to do with it.
edit:
Nevermind, I checked the raws, his cheat skill, 植物急成 shokubutsu kyuusei, presumably 植物急成長 (Shokubutsu Kyuu Seichou, lit. plant rapid growth) because his head partially blocks the screen, would just allow him to grow plants really fast (getting a seed of a world tree suddenly makes sense). Either way, it's plant-related, he can either grow plants from out of nowhere, or grow them incredibly fast.
He also does not have top-class weapon called "Reality", but just stated that he had a weapon of the highest
rarity. リアリティ(reality) vs レアリティ(rarity), yeah I can see why the translator might get confused.
Still, if his actual skill was just "growing plants really fast" and got carried by his weapon, I can see why he could lose to some thug. Bro needs a guy with an eye-patch that talks with a Kansai accent to train him spontaneously as he walks all throughout Ka
murobukichou.