Surely is incompetent. Forgot his main skill.
He can also find so many ways to use his skill in combat even though it takes one second to activate. With great timing and excellent prediction he can surely be OP.
He can also get(should've definitely got) supporting combat skills i.e. swordsman, this would enhance/polish his overall combat style, leaving more time for him to develop his style of combat instead of doing it all came consciously.
But mangaka does what mangaka wants.