@Riger He wants the sword out of his hand, but doesn't want to give Musashi an opening by pulling it out himself. So he's "giving" it back to him, while being able to keep a hand free for offense/defense.
You gotta love the big oaf. Says "Ouch" when kicked by a female waaaaay smaller than he is, yet stands there with a sword stuck into his hand, and only give it back because he can't do anything with it.