Samura launched his attack only after Hirohiko used banquet, and got affected by the illusion before he could cut him down fully.
I suppose the knowledge could be explained away with the blades and it's wielders being publically known - they are war heroes, after all, and the Hishaku might have...