They're the ones who have chosen this path. Because their egos wouldn't allow a single ronin to have slain the number #1 and number #2 of the Yoshioka clan, they pursued this blood-bathed, cowardly path and still failed miserably and now Ueda trying to guilt trip Otsu it's just utterly pathetic. If they cared so much for their "ill mother" or "wife and kid" then they would've forfeited pursuing this path. Musashi is not the sole culprit.