well from what we've seen i don't think he's missing a pinky and his adopted father told him to be kind so i don't think we'd see him get his hands too dirty or so (who knows maybe he got this job at his age due to him to begin with)
He doesn't have a son. He was the guy in charge of paying the old woman who ran the yakuza "childcare center", and at the time he was close to Tenkichi, the son of a member of his group.
It's entirely possible deadbeat dad will just be dead before the end of this backstory arc. He ran off with his woman, abandoning his group. They don't forgive you for that kind of behavior.