I think I'd side with the dad in this, but only by a little bit. Yes, he is neglectful towards his son, but I don't subscribe to the idea that parents must sacrifice their entire lives for their children. That might apply to their early formative years, but, as he said, Hitoyoshi is old enough now to not need to be coddled for everything. Let the parents take care of themselves.