The way that I see it, he didn't really "abandon" her, more like he never truly saw his worth outside of the "warrior" and "vice-captain" roles. I'm just going off speculation, but maybe at some point he thought he'd be a hindrance to his own sister, since he doesn't have to look after her...