It was putting some distance, not escaping her completely. There's a big difference between seeing someone a few hours every few days and living with them under the same roof.
She's his family and he loves her, so naturally she's free to visit him when she needs him (or simply wants to see...