Just because SHE was distraught doesn't mean HE was. He just wanted to go home. She kidnapped him, made him fight the demon lord put him through hell, his actual friends in that world all died, she could send him home any time she wanted, but wouldn't until he won, why wouldn't he hate her?