His mom was in a coma for 10+ years. His dad was a hunter that died in a dungeon.
"Hey mom. Rise and shine. I'm gonna go do a raid right now. Bye Bye."
I can understand why he is not going. Also he has some shadow soldier to keep an eye on them and in case can do a "shadow exchange" with them.