It's an interesting take so far. Examining how Isekai heroes would react to being brought back to Earth is a spin we haven't seen too often before.
Other ways I could see it being done:
Our protag returns to Earth,
[ul]but time has passed as normal on Earth (so he's been missing for months)
but time HASN'T passed on Earth, so he has 5, 10, 50 years of memories that he shouldn't
but someone follows him (villain, love interest, etc)
but he only does so at the end of one of the volumes as a twist ending to a book
but it's not a one way trip, and now he has to deal with a portal to a hostile fantasy realm in his bathroom / closet / shed / etc
[/ul]