I think Gongja is feeling guilty from leaving his partner alone and pursue his own wish.
Man I'm feeling scared of this immersion thing, it's like you will lose yourself from existence.
And the thing is the original butler does have feelings for the
villainess, so as Gongja tries to pursue his love for her, his immersion will increase and ends up paying the price.
I smell a whole lotta red flags