The reason of his outburst is because Pansy hits all the spots, can see through him and read him like a book, and more importantly, he can't even make a proper comeback at her which further proves she is right, that's what makes him ashamed of himself. He's got no ground to stand and he can't accept it.
This story must have the worst girls ever. Pansy is basically telling him to be your generic beta selfless goodie two shoes instead of a self-indulgent hypocrite. Honestly, every girl (except sansanqua) in this story is a fking hypocrite and acts like they are better than Joro, telling Joro to be a good person when they are trash themselves. Really makes me want to stretch my hand into another realm to strangle them.
He's lying when he told Pansy to leave. He wanted her to stay, but felt that he was holding her back with her loving him.
I think that Pansy is pretty bad at telling her feelings with him. I think she meant that he shouldn't do thing as a way to compare to others. Because of the old saying that there's always someone better than you. Because the note she left shows she wants him to be happy and that he was making himself unhappy by forcing himself to repay her debt and working so much.