Understandable. He was responsible for accounting and all of the paperwork, so he had to know about the prices and wages on the market, yet this absolute fucking fool still remained there? Maybe he's simply a masochist.
As for the previous party's situation - womp womp.
I... I'm actually not sure they were exploiting him consciously (aside from the low pay). The thoughts make it seems like they were genuinely unaware. If 'grantor' is pretty looked down on, combined with this constantly self-degrading speech and attitude they probably truly believed what he was saying about not doing much.