If I understood the situation properly, then I don't understand why they couldn't just take over the guy's territory while he and his army were holed up in the fortress. It sounds like a singularly dumb strategy to me. He's leaving all his productive land unguarded while he hides.