Hmm.
I must admit, I myself do lean toward removing the Duke with extreme prejudice, albeit swiftly and without suffering.
Rabid dogs should not be allowed to live, after all.
His position and influence are due solely to his birth, he has done nothing himself to justify it. He abuses his position to the extent possible. He is a pimple on the backside of the nobility, and should be popped. He has no personal honor. So long as he retains his position and influence, he will be a threat.
The MC is correct that he will now need to watch out for underhanded actions instigated by the Duke.
Too bad he can't announce that any untoward occurrences involving his household will be presumed to be instigated by the Duke, and retribution exacted. Most legal systems don't allow such to be done.