If he kills the person directly poisoning the prince, wouldn't they just get someone else to do it? You'll go through bodies and still have a problem on your hands.
It doesn't mean that they won't get punished, but it makes no sense to just start killing folks unless they're a "true believer."