IIRC, because the Duke is a heartless, selfish piece of shit that cares only for himself, that's why. He didn't even bother to protect or clear Keira's name and throw her into the angry mob in the previous timeline because "the prophecy only said one spirit user to protect the world, but now we have two, so it won't be a problem if I throw Keira to appease the angry mob and retain the Dukedom's honor and dignity". To make matter worse, he agreed to cast the time magic because he was on the verge of dying and his dukedom is destroyed along with the kingdom, not because he feel regret for killing Keira.