Pretty sure the elf girl told them.How do THEY know he defeated the wyvern? They haven't seen him firing it. They only have seen the ice lance come down.
If he's going to continue doing good deeds, I don't think he'd have to worry as much. Still waiting for him to unlock elf girl's OP skills.But knowing your good deeds grant you SPs kind of gives you pause whenever you spend SPs frok now on right?