@scarlet_summer his father was a soldier from the northen tribe. Since he was powerful he survived others' discrimination but miru couldn't, he came to their village for whatever reason and impregnated her mother which the villagers must have felt was scandalous (for some reason unknown to us yet) so they were chased out. Then coincidentally or not he's father was either killed or died on the first day he spoke something, after going through so much and from starvation, her mother was barely holding onto her sanity and would suffer paranoia too. That just made people believe that miru was cursed even further and hence the discrimination deepened. (Part of the reason people believed that miru was cursed is that they wanted to ease the guilt and justify their acts of discrimination against their family. Back then, being expelled from a village was a grave punishment and hence the people from the new village neither have accepted the new family nor have helped them survive somehow. The human hypocrisy was running deep in them and so they must have wanted to find an absolute reason to avoid the family and thus came the curse)
The discrimination he suffered in other places was just people's fear of the unknown.