That is the problem. Because they know that he is an evil bastard, they allows he do what he wants since they are Potimas, too.
...except that elf girl.
There are various types of Elves. As far as I can recall, he doesn't really have clones in the conventional sense.
The elves are more like bioengineered chimeras. Many of them are captured humans who he genetically modifies to be closer to him and gain elf traits. With the first generation elves he also implanted bits of his soul. As the elves reproduced, these bits of soul gets passed down but with enough generations it dilutes. Only the first generation are really aware of all of Potimas's schemes and intentions, and they're little more than puppets, so they seem like clones. Later generations interbreed with elves he creates from more humans, or just with humans to make half-bloods.
The more diluted the lineage, the less close the elves are to him. Potimas allows this because to maintain the elf population he knows he needs to maintain a degree of genetic diversity. It doesn't matter to him that they aren't 'him'. As long as he can manipulate the entire elf race, what's the real difference? Anyway, the pureblooded 'clones' are a small minority. Most elves are in the dark.