I get why people are already mad about the slavery aspect but, and I hate to say in defense of it, he knows what he is doing is wrong and straight up illegal. But by hiring (of course) female only slaves of ogres, goblins, and abandoned elves that would likely be turned into target practice or worse, he's, at least what we've been shown so far, is not only protecting them as well as properly feeding and caring for them, but keeping most of their existence a complete secret.
Probably the only one that usually goes out with him is that one elf. Yeah, it's still bad and will probably get worse, but it's the nature of nearly all fantasy worlds. Even non-Japanese fantasy novels or games contain slavery of some kind.