I don't think the guy necessarily is a bad guy, he's a product of the world and lack of any real interaction with women outside of a whorehouse. From the beginning chapter it was clear that he had to have done massive amounts of work to recuperate the fox girl to the point of even being able to walk let alone make, sort of, her own choices.
As shown here she went from being someone that would die if left alone or might even have been better off dead and now she's out their weeding the yard after looking I'm guessing happy? that he was concerned about her injuries the night before.
Yeah he's a brute and he ended up fingering her so hard while she was basically in a comatose state but stopped the instant he saw that something was wrong with her or the info he was given. So yeah, he's a scumbag but he's not a villain by the world's standards. Heck, even a slave here says he's the best a slave could ask for.
Again, not really defending it, but in the premise of the story he's an idiot that's learning.