No, that's not true at all. A lot of romance works have the theme of slaves within a brutal prejudiced society but most have happy endings (heck even most isekais nowadays have this trope). NTR in the first place isn't a very common trope/genre.
People being mad is to be expected cause...