No, it's really that simple. Haruki is getting the "better things", and "taking the role" he sees as his. He was a quiet guy in the real world, likely nerdy. So, in his mind with the "Hero" class, he's the protagonist, so he should be the one saving the day, getting the best girls, and being the center of attention.
His own inner thoughts don't even set it as feeling inferior. He sees it as Haruki stealing the place that is rightfully his as the "protagonist" (hero). That's why his thoughts were "This was my mission!" "Why are you taking my credit!" and "You brought this on yourself!". It's delusion
But Haruki doesn't want the glory, nor does he advertise his exploits. Sure he will report his kill quota to avoid punishment, but it's not like he boasts about it either. He just goes where he is told, protects his woman, kills mobs, and fucks his wife every chance they get.
And it's not like the Hero would be jealous of him having a wife as he should have been assigned one or more of his classmates due to his status. They were all paired up and fed aphrodisiacs for weeks. Sure he might feel jilted if she was his love interest in the before, but there is no indication of such.
His main issue is that he was treated as special due to his status and now feels threatened because some low key nobody kept saving everyones ass, then one shot the monster boss when he wasn't able to touch it. I'm sure Haruki having the stamina to energeticly bang his wife right after did rub salt in the wound, but that feels like it would be just another hit to his obviously fragile ego.
tl:dr
While seeing Haruki bang his wife on the battlefield might have been the straw that broke the Hero's back, the decisive blows to his ego were his lower kill count, being saved, and Haruki one shotting a boss he could barely fight.