The reason in game why he’s not ruler of his home country is because he’s initially framed by his younger brothers to make him lose his right to the throne. They basically set up a scenario where he gets drunk, gets found in a scandalous situation in some town bar or something and basically loses his right to the throne. And there were previous rumors of him being a womanizer spread by his younger brothers. So it was technically the final straw at that point.
He gets engaged to game Pride shortly after. How she broke him mentally at first is using the same people who were involved with setting up his downfall, mainly commoners who got paid to do it. Pride basically tells him that she captured the ones responsible and she basically tells him that he needs to execute them. She basically hands him a sword to do it. The commoners are begging for their lives. Pride makes a comment that if he doesn’t do it, she’ll basically destroy his kingdom out of fake love for him. He ends up killing each commoner one by one and that breaks him as he loves his citizens.
Shortly after he meets game Tiara and romance blooms but he’s still mentally broken from earlier. Pride gets fake jealous saying things like she’ll forgive cheating as long as he stays with her. He later finds out his home country is in shambles from his brothers who are now ruling. They beg him to return to fix things, however Pride knew about this and basically gave only two decisions he could make. He could leave her to go back but she would declare war on his home country and destroy it because she would feel angry out of fake love. Or he stays with her and she basically torments him constantly about how his home country is slowly going to ruin and his citizens dying of starvation. This breaks him further as either decision would mean his country gets destroyed one way or another. And he now knows the citizens are all suffering from the heavy taxes that are providing the luxuries that his brothers are enjoying.
Game Pride outdoes the other villainesses we usually see in terms of the evil they do imo. She’s willing to do whatever she wants. Like one scenario she offered “help” but she actually invaded a country simple cause she wanted their jewel mines. In her country, she either executed special power users who resisted her or enslaved them to use their special powers for herself.
There is an explanation why Game Pride is that bad, however currently with the light novel we haven’t reached the answer for that. The web novel does explain it, though whether or not it’ll be in the light novel, we’ll have to wait a volume or three depending.