@Miserys_end
I think they are saying that he needs to speed up getting his preferred Mistress & Children officially married and recognized, because once Prince Asshole becomes King Creepy Asshole, he'll take advantage of his position to 'apologize' for the previous situation by sending a woman of his choice to marry into the family again (probably as part of his Creeper Face Plot to take advantage of Ellen existing, somehow. Most likely an attempt to break the curse on the Royal Family for power or something. or just to screw with them. Either or).
Before, it was more of a 'Bro, when you're ready, go marry your mistress and bring your kids to the household. I'll accept them as your elder brother and the Hero and technical head of the household at the moment' sort of thing, I think.
@nexev
Eh, we don't have the full background on the Prince, but according to Origin, he's been an enemy of Rovel's for some time. He hasn't done anything massively overt yet, but he's starting subtle.
In this case, he used his extremely unpleasant younger sister as a way to force his father to abdicate and got the pair of them exiled so he can seize the throne, while also using her to basically blackmail Alberto, the Vankreift household's chamberlain with a promise to 'fix the situation' and never followed through (and then put him into a completely untenable situation just for kicks, where he'd be harshly punished for either disobeying a royal command or betraying his master), interfered with the divorce trial by framing his father into giving false witness (which is a major offense for the king in this country, apparently) and using the Royal Seal to claim that his little sister's daughter was Rovel's, and he just went full Creeper Face at planning to 'make Ellen his' somehow. Also, he suggested engaging Ellen to his son, which made Over Protective Papa Rovel twitch, which is an admittedly more minor offense.
Origin's flashback/explanation also implies that the previous generations weren't particularly great before Prince Creeper showed up, either. Rovel-as-a-little-kid thought it was unsurprising that the spirits dislike the Royal Family, so it seems that Agielle and Prince Creeper haven't changed a lot since they were younger.
Between that and Origin and Rovel revealing that the Royal Family is actually technically an enemy of the spirit realm, due to the curse caused ancestor who was Distinctly Like Their Descendant Agielle trying to seize Origin and use her against their will, resulting in them being cursed to be completely unable to use magic, as well as the Royal Family desiring to find a way to force Rovel to stay and lend his magical power to their kingdom (which is more reasonable, at least, since without it their neighbors will probably invade, but it's not making Rovel happy), the Prince has pretty much taken an antagonistic course to Ellen. Also explains why one of his requests as a reward for his services was 'the Royal family isn't allowed to mess around with my family'.
Also, he just seems to enjoy messing with Rovel, since he's really enjoying that Ellen's like Past Rovel.
Does come off as a little rushed and overly overt way of setting him up as The Current Villian, admittedly. Better pacing wouldn't have revealed him as being behind some of the troubles for a bit longer. More cut outs or patsies, another high ranking bureaucrat or noble, that sort of thing.
That's my take on things, anyways.