Still a stupid move, and his mom should have stopped him.
He is officially engaged to Rosemary, that's a contract between their families.
Even if the society look down on Rosemary and doesn't openly talk about it, that doesn't make his action justifiable.
If the opposition want to nitpick on him, this will just give them more ammunition to attack his reputation.
There's also the risk of Serena exposing those letters. She's known to have bad relationship with Rosemary, so they wouldn't think she's beyond doing that for petty revenge against her sister.