I don't mind a MC who is a little dumb or plots that are a little contrived, but this is... too much for me, I think. The dude just happens to be bragging in public where the MC is? Why did the MC's older brother have to be the one to marry Seraphina? Why did the protagonist only half-ass support Seraphina? Supposedly many years have passed, but it doesn't feel like he has any real goals or even an idea of what he wants, so what's the point in him going back in time?
That's the point though - he decided to save his brother, nothing more. To that end, he pitied Seraphina and helped her out and kept his distance - went to war to save his brother (or so he thought) but in the end it was pointless he found, as Shou still died, and he gave up once more on his life and went back to philandering and drinking, just like before. His entire goal was to save his brother, having been given a second chance.
However, in regard to the bragging - yeah I mean it's a drunk guy in a public bar for the plot - of course it could have been he frequents the bar and since the wedding was that day he was talking up himself for once, but it's just a plot device, whatever. He's gotta find out somehow that it wasn't just random violence or "fate" to motivate him once more - because Seraphina isn't enough alone we've seen.