Thing is that he’s strong because of the sword giving him the boosts. She was already strong before she was ever given the clothes. This is one of the reasons why she has more stamina than him (plus he had already fought 50 of the empire’s best knights before even fighting him so that took a toll on his strength). If they both started the fight fresh then it’d have been different as they’d be at a stalemate as one has the ultimate weapon while the other has the ultimate shield.
Btw are you forgetting that he got taken by a surprise attack after he thought he had killed her already? So not only did he have a decrease in stamina already from the 50 knights he also got hit while his guard was down. That would cause huge amounts of damage to even the strongest of fighters.
Am I going to have to put you down as another who complains about stuff that has explanations for why it is the way it is?
You seem to be forgetting Blum, as a young child, was taken in by the Palm family and trained 24/7, including torturing him with magic, until he developed magic resistance. That resistance is not from the sword, it's because he was subjected to attack magic until it nearly killed him, healed, nearly killed, healed, ad infinitum. He was trained in martial arts and told that if he's lacking motivation, the Palm house master will go and slaughter his previous family, to motivate him. Viola, since she was a knight, is from a well-off family and likely trained as well from early age, but you can bet your arms and legs she trained at most half as much as Blum, who was basically a slave forced to train and fight, nothing else. He's the stronger fighter of those two, by far. Viola just got the main prize in the divine weapon lottery, while Blum got the second prize.
I suspect you bought Viola's psychological attack of telling Blum that he's just an orginary person, nothing more. That's the line those of blue blood told to commoners, to appear something extraordinary and above the common rabble, even in real life, in the past.