All that violence and defeating other men should get the main character's blood pumping and he should naturally just take the girl in his arms and kiss her than have his way with her.
All this staring into each other's eyes when there are at least 4 times that they should have gotten closer physically is so annoying but I know it's the manga way of storytelling.
Also, what was the fisherman guy's crime? he seems a bit too normal for an island of murderers and sex offenders.