There needs to be a break off series of misfortune as that man tries to scam people in another world. Pretty much the inverted layout compared to normal isekai which has the hero going from town to town beating scammers, you instead have a scammer going from town to town trying to scam people, succeeding then everything falls apart and he has to go on the run. Sometimes losing everything and other times making off with the savings of a few people, which ultimately ends up being seized. He meets up with corrupt lords, thieves, scam artists with schemes to ruin the people, and when everything collapses his accomplices are caught red handed. I would suggest also having one of the people he has worked with is also able to escape and becomes a recurring character who does well on his own, but finds his house of cards collapsing every time he runs into the MC. I do suggest that once in a while there is a scam that works and either someone else double crosses him and tries to take the scam for themselves only for them fighting over it causing the scam to be revealed... I just realized why I have such a long post... I tend to ramble when I am tired and it is 3:46 AM... I need to get to sleep