Kinda sad that the only other comments here are defending Kazuya. Dude's exploiting people's desperation for his own amusement, it doesn't matter if they chose to bet, what he's doing is despicable regardless.
He has enough money that he could pay for all their debts and it wouldn't even make a dent on his wallet, yet he chooses to torture and humiliate them, abusing the fact that they are desperate enough that they'll accept anything for even the slightest chance of clearing their debts.
Of course the people who play his game are making a stupid decision, but when you're in such a dire situation it's hard to think clearly, and even then, Kazuya is still wrong for toying with their lives like that.
If his lame ass speech is all it takes to make you throw away your morals you're no better than Myoshi and Maeda.