@Zeroxjac the yakuza don't bother most people, and the Tokyo ones primarily deal in property meaning they want neighbourhoods they own property in to be as safe and profitable as possible, and their ways to acquire property at low prices are not always "clean" however they can't be obvious enough that they scare of prospective buyers.
So overall you will probably not encounter Yakuza ever in their professional sense unless you're either a property investor or use illegal or shady things yourself making them a non-issue for a huge chunk of the populus.
Visible, yes a problem, usually no.
Can only speak for tokyo however have no idea what they are up to in other parts of japan.