@Robbini and Potato I think that's the point, as a Jester he was dressed as a parody of a stage magician, though i think he would need a cape or mask, to complete the look if he had the tophat. His current look does look more like what you would expect of a Smooth Criminal Playboy, you can better picture him breaking out in magical dance moves, whereas a full sleight of hand magician wouldn't be able to pull it off as well.
Edit: the tophat would make it more "believable" when he pulls out random shit, and therefore not as impressive as the fedora.