It's interesting. My instinct is very much that the photo man is an evil spirit, and Mr. Kanamori was cursed by him. However, the fact is that the house itself was shown first, and yet bore no damage—this implies that the photo man's power, if it is a curse, is not so general as to be a curse of misfortune. Perhaps he can only affect his target, or only living things.
Another interesting tidbit is that Mr. Kanamori is the target of the photos first (his house is shown), and then he takes the lead in ridding the school of the photo man, but even as he does so, he is concerned first and foremost with the children getting ripped off. Therefore, we cannot say he was targeted out of vengeance. However, we could imagine that he might have somehow realized that the photo was of his house, or he saw the photo man outside taking his picture... both would be cause for concern.
Either way, the photo man feels much more sinister than Mr. Kanamori. He appears where he oughtn't. His stare is intense. He stands halfway in and out of a doorway. His presence is concerning to the main character...