I'm not familiar with Osomatsu, so I could 100% wrong but doesn't Kanzaki's lines about how "the story won't progress without me" simply means that without him, there was no way for Mogari to see the ghost?
I don't think it's outright confirmed, but in chapter 1, Mogari said "I can see them...