club -> more people -> less time to talk, even if they are both in the club. Especially since the story established that Kurokawa is very coveted among the class.
He'd also likely be walled off from her by those who consider him trash just like in the classroom. Less likely to happen in the Manga club though because all of us who would join that are trash...