I think everyone in this manga are decent people. Yugami just happens to rub them the wrong way. Especially Kadota and Kuzumi, even though they understand him still find him too irritating to look at him objectively. Everyone is the same as Yugami deep inside, the way he acts just causes people to cringe, because society doesn't allow for that frankness.
Kuzumi hiding the notebook might be better since now no one knows how many points they actually have. So they are forced to act right, not knowing when they could be pulled. Unless they are able to keep count.