Considering their power, i think their suicidal thoughts is incredibly hasty. Some can block his magic, some can fight him close range, some can read him, together they have bigger chance of winning
Not to mention, having people who know his face would spell new development. At this rate, the story is pretty much being resetted again.