Ah yeah. I just remembered they aren't a group like that but they are a joke of his master, and rumors popped up but because the higher ups know it is not actually an unorthodox or anything, they leave it alone which makes others think they are some Supreme shadow lord group too powerful for others to even take on, and therefore the others are just admirers working under in honor of them, or following their ideal or something. Misunderstandings aside they realize he's stupidly strong before, and so on
It's just confusing at a first glance as they live under the assumption his alias is the leader of a group so strong nobody goes near and yet they try to fight him several times. Meh