I think most of you are missunderstanding their childhood relationship.
Like dude literally invited Shigure to his house himself.
Kotaro never considered it bullying, he is not angry cause the sexual harrasment, he is angry cause he lose every time.
The four of them are just competitive to the extreme.
His vengance was just to defeat them cause he cannot stand losing.