They are all completely genderless. Despite them being called 'sir', all of the spirits don't consider themselves as a gender (except for the MC who has 17 years of being male).
However, their appearance can be more masculine or feminine, and all of these spirits seem to have feminine looks.