People are annoyed because it just doesn't mesh with the rest of the world portrayed. There is no way it happens this often if it had taken until this long to happen in the first place.
It is the same reason ppl get annoyed when they are reading a regular shoujo romance slice of life with no fantastical aspects, but every 3-4 chapters a character - who otherwise has no athletic ability - inexplicably pulls some stud moves by saving the fmc from falling airplanes by flying up and punching them. To everyone's shock and amazement, only to never actually be properly addressed why they can fly or punch airplanes out of existence or what happens to the passengers, and noone seems to care other than sometimes saying how amazing they are.
Even if it ends up being used as a plot device that does move the romance forward.
