I think the age regression was a way for the author to give us young adults in the audience an easier and more compelling connection to the MC, but still explaining why he’s so powerful and experienced in magic.
The people that are fretting over some sexual deviant conspiracy of an old man and a young girl are projecting their own desires.