You have a point.
Maybe in the end they'll just keep living in the temple without any closure cause the premice of the plot is him paying the debt helping the temple. But It would be a unpopular choice.
The issue with harems it that the ending only is good for a part of people.
Like you said...
Guys, what are the chances of a harem ending??
I know it usually don't go this way but... you know this plot has some harem like his father and Mia family.