Well also:
1) He has sunken cheeks + eyebrows that are drawn in a certain way. 100% villain once the glasses come off, just add fangs
2) He leaves the children to go out into the city by themselves because he has to avoid sunlight
3) Every religious figure is evil in almost all manga, not just in isekai genre
He's so far portrayed in a way kind of similar to the dude who was trying to revive his dead daughter by making a chimera in Full Metal Alchemist, I think right after the first arc

a very nice, and otherwise well-meaning person, but has been made deranged in some way and ultimately a tragic character. My guess is that this guy hates what he has become