It's probably called the Isekai Magic Church because the thing MC was researching magic in specifically said the physical world is considered separate from the one they draw magic power from. The title of this and those visions were sort of red herrings. This is probably just a world that has been devastated in its past, gone all old timey as a result, and the visions are just visions granted by that woman rather than memories. Even the main character's hair color being weird considering his family was probably just something slightly different than "I'm Japanese, so my reincarnation is arbitrarily carrying features from my previous life for no apparent reason other than to cause trouble."