The mistake here is the perspective not the story, this story makes much more sense as a light novel than a manga. Making the story a flashback while risky, I don't think is its biggest flaw. The way the story is cut, positioned and presented is where it lacks, but that's my optimism that later...