That's kind of absolute bullshit, but kind of not. Unfortunately, that's not the sort of feeling you should be having at this sort of moment. The only thing I feel that is a really fair justification is the Attack Titan not following anyone, but extrapolate all the way to borderline time travel and this is really pushing the envelope.
Time travel is an absolute bitch and not something that should be done so lightly. And once the smart people who've actually studied or at least thrown a lot of time into the concept show up here, I'm sure a shit ton of holes are going to start popping up, aside from the most obvious ones.
Regardless, at least in part, seems extremely lazy.