@InsaneInsomniac
1) His memory of the desert is from events YEARS in the future.
2) The thing about deserts of sand, the dunes move, the landscape of a desert changes every year with winds blowing the piles of sand into new piles. Things are always being covered, uncovered and covered again.
Its very likely that at this time, the temple is covered in sand (your probably right that hes standing right on it, that is a typical troupe)
and that years in the future when he had found it the first time was during a period it was uncovered.