"Tsukiuta." Gekijouban RABBITS KINGDOM THE MOVIE - Ch. 1

[MangaDex]

 

[MerlinD]

is this just a memory they are seeing or something else

 

[childishflower]

is this just a memory they are seeing or something else

Yes, it's like a memory! The setting of this universe is that a change in one timeline also influences other timelines and other worlds.
When Hajime and Shun pick up the book, they are influenced by their memories in another world and remember their lives/pasts in this fantasy world. The book is essentially a key item that connects the "normal" timeline to their "isekai" timeline, serving as an impetus that causes an overlap.