not necessarily, it depends on what is the focus on the story. besides in the case of this manga/novel there are reasons why the changes happen, even if the mc doesn't really think about it as much, the mc's actions change because the age and gender being changed, but that only happens after the mc meets the old guy and his family (that's an important detail to understand why the mc started to change to be closer to the body's physical age), but the mc still remains effected by the previews life in many ways. besides we don't know that much about the mc's previews life as much but we can tell for example, that the reason the mc cried when seeing the field with the sunset was because the mc felt relief and couldn't hold of the emotions, this would have happened even if the mc would have been the same gender as before.
besides as far as i can tell the mc's thought prosses hasn't changed after getting isekaied, and some changes that do happen to the mc's personality are not because of the gender swap but mostly do to the age change, even if the mc was a woman in the previews life, those changes would happen the same.
the biggest problem why a lot of thinks are not made clear, is because the mc doesn't completely understand his own emotions, but it is shown in other ways to the reader.
sure the story could have been as you say with the same gender in the previews life, and not much would have changed, but that could be said with almost every story, if one thing was changed it could be better (at least some readers think like that), but here is the think, when it comes to good writing the authors creative decisions are what matter most in order for the story to be genuine. if you take away these decisions then you would end up with something boring that basically already exist but slightly deferent.
for me the most important part is for the author to have the creative decision, even if i don't like a choice they made. if the author made changes because others think it was a bad choice, then it would lead to something boring or something that just feels out of place, or just ingenuine. there are plenty of examples of what i am saying out there, besides most good or popular books/games/shows/anime/manga or whatever, always have something i don't like, but i still love them for what they are. that is true for all forms of art, they are good as long as they are made by an artist that remain genuine.
there are plenty of examples that i can give, but even if you yourself think about a story that you love you will always find things in them that you hate or don't like.