While I understand what is the problem that the novel readers have with the manga, from my perspective as a manga and anime only. I can pretty much understand what is happening.
I know there must be things that are being skipped or postponed in both, but really, from the looks of it, it isn't changing the characters too much, nor the story.
I mean, just look at the manga, anime and novel of Tanya, they have key differences in the characters, and yet they all tell the same story. And I find a great joy in experiencing the same story told in three entertaining and sliglthly different ways. This same situation happens with other stories like the superheroes having the comics, the liveaction movies/shows and the animated movies/shows. The three of them have their own things to enjoy that the others don't.
Just look at it this way. It isn't as bad as The Promised Neverland season 2.