That's just a theory, but maybe they'll receive a punishment from god, if they hurt him directly, so they can't kill him. And as soon as his powers activate he can disappear and reappear any given time. Maybe other kids used that as a way to escape by fooling the guards. Maybe they also can't starve him to death, while he is in their confinement, as it would also count as murder. (I know. Many maybes here, but I blame it on plot holes.) So image what happens, when a child without the ability to enter her/his own world until she/he is coming of age, is thrown onto that island, that doesn't have any usable cultivable plants to eat. She/He can't plant anything in her/his and starves. The End.
MC is isekaied , with his ability to do so unlooked from the beginning and his abilites became active before their time. And the goddess even told him, that he could do it, beforehand. Maybe she doesn't know, why of those kids, she blessed before, suddenly died can't interfere any other way, even though those children before had her protection. This is a different approach to see what happens, if she throws somebody in, who has smarts right from the get-go, into a talented body.
Edit: Okay I've reread 1st chapter and edited my comment. I had it total differently in my memory.