He have a sociopath feel about him, his indifference for his torture and the death of that girl in the slums, isn't it a deal of having a adult mentality, it is a disregard for what happened, not that I don't like it, on the contrary, having the comprehension about this make it even more interesting, because sociopaths can be fully functional, but they have a 'breaking point'.
Or the autor straight up is shitty at writing a development that involves getting falsely accused, tortured, bullied, 'kidnapped' (since he was forcefully taken to the castle and the school) , and so on.
@edit
So, I went and read the other manga, that have 3 chapters... Dude, what. the. fuck. It is massively better. The torture is fast and he doesn't take such a huge beating, the person whom takes him to the castle is another and makes A LOT more of sense. I don't know why the quality in the story dropped in this version, but, if I need to read other work to understand a non-sequel, then, you did a bad job.