Everyone here is talking about how inaccurate coming back later and not wrecking shit up the moment he saw Filo getting abused is, but it actually falls in line perfectly with Naofumi's character. He's in a new world with new laws, with his stats brought back to zero. He doesn't have the rage shield that feeds off his rage or any other of his shields for that matter, plus he has a rather weak and unstable party. Being cautious in the face of the unknown and strategizing also falls well within his character. Without his complete party on top of everything else, Naofumi can't afford to go in guns blazing and you can see just how upset he is about the whole ordeal when they're talking about it on the roof.