@
FayeReagan
First of all: "Does Something Wrong?"...I believe should be: "Is something wrong?"
A little more obscure note with Japanese slang usage :
Saying you're "white" in Japanese is another way of them saying of saying: innocent, pure, or without ulterior motives...etc. In this context, he's basically saying he won't do anything to wrong them, to try and put them at ease, since they're freaking out over him being from a duke family and all. In other contexts it can mean as you've previously guessed: naivete, beginner, blank...etc, but it doesn't seem to be case here. The confusion from the other characters would be them only hearing "white" because the game characters wouldn't know Japanese slang meanings.
Overall, finding a dedicated proofreader would really help. I suggest putting a recruit page at the end of your chapter until you find one. If you can't fine one, then I recommend using a free plugin called Grammarly, which will help to a degree, but still nothing beats a native English speaker joining you. Good luck with your future endeavors.