Man this place can really be speedreader central sometimes
The reason our MC here doesn't want to house the girls is because he isn't ready to take on the responsibilities as a hero of another nation, something he all but swore off when he had his house and met ferris. He caved because it is just the only moral course of action to take in his situation, at least that's how he saw it. He was going to be convinced of it eventually anyway, and he mostly had doubts because he was considering how the house spirit would've felt about them. He wasn't confident about housing the two strangers, but he was perfectly fine with the sage elf when it came to it. Essentially, he wasn't ready to be at the beck and call of royalty nor become the centerpiece of a potentially huge and bloody war (mentioned previously in the chat he had when he first met the vice president), but he felt it was the right thing to do (heroic qualities for juxtaposition)