That loser attitude of the MC even in front of servants does match his general personality. From the beginning of the series he has always allowed everybody to push him around. His only response has always been to run away, which behavior the teleportation magic naturally serves well. He's not doing any favours to the maids this way, though. The maids will grow too big heads and might make a tragic mistake one day with some noble guest. But then again, I guess that's just the author's own particular sense of humour (masochistic sense, that is) with no deeper meaning. Nevertheless, it's pretty strange to see how the servants of the household have a higher position in the hierarchy than the nobleman's legitimate son.