According to Haru's reaction when the butler said the head of the family is in danger (Haru immediately when "mother?" instead of her supposed Father). It is probably likely there is no Father involved or the Mother is the actual person in charge of the houehold, being the head.
Inb4 the moans...