I think it’s because the MC would rather plow the fields off screen and this is all bonding between the 2 instead.
More like he is busy fixing stuff outside than staying with his family.
I mean, we have only seen one chapter showcase MC sleeping in same bed as Rico after they married (on the day they married), and nothing even happened that night.
Plus, MC has never been shown or mentioned to be sharing bed with Mei either, who sadly had been turned to a background character since the marriage plot, until the recent "breaking Self-Restriction" plot, after which she is being ignore again (aside from these Extra side chapters focusing on both Rico & her).
I mean, just see how MC had been married to both Rico & Mei longer than Rico's Elder Brother (aka., the Current King) had been in relationship with his Commoner Fiancee, but he had already managed to get her knocked-up.
While i can't be sure about how fertile Rico is as a human, but i doubt Mei who is a child of an Orc Father & Sheep-race Mother would have low fertility.
This is quite different compared to the Web Novel, where it was specifically mentioned that MC alternatively sleep with both of them and has a pretty busy night life with them on regular basis (no details, just mentioned; also not sure how it's in the LN though).
Edit : I found the LN raws, and while i couldn't check how it is in the LN regarding the sex life of MC, i did find out that currently the Manga is covering LN Vol.3 plot, and the series had ended at LN Vol.5.
Also,
Rico & Mei both give birth to MC's children in the final 5th Volume.
There are also a lot of other new girls, but not sure whether they are part of MC's Harem or not.