For someone looking to lighten his work load, MC is an expert at adding new things to his todo list.
As for the complaint about the plot, the whole point of adding men to the village was originally because he couldn't handle all of the women himself. Now instead they are making a second village with men to be used as laborers. Which doesn't actually fix the problem that was initially stated, that he was the only fertile male in the current village (until the Beastmen boys and his son grow up). Plus is doubles back on the statement that "oh, no one can work on the grounds except for me because I have the AFT", by stating that men can work on the fields just fine as long as we have enough of them. Why? Why can they work the fields without the Farming Tool, but the women can't? It makes no sense. He's overworked. He needs help. The women can't help, but the men, who are weaker than the women, are somehow able to help?
Also how exactly are they suppose to commute from their village to Taiju village through the forest of death to work on the fields. Since they have sent out a scouting party, it sounds like they aren't putting the new village adjacent to the current one, so you would have to traverse through the forest to get to Taiju village to work the fields to traverse back at the end of the day. The logistics of the whole thing is horrific. There's a reason why the workers quarters is generally in the middle of the plantation/field. Or will the angels/dragons provide a psuedo taxi service to move the men back and forth daily?