That wasn't universally true. In northern europe winters and long nights significantly reduced the number of hours they could work. Sure, they worked extremely hard during planting and harvesting seasons, but the rest of the time not that much. Seasonal farming means that the result wasn't proportional to the work put in but rather to whether the crops got the right amount of sun and rain.

Before that, hunter gatherer groups worked even less because they were so few and the food so plentiful.