No need to get agitated. Some people enjoy sitting in the car for multiple hours, some people don't. I found it changed for me as I got older.
However, about those 20 minutes — this, again, is a misconception. Stop at a busy rest area or a gas station and actually start your stopwatch. If you are a single male, it might take you shorter. If you are with a family, 20 minutes is pretty much the minimum.
As a citizen I wish cars had similar driving break requirements as trucks do: mandatory 30 minutes break after 4.5 hours, 9-ish hours per day, minimum 9~11 ish hours break between driving days.
Such minimum reasonable rest is what keeps you from killing innocents with your 2-ton weapon/tool.
