This is the exact same reason that it always seems like you are in “the slow lane” in congested traffic. You spend considerably more time going slow than going fast, and while your lane is slow you have to watch all the other sporadically fast lanes go by.