> the vast majority of software basically still uses only a single core
And that which does use multiple cores, sometimes only scales well to a few because then other bottlenecks† start to become most significant.
Many things are not so “embarrassingly parallelisable” that they can easily take full advantage of the power available from expanding the number of processing units available beyond a certain point.
--
[†] things no longer being “friendly” to the amount or arrangement of shared L2/L3 cache as the amount of threads grow causing more cache-thrashing, hitting other memory bandwidth issues, or for really large data you might be considering issues as far down as the IO subsystem or network.
This is considering the vast majority of software basically still uses only a single core