The only major "improvement" has been the ability to cheat the tests and pass the regulations, because those who made them didn't realize you cannot dramatically improve such an old technology.
>The power output of an engine that achieves 30mpg today is easily double that of just 15 years ago.
And you're completely wrong. And this can be trivially easily confirmed by looking up the power outputs of vehicles that get any given fuel economy today vs the power outputs of vehicles of like fuel economy from decades past.
With respect to small diesels specifically, ever since diesels went from naturally aspirated indirect injection to direct injection and turbos power output has skyrocketed while fuel economy has mostly flat-lined.
Fuel economy is mostly dependent on the displacement of the engine and the load to do work (like cruise the highway or plow a field at part throttle) so it doesn't vary much unless the efficiency of converting fuel to force improves (hard) or the demands of the task decrease (hard for cars, even harder for industrial applications).
I actually looked it up.
A late 19th century engine had a 26% efficiency.
1960's engines had ~34% efficiency and in the 2010's it was ~43%.
So the same fuel consumption would give you 26HP in 1900, 34HP in 1960, and 43HP today.
Edit:
Also, you're confusing theoretical and practical efficiency. For instance, a V8 engine can be 30% efficient in absolute terms, but if you can use an I4 for the same application because you now have a Turbo, your real world efficiency is way higher. This is exactly what we see happening in performance cars an large trucks: big displacement being replaced with turbos and hybrid systems.
>The power output of an engine that achieves 30mpg today is easily double that of just 15 years ago.
That's a complete lie.