I believe that geniuses like Einstein are just the tip of the iceberg. We need them, but they are not the limiting factor, and producing more of them may not have that much of an influence on scientific progress.
What really drives progress, I think is production and technology. Like better telescopes. For the cosmic distance ladder, the clever mathematical tricks are nice, and necessary, but the real important part is the precise measurements. And for that, we need precise instruments. And precise instruments need good quality materials, skilled tradesmen, good tools, etc... The tools themselves need the same, and the tools to make the tools, etc... A lot of the measurements involve travel and long distance communication, which wasn't trivial either. Mounting an expedition to the other side of the world requires a ship, a crew, etc... Then we need to collect these data, and people have to write these books, we need librarians, archivists, etc... All that so that finally, one guy can look at the data and do the maths.
And all that is far from the whole story. Skilled craftsmen and sailors need food and shelter, and the raw materials to practice their craft of course. They need farmers, miners, lumberjacks, etc... Farmers, miners and lumberjacks need their tools too. Advancing in the cosmic distance ladder is the work of millions of people over several lifespans, and some of them worked in cotton fields. We could wish the guy in the cotton fields was treated better, but it doesn't change the fact that in order to advance science, we actually need more people in cotton fields than people doing maths.
With technological progress, we can afford to have more people doing maths and provide them with enough data to be useful, but that's a very recent development.
A better theory is that it is legal innovation. If the wealthiest and most powerful are threatened by progress ... there will be no progress. If the wealthiest and most powerful get more power and status from progress, there will be progress. Without insurance, for example, it isn't possible to run serious sea trade. Without the joint stock corporation, it isn't feasible for the British to organise and conquer India. It takes democracy to really align the political class with increasing prosperity for the median citizen; otherwise states tend towards tinpot dictatorships.
There isn't a shortage of smart people or opportunities to make technological progress. The handicap is powerful humans looking at new approaches and saying "No" with the firmness of a man who sees a threat to their position. Even today I have a list of things that from an engineering perspective would probably result in improved prosperity, it just isn't possible to get them through the political process (nuclear power springs to mind, but I'm sure there are less controversial examples - Uber v. the Taxi industry maybe).
I get your point about not being able to do everything in a vacuum, but I think there's also just a very limited set of geniuses that are of the Einstein/Newton level that might only exist once a century if you're lucky. We have millions in academia now and how many more physicists than the 1800s? There has certainly been progress, but it seems diminished.
There's a lot of discussion on this online and some folks speaking past each other. Yeah, there has been the invention of the internet, faster computers, the blue LED, sequencing the human genome...etc, but that is argued to mainly be engineering innovations on already understood physics. Where is the next discovery on the level of general relativity? Are we just at diminishing returns now where all the low hanging fruit has been found? How many physicists wasted away researching string theory? Were we just putting resources in the wrong place?
I do strongly think that modern research has become so beauracratic that it gets in the way of actual progress. The endless paperwork, presentations, teaching...etc isn't very conducive to discovery. Your average professor is more like a project manager than what Newton did.
The physics of the 1800s had a lot of low hanging fruit. Most undergrads in physics can show you a derivation of Maxwell's equations from first principles, and I think a fair few of them could have come up with it themselves if they were in Maxwell's shoes. The hard truth is that the physics/math of today is just much further afield, and much harder.
Very little of the stuff physicists came up with in the 19th century was obvious or low hanging at the time. And no your undergrads would likely have never came up with Maxwell's equations on their own.
Yeah, I agree with you on this one. It's kind of easy once you understand it, but it took Maxwell to figure it out. I might not have figured that out in ten centuries by myself lol.
I definitely think it takes a Maxwell to describe the physics behind displacement current (which I think was not around in Maxwell's initial drafts), but the derivations of the four classic Maxwell equations are fairly straightforward multivariable calculus problems that start with fairly simple assumptions. Multivariable calculus was relatively young in his era, but I think a motivated student from the current era could discover it given a summer of thought experimentation.
That's a highly biased opinion. The Newtonian conception of physics is trivial for us, and we can say that most people could come up with the ideas by their own, but that's because our world conception is based on those ideas; it's already implicit in how we understand the world. That's why Newton was so important, there was a shift in the whole conceptualization of the physical world. With Maxwell's equations is similar. The interpretation as waves, the fact that the equations are Lorentz symmetric but no Newton symmetric, etc. All that is free for us, and it is not obvious at all.
Spend a trillion dollars building a bigger supercolider if you want more fundamentals. I think the application of these discoveries is much more important at this point. What do we actually use GR for? Corrections to gps, and ... it's not like it's gonna cure cancer. Technology will do that. We've exhausted all the fundamentals that we can access, that's what's happened. You want more fundamentals, you need more access. You want more access, Get ready to ride the exponential curve of costs
That is pretty contentious even in physics circles (from what I've heard in videos). The build me a bigger particle collider might not have any impact. We might not see any of the hypothetically proposed stuff.
You need a certain amount of motivation, grid, etc. If these skills would be broadly available and the situation also, we would have a lot more Einsteins today.
There is plenty of things which do not require global projects building multibillion devices.
I'm not so sure. It seems these kinds of people are exceedingly rare that grasp a fundamental truth we've all missed. ChatGPT is really cool, but building a really really big neural network and training it off all the stuff on the internet certainly seems less impressive than discovering calculus. I mean ..LLMs are impressive and magical of course.
The the single best way to drive science and technology: teach everyone to format a json string share it via a URL.
The people on the bleeding edge of science and technology aren't just sitting in futuristic labs, surrounded by holograms and AI tech assistants, spouting one brilliant insight after another.
They are spending most of their days arranging various types of poorly formatted data, be it scientific metadata, purchase orders, journal proofs, grant proposals, interview schedules, their shopping list, or maybe (for a brief moment) actual scientific data. The real productive geniuses are the ones who figure out a way to ignore all this noise for along enough to get something done. It all comes down to logistics in the end.
A huge fraction of this work could conceivably be automated, or just trivial if people migrated to a better data structure. Maybe LLMs will help us bring order here, but that's only half the battle: part of learning to format data is thinking about the formal structure, what consumers will actually need, what parts are duplicated, etc.
20 years ago Tim Burners-Lee was advocating to clean this up via a better structured web [1], but, for various reasons, we haven't made much progress. I suspect a big reason is that the vast majority of people would struggle to understand the purpose of structured data: like basic literacy it's only useful if other people in your community use it.
I love Scott, but I think OP's meaning is not about verifying discoveries, but rather having the tools necessary to create discoveries in the first place. You can't invent an electric motor without electricity. Alternatively, think about how Da Vinci has many notebooks full of ideas- but lacked the resources to create them.
Scott's article is mostly on point here, but he dismisses the "respect" idea too quickly. As humans we are confronted with an overwhelming array of information and opinions from a huge number of sources. It's natural to apply a filtering based on our perceived reputation of said sources. And hearing that someone does not believe that HIV causes Aids despite extremely compelling evidence to the contrary is good cause for decreasing that reputation value.
What really drives progress, I think is production and technology. Like better telescopes. For the cosmic distance ladder, the clever mathematical tricks are nice, and necessary, but the real important part is the precise measurements. And for that, we need precise instruments. And precise instruments need good quality materials, skilled tradesmen, good tools, etc... The tools themselves need the same, and the tools to make the tools, etc... A lot of the measurements involve travel and long distance communication, which wasn't trivial either. Mounting an expedition to the other side of the world requires a ship, a crew, etc... Then we need to collect these data, and people have to write these books, we need librarians, archivists, etc... All that so that finally, one guy can look at the data and do the maths.
And all that is far from the whole story. Skilled craftsmen and sailors need food and shelter, and the raw materials to practice their craft of course. They need farmers, miners, lumberjacks, etc... Farmers, miners and lumberjacks need their tools too. Advancing in the cosmic distance ladder is the work of millions of people over several lifespans, and some of them worked in cotton fields. We could wish the guy in the cotton fields was treated better, but it doesn't change the fact that in order to advance science, we actually need more people in cotton fields than people doing maths.
With technological progress, we can afford to have more people doing maths and provide them with enough data to be useful, but that's a very recent development.