I'm not surprised at all. The ML research community isn't a community any more, it's turned into a dog-eat-dog low-trust fierce competition. So much more people, papers, churn, that everyone is just fending for themselves. Any moment that you charitably spend on community service can be felt as a moment you take away from the next project, jeopardizing the next paper, getting scooped, delaying your graduation, your contract, your funding, your visa, your residence permit, your industry plans etc. It's a machine. I don't think people outside the phd system really understand the incentives involved. People are offered very little slack in this system. It's sink or swim, with very little instruction or scientific culture or integrity getting passed on. The PhD students see their supervisors cut corners all the time too, authorship bullshit jockeying even in big name labs etc. People I talked to are quite disillusioned, expect their work to have little impact and get superseded by a new better model in a few months so it's all about who can grind faster, who can twist the benchmarks into showing a minimal improvement etc. And the starry eyed novices get slapped by reality into thinking this way fairly early.
To be clear this is not an excuse but an explanation why I am not surprised.
And the real punchline is that the deluge of papers barely matters, as the academic field is barely moving, and the most interesting innovations are happening on the product side.
I have been in both academia and industry for years, and I don't think the model you describe is true anymore. It was definitely true 10 years ago, but the situation has flipped. Now, I see really ambitious and impactful research coming out of industry labs. Academia is often lagging behind the state of the art because they lack the resources (data, compute, and skills) to compete.
Academia is also incentivized such that everyone works on the same popular topics to secure grants and citations. This is currently LLMs, where academia needs to compete with multi-billion corporations on a technology that is notoriously expensive. In effect, many researchers work on topics that are pretty non-consequential from the get go (such as N+1th evaluation dataset), but it's the only way for them to stay relevant.
I recently talked with a PI from a well-known university lab, and asked why they were doing a startup, given the ML research problems they were working on.
They said a company was the only way to get access to the compute power they needed for that research.
A startup sounds like probably a good solution, if they get paired with the right product- and business-minded people, and together they find a winning collaboration. (Edit: Or if they get acquired rapidly in the AI boom, and negotiate the right deal to enable their research longer-term.)
One key reason you’re wrong is that many interesting things aren’t even getting published, they’re on the DL for years and eventually make it to public spheres and products.
Academia is just a daycare at this point, and many labs shouldn’t exists or get funding. The people who move the field aren’t necessarily the ones with the most citations, they’re usually hard at work in places that don’t publish at all.
To be clear this is not an excuse but an explanation why I am not surprised.