Depends on what you want to do. But my 2 cents are that like all new technology, LLMs will become a commodity. Which means that everybody uses them but few people are able to develop them from scratch. It's not different from other things like databases, GPU drivers, 3D engines for games, etc. That all involves a lot of hardcore computer science and math. But lots of people use these things without being hindered by such skills.
It probably helps a little to understand some of the internals and math. Just to get a feel for what the limitations are.
But your job as a software engineer is probably to stick things together and bang on them until they work. I sometimes describe what I do as being a glorified plumber. It requires skills but surprisingly few skills related to math and algorithms. That stuff comes in library form mostly.
So, get good at using LLMs and integrating what they do into agentic systems. Figure out APIs, limitations, and learn about different use cases. Because we'll all be doing a lot of work related to that in the next few years.
LLMs already are a commodity. Google has already kicked off the competitive price wars. Plus I’ve already seen some local companies just buy a beefy GPU server and deploy an open LLM model. While OpenAI is still trying to push quality, their competitors have already positioned themselves to offer the lowest possible prices. And since Nvidia has no easy path for scaling up compute anymore, I also wouldn’t bet on much larger LLMs anytime soon.
That means, if you learn more about the internals of LLMs, your market angle is going to be artisanal customised models. Fashion is commoditised, but people still pay for a custom tailored suit. In the same way companies will continue to pay for finetunes optimised for their business usecase.
If you decide to focus more on the application of LLMs, you should really invest into high-level architectural skills. Good “code completion” models can already do what an outsourced 10 bucks per hour developer used to do. Your job in the future is going to be to decide the structure of which fuse and against the towel and or which type of state is being stored and managed. But the actual coding of the UI forms and the glue code to synchronise from an SQL query to the client state, that part is probably going to be fully outsourced to LLMs.
I think this is the key point - LLMs will go through a commoditization phase and I think you left out a key example from a technology and business context: search engines. There was a huge trend where everyone needed search and was building search, etc. and a couple decades later there are lots of companies that evaporated and a few left standing.
There was also a dot com bubble, mostly bursting not because of search but because there were a lot of what today would be "AI startup" but is just a web app calling AI Api's. So there's likely to be some bubble burst but it should be smaller maybe hitting more of these small tools that eventually become features.
>It's not different from other things like databases, GPU drivers, 3D engines for games, etc.
Not quite the same. E.g. databases are a part of the system itself. It's actually pretty helpful for a SWE to understand them reasonably deeply, especially when they're so leaky as an abstraction (arguably, even the more nuanced characteristics of your database of choice will influence the design of your whole application). AI/LLMs are more like dev tooling. You don't really need to know how a text editor, compiler or IDE works.
We have a service at work which categorizes internal documents and logs, then triggers some automation depending on the category. It processes maybe 100 per day. Previously we only used some combination of metadata, regex, and NLP to categorize. Now a call to a LLM is part of that service. We save a lot of manual time where we used to have to resolve unknown documents. The LLM can help fill out missing data, too. It's all stored as annotations so it's clear who/what edited the data.
Granted this is a pretty simple task and a low stakes scenario, but I don't think we should limit ourselves to assuming AI will always only be dev tooling.
That said, I think there is this thing in between of developing LLMs and using LLMs via APIs and the lines are of cause blurry: Training LLMs (or other neural networks). So best I think is to start digging on the surface and going deeper as long as you feel comfortable. Maybe at a certain point you will have the wish for more power full hardware. Thats the point where you need to decide how much to get invested or to rent a cluster.
But the question is what mindset will allow you to put yourself ahead of the rest. Because I suppose the OP doesn't want to end up as just another mediocre programmer.
Like any skillset, programming skills likely form a distribution pattern. There are exceptional programmers out there, I've worked with a few. "Every programmer really is just mediocre" merely indicates you have only worked with mediocre colleagues and are one yourself.
> There is no perfect software yet.
"Software" you refer to is actually 'software product', not merely 'code'. So the reality is that even with exceptional programming talent, the art of making great software products is out of reach of most teams and companies. Vision, management, product development, accurate grasp of the user needs, ..., none of these are "programming" skills.
Same. Yet being a generalist has always been the most interesting to me so I carried on that path. Ironically, now I can use an LLM for depth, I’m the one being asked how I manage to ship so much. It’s in part due to how I use LLMs for depth whilst relying on my natural breadth.
It probably helps a little to understand some of the internals and math. Just to get a feel for what the limitations are.
But your job as a software engineer is probably to stick things together and bang on them until they work. I sometimes describe what I do as being a glorified plumber. It requires skills but surprisingly few skills related to math and algorithms. That stuff comes in library form mostly.
So, get good at using LLMs and integrating what they do into agentic systems. Figure out APIs, limitations, and learn about different use cases. Because we'll all be doing a lot of work related to that in the next few years.