I think this post unveils a great truth that I never grasped: estimates are a political tool to decide what gets done and what doesn't get done. Thanks for putting it so nicely!
One thing that I'd like to understand then is _why_... Why doesn't management use a more direct way of saying it? Instead of asking for estimates, why don't they say: we have until date X, what can we do? Is it just some American way of being polite? I am sincerely curious :)
Because manager have budgets that are translated in human hours/days of work.
So they need to know the cost of each feature to decide which they're going to pick with their budget and deadlines.
Think of managers as kids in a toy/candy shop with a $X bill in hand.
If items don't have price, how are they suppose to choose? They want everything, but they are limited by their budget.
I think because capitalist employment is inherently adversarial. If employers (and managers) reveal the time budget, employees may take advantage and reduce output to expand to fill the deadline. Tight schedules squeeze employees, so hiding the real time constraint allows management to exert pressure by adjusting the deadline. Employees that realize the bluff and ignore fake schedule pressure can be identified, marginalized, and eliminated.
Avoiding this degrading game is half the reason I preferred contracting.
As a person that has never encountered a complex software project that can be accurately estimated, I am being a bit skeptical.
The author did make examples of when estimation is possible: easy projects with a very short time horizons (less than an a couple of days, I'd say).
I'd love to hear some examples of more complex software projects that can be estimated within a reasonable variance.
However, I think it should also be acknowledged that the point of the article seems to be in a different direction: it _doesn't really matter_ that you have a good time estimate, because asking for an estimate is just a somewhat strange way for the management chain to approach you and then tell you how much time you have to deliver.
> easy projects with a very short time horizons (less than an a couple of days, I'd say).
The example I quoted said hours, not days. But even taking your claim of days as estimable, I have seen much better.
An example of weeks-long projects I regularly estimate accurately would be things like “in our Django monolith, add this new field/model, and update the state machine with these new transitions, and update the API, and surface the feature in the UI, including standard e2es and UT coverage”. With a team of 10-15 we regularly hit days-to-weeks estimates with in the ballpark of 90% accuracy. (Ie 1-in-10 slips)
An example of year-long projects I have seen waterfall’d successfully are IP protocol implementations where the RFC is clear, base frameworks exist, and the org has engineers with decades of individual experience implementing protocols in the same framework. IOW you have senior-staff or principal engineers on the project.
> the point of the article seems to be in a different direction: it _doesn't really matter_ that you have a good time estimate
I think the idea that you always start with time and define the work is also myopic. In some dysfunctional orgs I’m sure this is true, but it’s not the whole picture.
For the fully-generalized principle at play here, I’m a big believer in the “cost / time / scope” tradeoff triangle. In other words, pick two as your free parameters, and the third is then determined. Sometimes time is the output of a calculation, and resource/scope are the input. Sometimes time can be the input and scope the output. It depends.
But the article opens by claiming it’s impossible to estimate time, given a fixed scope and cost(resource) input, which is simply false/over-generalizing.
> An example of year-long projects I have seen waterfall’d successfully are IP protocol implementations where the RFC is clear, base frameworks exist, and the org has engineers with decades of individual experience implementing protocols in the same framework.
Article responds to you with:
“For most of us, the majority of software work is not like this. We work on poorly-understood systems and cannot predict exactly what must be done in advance. Most programming in large systems is research…”
It is good to target the strongest point, not the weakest.
Yeah, that's a cop-out. All the easy things are easy! Well, yes, they are. It's the hard ones that -you know- are hard. That the easy things are easy is no real objection to TFA, it's just pedantry.
And then goes on describing two things for which I bet almost anyone with enough knowledge of C and Redis could implement a POC in... Guess what? Hours.
At this point I am literally speechless, if even Antirez falls for this "you get so quick!!!" hype.
You get _some_ speed up _for things you could anyway implement_. You get past the "blank screen block" which prevents you from starting some project.
These are great useful things that AI does for you!
Shaving off _weeks_ of work? Let's come back in a couple of month when he'll have to rewrite everything that AI has written so well. Or, that code would just die away (which is another great use case for AI: throw away code).
People still don't understand that writing code is a way to understand something? Clearly you don't need to write code for a domain you already understand, or that you literally created.
What leaves me sad is that this time it is _Antirez_ that writes such things.
I have to be honest: it makes me doubt of my position, and I'll constantly reevaluate it. But man. I hope it's just a hype post for an AI product he'll release tomorrow.
> The engineer who truly understands the problem often finds that the elegant solution is simpler than anyone expected.
> The engineer who starts with a solution tends to build complexity in search of a justification.
I do agree this is a good point, I just find it funny that it comes from "staying 14 years at Google".
This is literally the reason why I left Google first, and Meta second. Finding simple solutions will get you absolutely nowhere in a place like those. You have to find complex solutions with a lot of stakeholders, alignment, discussions, escalations... Why ship one button if you can ship 100 and get you, your team and your manager promoted in the process?
I spent the last 14 days chasing an issue with a Spark transform. Gemini and Claude were exceptionally good at giving me answers that looked perfectly reasonable: none of them worked, they were almost always completely off-road.
Eventually I tried with something else, and found a question on stackoverflow, luckily with an answer. That was the game changer and eventually I was able to find the right doc in the Spark (actually Iceberg) website that gave me the final fix.
This is to say that LLMs might be more friendly. But losing SO means that we're getting an idiot friendly guy with a lot of credible but wrong answers in place of a grumpy and possibly toxic guy which, however, actually answered our questions.
Not sure why someone is thinking this is a good thing.
What I always appreciate about SO is the dialogue between commenters. LLMs give one answer, or bullet points around a theme, or just dump a load of code in your IDE. SO gives a debate, in which the finer points of an issue are thrashed out, with the best answers (by and large) floating to the top.
SO, at its best, is numerous highly-experienced and intelligent humans trying to demonstrate how clever they are. A bit like HN, you learn from watching the back and forth. I don't think this is something that LLMs can ever replicate. They don't have the egos and they certainly don't have the experience.
Whatever people's gripes about the site, I learned a hell of a lot from it. I still find solutions there, and think a world without it would be worse.
The fundamental difference between asking on SO and asking an LLM is that SO is a public forum, and an LLM will be communicated with in private. This has a lot of implications, most of which surround the ability for people to review and correct bad information.
The other major benefit of SO being a public forum is that once a question was wrestled with and eventually answered, other engineers could stumble upon and benefit from it. With SO being replaced by LLMs, engineers are asking LLMs the same questions over and over, likely getting a wide range of different answers (some correct and others not) while also being an incredible waste of resources.
Providing context to ask a Stack Overflow question was time-consuming.
In the time it takes to properly format and ask a question on Stack Overflow, an engineer can iterate through multiple bad LLM responses and eventually get to the right one.
The stats tell the uncomfortable truth. LLMs are a better overall experience than Stack Overflow, even after accounting for inaccurate answers from the LLM.
Don't forget, human answers on Stack Overflow were also often wrong or delayed by hours or days.
I think we're romanticizing the quality of the average human response on Stack Overflow.
The purpose of StackOverflow was never to get askers quick answers to their specific questions. Its purpose is to create a living knowledge repository of problems and solutions which future folk may benefit from. Asking a question on StackOverflow is more like adding an article to Wikipedia than pinging a colleague for help.
If someone doesn't care about contributing to such a repository then they should ask their question elsewhere (this was true even before the rise of LLMs).
StackOverflow itself attempts to explain this in various ways, but obviously not sufficiently as this is an incredibly common misconception.
What I'm appreciating here is the quality of the _best_ human responses on SO.
There are always a number of ways to solve a problem. A good SO response gives both a path forward, and an explanation why, in the context of other possible options, this is the way to do things.
LLMs do not automatically think of performance, maintainability, edge cases etc when providing a response, in no small part because they do not think.
An LLM will write you a regex HTML parser.[0]
The stats look bleak for SO. Perhaps there's a better "experience" with LLMs, but my point is that this is to our detriment as a community.
Humans do not know what’s right. What’s worse is the phenomenon of people who don’t actually know but want to seem like they know so they ask the person with the question for follow up information that is meaningless and irrelevant to the question.
Hey, can you show me the log files?
Sure here you go. Please help!
Hmm, I don’t really know what I’m looking for in these. Good luck!
> What I always appreciate about SO is the dialogue between commenters.
Stack Overflow is explicitly not for "dialogue", recent experiments (which are generally not well received by the regulars on the meta site) notwithstanding. The purpose of the comments on questions is to help refine the question and ensure it meets standards, and in some cases serve other meta purposes like pointing at different-but-related questions to help future readers find what they're looking for. Comments are generally subject to deletion at any time and were originally designed to be visually minimal. They are not part of the core experience.
Of course, the new ownership is undoing all of that, because of engagement metrics and such.
SO was somewhere people put their hard won experience into words, that an LLM could train on.
That won't be happening anymore, neither on SO or elsewhere. So all this hard won experience, from actually doing real work, will be inaccessible to the LLMs. For modern technologies and problems I suspect it will be a notably worse experience when using an LLM than working with older technologies.
It's already true for example, when using the Godot game engine instead of Unity. LLMs constantly confuse what you're trying to do with Unity problems, offer Unity based code solutions etc.
> Isn’t back and forth exactly what the new MoE thinking models attempt to simulate?
I think the name "Mixture of Experts" might be one of the most misleading labels in our industry. No, that is not at all what MoE models do.
Think of it rather like, instead of having one giant black box, we now have multiple smaller opaque boxes of various colors, and somehow (we don't really know how) we're able to tell if your question is "yellow" or "purple" and send that to the purple opaque box to get an answer.
The result is that we're able to use less resources to solve any given question (by activating smaller boxes instead of the original huge one). The problem is we don't know in advance which questions are of which color: it's not like one "expert" knows CSS and the other knows car engines.
It's just more floating point black magic, so "How do I center a div" and "what's the difference between a V6 and V12" are both "yellow" questions sent to the same box/expert, while "How do I vertically center a div" is a red question, and "what's the most powerful between a V6 and V12" is a green question which activates a completely different set of weights.
I don't know if this is still the case but back in the day people would often redirect comments to some stackoverflow chat feature, the links to which would always return 404 not found errors.
You can ask an LLM to provide multiple approaches to solutions and explore the pros and cons of each, then you can drill down and elaborate on particular ones. It works very well.
It's flat wrong to suggest SO had the right answer all the time, and in fact in my experience for trickier work it was often wrong or missing entirely.
The example wasn't even finding a right answer so I don't see where you got that..
Searching questions/answers on SO can surface correct paths on situations where the LLMs will keep giving you variants of a few wrong solutions, kind of like the toxic duplicate closers.. Ironically, if SO pruned the history to remove all failures to match its community standards then it would have the same problem.
"But losing SO means that we're getting an idiot friendly guy with a lot of credible but wrong answers in place of a grumpy and possibly toxic guy which, however, actually answered our questions."
For the record I was interpreting that as LLMs are useless (which may have been just as uncharitable), which I categorically deny. I would say they're about just as useful without wading through the mire that SO was.
>> Eventually I tried with something else, and found a question on stackoverflow, luckily with an answer. That was the game changer and eventually I was able to find the right doc
Read carefully and paraphrase to the generous side. The metaphor that follows that is obviously trying to give an example of what might be somehow lost.
Yes, it does answer you question, when the site lets it go through.
Note that "answers your question" does not mean "solving your problem". Sometimes the answer to a question is "this is infeasible because XYZ" and that's good feedback to get to help you re-evaluate a problem. Many LLMs still struggle with this and would rather give a wrong answer than a negative one.
That said, the "why don't you use X" response is practically a stereotype for a reason. So it's certainly not always useful feedback. If people could introspect and think "can 'because my job doesn't allow me to install Z' be a valid response to this", we'd be in a true Utopia.
It entirely depends on the language you were using. The quality of both questions and answers between e.g. Go and JavaScript is incredible. Even as a relative beginner in JS I could not believe the amount of garbage that I came across, something that rarely happened for Go.
I'm not sure he intended it as an insult. Despite not using LLMs for HTML or JavaScript often (if at all), I was under the impression that it was one of their strong areas.
In my experience, at least in circles I've been around, I have found other developers who are a little too high on their own supply so to speak use "hurr durr web dev" as the "easy not real dev" insult.
So I both wanted to shame that in case they were and also categorically show otherwise.
Because what you’re describing is the exception. Almost always with LLM’s I get a better solution, or helpful pointer in the direction of a solution, and I get it much faster. I honestly don’t understand anyone could prefer Google/SO, and in fact that the numbers show that they don’t. You’re in an extreme minority.
> But losing SO means that we're getting an idiot friendly guy with a lot of credible but wrong answers in place of a grumpy and possibly toxic guy which, however, actually answered our questions.
Which by the way is incredibly ironic to read on the internet after like fifteen years of annoying people left and right about toxic this and toxic that.
Extreme example: Linus Torvalds used to be notoriously toxic.
Would you still defend your position if the “grumpy” guy answered in Linus’ style?
> Would you still defend your position if the “grumpy” guy answered in Linus’ style?
If they answered correctly, yes.
My point is that providing _actual knowledge_ is by itself so much more valuable compared to _simulated knowledge_, in particular when that simulated knowledge is hyper realistic and wrong.
Sadly, an accountable individual representing an organization is different from a community of semi-anonymous users with a bunch of bureaucracy that can't or doesn't care about every semis anonymous user
That grumpy guy is using an LLM and debugging with it. Solves the problem. AI provider fine tunes their model with this. You now have his input baked into it's response.
How you think these things work? It's either a human direct input it's remembering or a RL enviroment made by a human to solve the problem you are working on.
Nothing in it is "made up" it's just a resolution problem which will only get better over time.
I'm hoping increasing we'll see agents helping with this sort of issue. I would like an agent that would do things like pull the spark repo into the working area and consult the source code/cross reference against what you're trying to do.
Once technique I've used successfully is to do this 'manually' to ensure codex/Claude code can grep around the libraries I'm using
> Logs were designed for a different era. An era of monoliths, single servers, and problems you could reproduce locally. Today, a single user request might touch 15 services, 3 databases, 2 caches, and a message queue. Your logs are still acting like it's 2005.
Perhaps it's time to take back the good things from 2005.
Aren't we just reinventing programming languages from the ground up?
This is the loop (and honestly, I predicted it way before it started):
1) LLMs can generate code from "natural language" prompts!
2) Oh wait, I actually need to improve my prompt to get LLMs to follow my instructions...
3) Oh wait, no matter how good my prompt is, I need an agent (aka a for loop) that goes through a list of deterministic steps so that it actually follows my instructions...
4) Oh wait, now I need to add deterministic checks (aka, the code that I was actually trying to avoid writing in step 1) so that the LLM follows my instructions...
5) <some time in the future>: I came up with this precise set of keywords that I can feed to the LLM so that it produces the code that I need. Wait a second... I just turned the LLM into a compiler.
The error is believing that "coding" is just accidental complexity. "You don't need a precise specification of the behavior of the computer", this is the assumption that would make LLM agents actually viable. And I cannot believe that there are software engineers that think that coding is accidental complexity. I understand why PMs, CEOs, and other fun people believe this.
Side note: I am not arguing that LLMs/coding agents are nice. T9 was nice, autocomplete is nice. LLMs are very nice! But I am starting to be a bit too fed up to see everyone believing that you can get rid of coding.
Technical problems are generated by lack of knowledge. One type of lack of knowledge is interaction with people. You'll never know everything that another person wants to communicate to you because of several reasons.
But even in the case of magically fixing people problems - for example, if you are working on a solo project - you will still have technical debt because you will still have lack of knowledge. An abstraction that leaks. A test that doesn't cover all the edge cases. A "simple" function that was not indeed that simple.
The mistake you want to avoid at all costs is believing you don't have a knowledge gap. You will always have a knowledge gap. So plan accordingly, make sure you're ready when you will finally discover that gap.
One thing that I'd like to understand then is _why_... Why doesn't management use a more direct way of saying it? Instead of asking for estimates, why don't they say: we have until date X, what can we do? Is it just some American way of being polite? I am sincerely curious :)
reply