As an engineer it's much easier to "polish" (or work on useless stuff) than to deliver real products. I see that all the time, specially with "platform" teams that have no external product requirements. They waste a lot of time trying to figure out the most amazing way to deliver something instead of delivering something in a timely fashion.

If you push in any way they start to scream "tech debt" and everyone just accepts it. I've been through a migration mandated by an infrastructure team where where were 0 improvements for the teams that used the platform, all benefits were for the platform team only, and this was green-lighted and forced upon everyone without a second thought. It's unbelievable.

> If you push in any way they start to scream "tech debt" and everyone just accepts it.

Just as real-life debt, building a company without it is unrealistic and unwise. You just have to manage it.

Its also impossible, even the "amazing" solution will be found to have holes once it hits the real world, the sooner you can get some real feedback for what you're building the easier it is to course correct and move towards the real user needs.

> To clarify, that doesn't mean "don't have multi-year roadmaps", it means "your multi-year roadmaps must deliver wins at a consistent cadence".

What you describe is exactly the opposite of research: which is collecting neverendin failures.

An environment that lives by such logic cannot really lead to major technological breakthroughts. And in fact, Amazon has very little of those to show compared to the rest of the SV.

Not all places are doing research. In fact most are not! Knowing whether you're at a company willing to bet money on R&D, or whether you're at a company that wants you to come up with actionable, is pretty crucial. As you said, Amazon is not really out there doing research - so engineers working there would do well to assume they need deliverables to keep managers happy.

I agreed but in the case of the news we're commenting this feels an awful lot like an effort that would've benefitted from a proper R&D team where only the leadership would've had to be involved in company politics, goal-setting, etc.

I think what you say is true, yet some base research takes time. If you have a “ship it” attitude, you might push teams towards taking smaller bets that they know are within reach? I don’t know how the transformer breakthroughs at Google/DeepMind happened, and it’s likely they were “shipping” things internally, but it seems clear that the people on those teams were working in a very different environment than what the author is describing.

If you look at all the defining products of Apple, they also took years from the “germ of an idea” until they could be launched, and though they might have “shipped” internally, they gained a lot by not having pressure to ship things piecemeal to customers.

Google Brain spent oodles of money developing that tech only to watch other people capitalize on the research and potentially make Google Search (one of the stickiest products I've ever seen) obsolete. Freeform, self-directed, open-ended research labs are certainly a great approach from a technology breakthrough PoV, if you have 2010s Google margins.

But it's not obvious to me that approach was even a net win for Google as a business. Did Google Brain invent the technology that killed Google? TBD I think.

I wonder what would be the alternative, though? Other companies/universities would eventually have made the same breakthroughs, and I don’t think the answer to the innovators dilemma is to do less ambitious innovation?

In the case of Google there’s a lot of internal reasons why they didn’t leverage this opportunity, but if they had done that they might have ended up making their main product even more sticky.

This _highly_ depends on your field and what you're working on.

Working on the latest and greatest social media website? Sure, ship early, ship often.

Working on medical devices? You better not ship a prototype.

Working on hardware? Too expensive to pivot from learnings, better get it right the first time.

Working for NASA? You better get it right the first time and predict all future issues that might be possible, and you better document it 9 ways to sunday.

The parent's point though is it actually doesn't. For example with NASA, there are 1000s if not 100s of 1000s of intermediate goals on a project like SLS. For example, IDK, make sure the engine hits 95% of thrust spec. Hardware? You design each IP in isolation, test in isolation, build up step by step. A0 tapeout, A1 tapeout...etc.

It's a catch 22 with ML though. What you wrote is completely true however with ML you cannot say "We will get to 98% precision and 92% recall by Q4". You do not know how long it will take (see self-driving cars). However of course you can always lie.

> "It's a catch 22 with ML though. What you wrote is completely true however with ML you cannot say "We will get to 98% precision and 92% recall by Q4"."

This applies also to ML - it applies to all tech projects, though yeah, it's harder in ML. But not figuring out the intermediate products is not an option though - your stuff will get killed prematurely if you don't.

The trick with ML is not to promise "98% precision and 92% recall by Q4", it's to figure out what kind of product is shippable with lower precision and recall. Or perhaps a stepping-stone model that allows some simpler use case, but gives you progress towards the greater goal.

It's always case-specific, but as a ML team you do need to figure out what your intermediate checkpoints are. You need to demonstrate not only progress, but that your progress is contributing to the company's goals.

I think of it as building a staircase. If we need to hit 98% eventually, we need to make certain progress now. It is possible that a new breakthrough could appear in 2 years to spike that, but the safer assumption is that our near term progress will harvest the low hanging fruit.

Therefore, intermediate and measurable milestones are to derisk something. Even if you are doing a moonshot, there are still steps you can identify. The namesake, Apollo, didn’t try for the moon on the first launch!

Exactly this. Intermediate shippables are derisking. There are a few things that this strategy represents to executives:

- "You have already received benefit X for your investment in our team, and X has been well worth it. Continued investment in us will yield benefit Y."

- "Even if the project is terminated early, or later stages become blocked due to business or technical impracticalities, the company will have gained a tangible benefit already in the form of X. There is a payout, even if it is not the full thing we wanted."

- "The intermediate products validate the technology/product/business model, such that every incremental deliverable means an overall reduction in risk to the final goal."

100%. And the sooner low level folks like me can provide this signal, the sooner the execs can make decisions. That saves cycle time and is the real secret sauce for growth.

That’s why my favorite interview question (I’ve done hundreds of interviews) is “tell me about a time you cut corners on a project”. My role as a data person is to provide enough signal as early as possible - not necessarily to give a precise answer to everything we want to know.

Right but the "how" is the tricky part. For example, wrt to ML, what is the milestone and how do you know you will hit it by a particular date? Very difficult to say with ML.

This is where smart analytics teams are super useful. It is a really hard projection estimation! Even understanding what the F1 score should be to be "good enough" is nearly impossible, much less understanding what it takes to go from "where we are" to "good enough".

What you can do is estimate something like "we think we need $measurement_value on $metric by $date, we are at $current_value. If we're not at 75% of the gap in 50% of the project time (when we have low hanging fruit), we cancel."

Yep, time-boxing is kind of the only solution.

Agreed, it's very hard, I'd like to see how many orgs have actually done this successfully and what those shippable intermediate products actually were.

1000 times this - you need to deliver something. Even very long projects such as medical research have intermediate stages and goals to meet.

Very experienced people tend to forget this from time to time too and get excited or convince themselves "big risk big reward"... I've never seen that work out.

That's true, and it justifies why some things are still better done in a university environment (or as part of an academic collaboration) - it's not possible for everything to carve out something useful in the middle.

Executive patience, focus and planning horizon are the immediately next 1-4 quarters, years 2 and 3 and perhaps 5 if you are lucky and that's it, and they might not even be around in five years.

In academia, if you are stubborn and tenured and don't care about your short term success (publications, citations, awards) you can actually decide to implement a very long-term vision, depending on how much additional funding you need (if that is a lot, you will need to also convince funding agencies or philanthropists of your vision).

Even in academia, short consistent wins are needed to ensure your lab stays open.

Heck even Andrew Wiles, someone who only needed pen and paper for research, had to publish papers during his 7 years working on Fermats last theorem.

Big tech isn't able to do certain innovations. It's not that intermediate wins don't matter, but that innovative capabilities don't come out fully formed with a business model. And I don't know of any big techs that effectively do business model and tech innovations at the same time.

If your goal is only promotions inside of big tech then definitely throw most innovative ideas out the window. But if you're interested in innovation, then big tech either needs to get a lot more entrepreneurial or less big.

I agree with this. Said more concisely, have a long term vision and an incremental path to get there. With each step or two, re-evaluate the long term vision. It may change, it may not. If it changes, update your incremental steps, if it doesn't you have more confident you're building the right thing.