Changing the definition of drop-in definitely has me concerned and makes me not take this any seriously than other projects open-sourced by Cloudflare, particularly the ones focused on more critical parts of their systems – e.g. pingora and ecdysis.
While neither am I nor the company I work for directly impacted by this outage, I wonder how long can Cloudflare take these hits and keep apologizing for it. Truly appreciate them being transparent about it, but businesses care more about SLAs and uptime than the incident report.
I’ll take clarity and actual RCAs than Microsoft’s approach of not notifying customers and keeping their status page green until enough people notice.
One thing I do appreciate about cloudflare is their actual use of their status page. That’s not to say these outages are okay. They aren’t. However I’m pretty confident in saying that a lot of providers would have a big paper trail of outages if they were more honest to the same degree or more so than cloudflare. At least from what I’ve noticed, especially this year.
Azure straight up refuses to show me if there's even an incident even if I can literally not access shit.
But last few months has been quite rough for Cloudflare, and a few outages on their Workers platform that didn't quite make the headlines too. Can't wait for Code Orange to get to production.
Bluntly: they expended that credit a while ago. Those that can will move on. Those that can't have a real problem.
As for your last sentence:
Businesses really do care about the incident reports because they give good insight into whether they can trust the company going forward. Full transparency and a clear path to non-repetition due to process or software changes are called for. You be the judge of whether or not you think that standard has been met.
I might be looking at it differently, but aren't decisions over a certain provider of service being made by the management. Incident reports don't ever reach there in my experience.
Every company that relies on their suppliers and that has mature management maintains internal supplier score cards as part of their risk assessment, more so for suppliers that are hard to find replacements for. They will of course all have their of thresholds for action but what has happened in the last period with CF exceeds most of the thresholds for management comfort that I'm aware of.
Incident reports themselves are highly technical, so will not reach management because they are most likely simply not equipped to deal with them. But the CTOs of the companies will take notice, especially when their own committed SLAs are endangered and their own management asks them for an explanation. CF makes them all look bad right now.
In my experience, the gist of it does reach management when its an existing vendor. Especially if management is tech literate
Becuase management wants to know why the graphs all went to zero, and the engineers have nothing else to do but relay the incident report.
This builds a perception for management of the vendor, and if the perception is that the vendor doesnt tell them shit or doesnt even seem to know theres an outage, then management can decide to shift vendors
Reading this really makes me wonder how does Chrome actually optimize for the plethora of devices running v8 (under Chrome). Definitely involves tricky decisions to be taken for great performance.
I believe it is to make sure that the product remains compliant with the data guarantees that Workspace provides. You aren't paying for the latest and the greatest features, you're paying for the support and compliance guarantees your business expects.
You can also reply to incoming emails from what I know, you just cannot initiate any email directly to prevent the obvious abuse. I wonder how they plan to mitigate that apart from keeping the pricing sane.
Anthropic has by far been the most unreliable provider I've ever seen. Daily incidents, and this one seems to have taken down all their services. Can't even login to the Console.
The tongue-in-cheek jokes are kind of obvious, but even without the snark I think it is worth asking why the supposed 100x productivity boost from Claude Code I keep hearing about hasn't actually resulted in reliability improvements, even from developers who presumably have effectively-unlimited token budgets to spend on improving their stack.
I love how people like Simon Willison and Pete Steinberger spend all this effort trying to be skeptical of their own experiences and arrive at nuanced takes like “50% more productive, but that’s actually a pretty big deal, but the nature of the increase is complicated” and y’all just keep repeating the brainrotted “100x, juniors are cooked” quote you heard someone say on LinkedIn.
AI gives you what you ask for. If you don't understand your true problems, and you ask it to solve the wrong problems, it doesn't matter how much compute you burn, you're still gonna fail.
And yet even dogfooding their own product heavily, it's still a giant janky pile. The prompt work is solid, the focus on optimizing tools was a good insight, and the model makes a good agent, but the actual claude code software is pretty shameful to be the most viable product of a billion dollar company.
The source for one of the initial versions got leaked a while ago and let’s say it’s not very good architecturally speaking, specifically when compared with the Gemini CLI, which it open source.
The point of Claude Code is deep integration with the Claude models, not the actual CLI as a piece of software, which is quite buggy (it also has some great features, of course!)
At least for me, if I didn’t have to put in the work to modify the Gemini CLI to work reliably with Claude (or at least to get a similar performance), I wouldn’t use Claude Code CLI (and I say this while paying $200 per month to Anthropic because the models are very good)
There are a lot more stuff (both released and still cooking) on my products page (https://sibylline.dev/products), I will be doing a few drops this week, including hopefully something pretty huge (benchmark validation is killing me but I'm almost good to cut release).
I've been paying for the $20/m plan from Anthropic, Google, and OpenAI for the past few months (to evaluate which one I want to keep and to have a backup for outages and overages).
Gemini never goes down, OpenAI used to go down once in a while but is much more stable now, and Anthropic almost never goes a full week without throwing an error message or suffering downtime. It's a shame because I generally prefer Claude to the others.
Same here, but for API access to the big three instead of their web/app products, and Gemini also shows greater uptime.
But even when the API is up, all three have quite high API failure rates, such as tool calls not responding with valid JSON, or API calls timing out after five minutes with no response.
Definitely need robust error handling and retries with exponential backoff because maybe one in twenty-five calls fails and then succeeds on retry.
Invalid JSON and other formatting issues is more towards the model behavior I would say since no model guarantees that level of conformance to the schema. I wouldn't necessarily club it with the downtime of the API.
All the AI labs are but Anthropic is the worst. Anyone serious about running Claude in prod is using Bedrock or Vertex. We've been pretty happy with Vertex.
A core research library for MATLAB I used in a course project used to be on BitBucket, though thankfully didn't have to deal with a lot of collaboration there.
Using Workers is now what Cloudflare recommends by default, with "Static Assets" to host all the static content for your website. Pages, as I understand, is already built on the Workers platform, so it's all just simplifying the DX for Cloudflare's platform and giving more options to choose what rendering strategy you use for your website.
Had me confused for a second too, but I think it is the former that they meant.
K8s has unneeded complexity which is really not required at even decent enough scales, if you've put in enough effort to architect a solution that makes the right calls for your business.
reply