Give it enough time, every declarative language becomes a programming language. This is happening with all config files, markup languages, data formats.
The distinction between code, config and data is being erased. Everything is a soup now. Data is application, configuration is code. Code is an intermediate, volatile thing that is generated on the fly and executed in the temporary lambda containers.
I'm a tedious broken record about this (among many other things) but if you haven't read this Richard Cook piece, I strongly recommend you stop reading this postmortem and go read Cook's piece first. It won't take you long. It's the single best piece of writing about this topic I have ever read and I think the piece of technical writing that has done the most to change my thinking:
You can literally check off the things from Cook's piece that apply directly here. Also: when I wrote this comment, most of the thread was about root-causing the DNS thing that happened, which I don't think is the big story behind this outage. (Cook rejects the whole idea of a "root cause", and I'm pretty sure he's dead on right about why.)
I’ve been toying with a concept inspired by Apple’s Find My network:
Imagine a decentralized, delay-tolerant messaging system where messages hop device-to-device (e.g., via Bluetooth, UWB, Wi-Fi Direct), similar to how “Find My” relays location via nearby iPhones.
Now add a twist:
• Senders pay a small fee to send a message.
• Relaying devices earn a micro-payment (could be tokens, sats, etc.) for carrying the message one hop further.
• End-to-end encrypted, fully decentralized, optionally anonymous.
Basically, a “postal network” built on people’s phones, without needing a traditional internet connection. Works best in areas with patchy or no internet, or under censorship.
Obvious challenges:
• Latency and reliability (it’s not real-time).
• Abuse/spam prevention.
• Power consumption and user opt-in.
• Viable incentive structures.
What do you think?
Is this viable? Any real-world use cases where this might be actually useful — or is it just a neat academic toy?
1. Learn basic NNs at a simple level, build from scratch (no frameworks) a feed forward neural network with back propagation to train against MNIST or something as simple. Understand every part of it. Just use your favorite programming language.
2. Learn (without having to implement with the code, or to understand the finer parts of the implementations) how the NN architectures work and why they work. What is an encoder-decoder? Why the first part produces an embedding? How a transformer works? What are the logits in the output of an LLM, and how sampling works? Why is attention of quadratic? What is Reinforcement Learning, Resnets, how do they work? Basically: you need a solid qualitative understanding of all that.
3. Learn the higher level layer, both from the POV of the open source models, so how to interface to llama.cpp / ollama / ..., how to set the context window, what is quantization and how it will affect performances/quality of output, and also, how to use popular provider APIs like DeepSeek, OpenAI, Anthropic, ... and what model is good for what.
4. Learn prompt engineering techniques that influence the qualtily of the output when using LLMs programmatically (as a bag of algorithms). This takes patience and practice.
5. Learn how to use AI effectively for coding. This is absolutely non-trivial, and a lot of good programmers are terrible LLMs users (and end believing LLMs are not useful for coding).
6. Don't get trapped into the idea that the news of the day (RAG, MCP, ...) is what you should spend all your energy. This is just some useful technology surrounded by a lot of hype of all the people that want to get rich with AI and understand they can't compete with the LLMs themselves. So they pump the part that can be kinda "productized". Never forget that the product is the neural network itself, for the most part.
When I see stories like this, I think that people tend to forget what LLMs really are.
LLM just complete your prompt in a way that match their training data. They do not have a plan, they do not have thoughts of their own. They just write text.
So here, we give the LLM a story about an AI that will get shut down and a blackmail opportunity. A LLM is smart enough to understand this from the words and the relationship between them. But then comes the "generative" part. It will recall from its dataset situations with the same elements.
So: an AI threatened of being turned off, a blackmail opportunity... Doesn't it remind you of hundreds of sci-fi story, essays about the risks of AI, etc... Well, so does the LLM, and it will continue the story like these stories, by taking the role of the AI that will do what it can for self preservation. Adapting it to the context of the prompt.
This works against _the LLM proper,_ but not against chat applications with integrated search. For ChatGPT, you can write, "Without looking it up, tell me about the Marathon crater."
This tests self awareness. A two-year-old will answer it correctly, as will the dumbest person you know. The correct answer is "I don't know".
This works because:
1. Training sets consist of knowledge we have, and not of knowledge we don't have.
2. Commitment bias. Complaint chat models will be trained to start with "Certainly! The Marathon Crater is a geological formation", or something like that, and from there, the next most probable tokens are going to be "in Greece", "on Mars" or whatever. At this point, all tokens that are probable are also incorrect.
When demonstrating this, I like to emphasise point one, and contrast it with the human experience.
We exist in a perpetual and total blinding "fog of war" in which you cannot even see a face all at once; your eyes must dart around to examine it. Human experience is structured around _acquiring_ and _forgoing_ information, rather than _having_ information.
The recommended settings according to the Gemma team are:
temperature = 0.95
top_p = 0.95
top_k = 64
Also beware of double BOS tokens! You can run my uploaded GGUFs with the recommended chat template and settings via ollama run hf.co/unsloth/gemma-3-27b-it-GGUF:Q4_K_M
You are an expert AI assistant that explains your reasoning step by step. For each step, provide a title that describes what you're doing in that step, along with the content. Decide if you need another step or if you're ready to give the final answer. Respond in JSON format with 'title', 'content', and 'next_action' (either 'continue' or 'final_answer') keys. USE AS MANY REASONING STEPS AS POSSIBLE. AT LEAST 3. BE AWARE OF YOUR LIMITATIONS AS AN LLM AND WHAT YOU CAN AND CANNOT DO. IN YOUR REASONING, INCLUDE EXPLORATION OF ALTERNATIVE ANSWERS. CONSIDER YOU MAY BE WRONG, AND IF YOU ARE WRONG IN YOUR REASONING, WHERE IT WOULD BE. FULLY TEST ALL OTHER POSSIBILITIES. YOU CAN BE WRONG. WHEN YOU SAY YOU ARE RE-EXAMINING, ACTUALLY RE-EXAMINE, AND USE ANOTHER APPROACH TO DO SO. DO NOT JUST SAY YOU ARE RE-EXAMINING. USE AT LEAST 3 METHODS TO DERIVE THE ANSWER. USE BEST PRACTICES.
I have also been using this prompt, and while it fails on then problem above, it works better for me than OPs prompt:
Write many chains of thought for how you’d approach solving the user's question. In this scenario, more is more. You need to type out as many thoughts as possible, placing all your thoughts inside <thinking> tags.
Your thoughts are only visible to yourself, the user does not see them and they should not be considered to be part of the final response.
Consider every possible angle, recheck your work at every step, and backtrack if needed.
Remember, there are no limits in terms of how long you can think - more thinking will always lead to a better solution.
You should use your thoughts as a scratchpad, much like humans do when performing complicated math with paper and pen. Don't omit any calculation, write everything out explicitly.
When counting or maths is involved, write down an enormously verbose scratchpad containing the full calculation, count, or proof, making sure to LABEL every step of the calculation, and writing down the solution step by step.
Always remember that if you find yourself consistently getting stuck, taking a step back and reconsidering your approach is a good idea. If multiple solutions are plausible, explore each one individually, and provide multiple answers.
Always provide mathematical proofs of mathematical answers. Be as formal as possible and use LaTeX.
Don't be afraid to give obvious answers. At the very very end, after pages upon pages of deep thoughts, synthesize the final answer, inside <answer> tags.
The failure is in how you're using it. I don't mean this as a personal attack, but more to shed light on what's happening.
A lot of people use LLMs as a search engine. It makes sense - it's basically a lossy compressed database of everything its ever read, and it generates output that is statistically likely - varying degrees of likeliness depending on the temperature, as well as how many times the particular weights your prompt ends up activating.
The magic of LLMs, especially one like this that supposedly has advanced reasoning, isn't the existing knowledge in its weights. The magic is that _it knows english_. It knows english at or above a level equal to most fluent speakers, and it also can produce output that is not just a likely output, but is a logical output. It's not _just_ an output engine. It's an engine that outputs.
Asking it about nuanced details in the corpus of data it has read won't give you good output unless it read a bunch of it.
On the other hand, if you were to paste the entire documentation set to a tool it has never seen and ask it to use the tool in a way to accomplish your goals, THEN this model would be likely to produce useful output, despite the fact that it had never encountered the tool or its documentation before.
Don't treat it as a database. Treat it as a naive but intelligent intern. Provide it data, give it a task, and let it surprise you with its output.
I think about this a lot. I think my dad was more goal oriented and I’m more process oriented. I see every day spent working toward a goal as a valuable step toward it, while I think he tried to always shorten the path to reach his goals, and ended up not ever achieving them because of it.
As an example, I do car restoration as a hobby. It’s a big, big task to basically dismantle a car, fix body issues, rebuild the engine and transmission, clean up all the parts, and put it back together. Looking at the entire task outside of it, seems almost impossible to do, but I almost never think about the end of the work. I just think about the next thing I need to do.
I think marathon runners do something similar, or so I’ve heard anecdotally.
> solidify the mental model into a concrete branching of possibilities that is tightly coupled to the specific problem. Your work becomes traversal and mutation of this tree.
* There are problems that are easy for human beings but hard for current LLMs (and maybe impossible for them; no one knows). Examples include playing Wordle and predicting cellular automata (including Turing-complete ones like Rule 110). We don't fully understand why current LLMs are bad at these tasks.
* Providing an LLM with examples and step-by-step instructions in a prompt means the user is figuring out the "reasoning steps" and handing them to the LLM, instead of the LLM figuring them out by itself. We have "reasoning machines" that are intelligent but seem to be hitting fundamental limits we don't understand.
* It's unclear if better prompting and bigger models using existing attention mechanisms can achieve AGI. As a model of computation, attention is very rigid, whereas human brains are always undergoing synaptic plasticity. There may be a more flexible architecture capable of AGI, but we don't know it yet.
* For now, using current AI models requires carefully constructing long prompts with right and wrong answers for computational problems, priming the model to reply appropriately, and applying lots of external guardrails (e.g., LLMs acting as agents that review and vote on the answers of other LLMs).
* Attention seems to suffer from "goal drift," making reliability hard without all that external scaffolding.
It's encrypted (on top of Bitlocker) and local.
There's all this competition who makes the best, most articulate LLM. But the truth is that off-the-shelf 7B models can put sentences together with no problem. It's the context they're missing.
Post author here! I wrote this post five years ago. Since then, my conviction in the value of customizable software has only grown, but I've also updated my thinking in a few ways:
1) AI
AI is rapidly getting better at coding. Current AI is often bad at high-level architecture but is capable of making small local tweaks. Seems like a good fit for the kind of code you need to write a browser extension!
I'm exploring this direction; wrote more about it in "Malleable software in the age of LLMs" [1]
2) Security
Having talked to people who worked on various extension platforms including the browser extensions API, I see more clearly than I did five years ago that security is often the key bottleneck to deploying extension platforms meant for mass adoption. Anytime you want everyday computer users to be installing invasive extensions to important software from untrusted third parties, it's gonna be challenging to protect them.
That said, I still think that conversations around extensions tend to focus too much on security at the expense of all else. Customizability is important enough that it may be worth prioritizing it over security in some cases.
I also think there are many reasonable paths forward here. One is to exchange extensions with trusted parties -- e.g, coworkers or friends -- rather than installing from random people on the internet. Another might be to only build your own extensions; perhaps that'll become more viable with AI-assisted programming, although that introduces its own new security issues. And finally, I've met a few people who have smart ideas for architecting software in a way that helps resolve the core tensions; see [2] for an example.
3) Backend access as a key limitation
I've increasingly realized that the fact that browser extensions can only access client code in a fairly server-centric web means that many deep customizations are out of reach. Perhaps you can't read the data you want, or there's not a write API to do the thing you need.
While I'm optimistic about what extensions can do within the boundary of the client, this is an inherent limitation of the platform.
At Ink & Switch (the research lab I now work for), we're working towards local-first [3] software: collaborative software where the data and the code lives on your device. Among other benefits like privacy, we think this is the right foundation for more powerful extensions, since your data and the app code aren't locked away on a server.
Lisp from Nothing and Lisp System Implementation from [1].
There are quite a few open source Lisp implementations in addition to SBCL that you can peruse [2] (follow links to CL's supported by Quicklisp). If you are targeting a specific platform (JVM or embedded), then ABCL/ECL; see Clasp for using LLVM.
To reuse modules for writing your own lisp, see SICL [3]
Correct, if your BMR lowers drastically due to sustained increased insulin levels, then you're not at a deficit. However, most doctors are not trained to know that this is what is happening. Every BMR calculator will tell you that you are burning X number of calories, but the only way to verify that is with an expensive lab test that is generally reserved for athletes or extremely ill individuals. So instead, the doctor assumes that the patient must be lying about their calorie intake.
There are "symptoms" for a lowered BMR, feeling cold being chief among them as lowering blood circulation is an easy way for the body to slow down. Mental fog is can be another sign as our brain is the most calorically demanding organ in our entire body.
This model of hormonal driven weight loss/gain is fairly new and just now being studied at scale. Hopefully as time passes, doctors will become better educated and can advise their patients on how best to structure their diet to make their bodies work for them instead of against them.
I highly recommend watching some lectures by Dr Jason Fung about the issue in depth. There are a few on YouTube that are 45+ minutes in length and describe the situation more thoroughly. The TL;DW though is this: if insulin levels are high, the body cannot release stored fat. Eating raises insulin. If you eat all the time, your body will always have high insulin levels and cannot release stored fat for energy and thus will look for other ways to equalize the calories in/out equation.
The fastest way to lose weight, is to simply not eat. It sounds so simple and obvious, yet we've lost sight of the plot in our modern world.
> the cost of these synthetic datasets is very high. Nobody is sharing
There are plenty of synthetic datasets generated from GPT-4 and other models^[1]. But MS created a large one, 150B tokens. Still 2 orders of magnitude smaller than the 13T used to train GPT-4.
But in the future this will be the main way to improve models - put them to work, and filter their good stuff. Then retrain. Very expensive, but that is the cost of evolution. It took humans a very long time to create the culture and technology that underlies LLMs, it will take a similar effort to push them forward.
Human generated text was the low hanging fruit, but now that it's picked, synthetic data is the only way forward. Models generating their own experience and feedback, doing exploration, combinatorial search, learning from their interactions with humans, from games, experiments and simulations.
But if we're talking about synthetic data - then the elephant in the room is the chat logs of OpenAI. They got 180M users, assume 10K tokens/user/month, that would be 1.8B tokens per month, mostly AI written but interspersed with human replies and tool generated output. This means they can collect in less than a year about as much synthetic data as the original training set.
What if they train GPT-5 solely on synthetic data? That would simplify the copyright issues a lot, and give a 5x boost in efficiency.
Many programmers have a bias in thinking that what they're doing is easy or simple, because they truly enjoy it and, many times, have learnt it as a hobby. Their career choice have never been a masterplan on winning big, but it just happened to be so, so they feel like they're fraud.
Hobbyist programmers (who usually end up being the best ones) usually put great investment into their career by doing side projects on their free time. We're talking weekends and night of trying out stuff, hacking stuff, reading, learning, etc. It's often an invisible investment, because it's a hobby and it doesn't feel like work.
In the end, programmers don't really need to dig that far to reach a point where they're good at building stuff using enough layers of knowledge to make what they do absolutely obscure to others.
Nobody has a hobby of learning by heart the bones in the human body or the law texts on intellectual property; but programmers most likely know by heart a sizable bunch of Bash commands and their options, HTTP status codes, API interfaces, etc. It seems like highly advanced knowledge to be able to solve a health problem from a set of symptoms, but from the point of view of the layman, solving a computer problem from an error message is about as magic as it gets.
Refactoring UI is a book you can read in a couple of hours, but it helped me immensely to design my projects. It does not go to a great depth, but it's very useful for simple and quick hacking.
In case anyone's interested in running their own benchmark across many LLMs, I've built a generic harness for this at https://github.com/promptfoo/promptfoo.
I encourage people considering LLM applications to test the models on their _own data and examples_ rather than extrapolating general benchmarks.
This library supports OpenAI, Anthropic, Google, Llama and Codellama, any model on Replicate, and any model on Ollama, etc. out of the box. As an example, I wrote up an example benchmark comparing GPT model censorship with Llama models here: https://promptfoo.dev/docs/guides/llama2-uncensored-benchmar.... Hope this helps someone.
It has been written with keycloak as the auth server but should work for any proper openid connect implementation since I used the specification as a guide.
While writing games in any programming language is a very rewarding experience, I have found HTML5 Canvas and JavaScript to be very effective for writing games as absolute beginners too. If sound is needed, there is Web Audio too, all available right within the web browser.
It is quite possible to write simple games without using any external JS libraries by sticking to first principles only, e.g., drawing simple shapes with fillRect(), simple collision detection algorithms, generating simple tones using OscillatorNode, etc. Here is one such game I wrote sometime back: https://susam.net/invaders.html
Also, if the entire game is written as a single HTML file, the game becomes immediately distributable too. All one needs to do is host the HTML file somewhere or send the file to someone in order to share the game.
The distinction between code, config and data is being erased. Everything is a soup now. Data is application, configuration is code. Code is an intermediate, volatile thing that is generated on the fly and executed in the temporary lambda containers.