did GPT-4 just solve the halting problem?

As soon as you add a 2nd layer of loops however, you reach Turing-completeness and suddenly the halting problem becomes unsolvable.

-------

So you don't need to deny jmp/loops. You just need to deny _nested_ loops. And... find that old paper I read like 15 years ago to figure out the details to discriminate halt/not-halt in the single-layer loop language.

    while (state !== STOP):
       read = TAPE[index]
       [state, write, dir] = TABLE[read, state]
       TAPE[index] = write
       index = index + dir

E.g. this Ruby program is undecidable:

    gets

    while getsWithTimeout != "halt"
    end

https://vbpf.github.io/assets/prevail-paper.pdf

For (;;) { print(“hahaha you didn’t say the magic word!”); }

In another loop would also prevent termination without program shutdowns.

To clarify, when I said "for loop", I meant what is sometimes called a "counted for loop" (or simply "counted loop"): there is an (maximum, if you allow early exit) iteration count that is computed before executing the loop and can't increase later.

In C syntax, it is for (int i = 0, e = ...; i < e; ++i) { ... } and the body of the loop is not allowed to change the value of either i or e.

Edit: actually I may have been unclear. When I said "for loops are guaranteed to terminate", in the context of the discussion, I meant "if the only kind of loops you allow are (counted) for loops in a language where loop-free expressions are guaranteed to terminate, you get a language where all expressions are guarantee to terminate". So loops can contain other loops, as long as they are all of the "counted for loop" kind.

The Halting Problem is a truly interesting result, and for the most part uninteresting in practice.

Say your Turing machine is searching for a contradiction in ZFC. It enumerates all the possible proofs, and checks if they are valid and they prove 0=1. You can prove in ZFC that you can't prove that it halts, nor you can prove that it doesn't halt.

Now your Turing machine won't halt, according to ZFC. (It can halt in practice, if ZFC has a contradiction.) Same for any other undecidable programs.

GGP was talking about programs halting, not halting, and "something else", which GGP called "undecidable".

There are programs that ZFC cannot prove if they halt or not. For example, searching for a contradiction in ZFS is such a program. Its undecidability means that there is no sequence of axioms of ZFC that ends with "this program halts" or "this program does not halt".

But then this program does not halt, since its halting would mean that ZFC can prove that this program halts.

In any case, programs can halt or not halt. There are programs that ZFC cannot prove that they halt or not, but those programs do not halt.

That step is not logically consistent. It could halt, it's just that ZFC can't prove it will ever do so. For example, the program which computes the 8000th busy beaver number halts (per definition of the busy beaver function), but is undecidable in ZFC[1].

An undecidable program can halt. An undecidable program can run forever. But whatever axiom system is being used to decide that can't prove which (without running the program, potentially for an infinite number of steps).

[1] https://scottaaronson.blog/?p=2725

I don't believe that there exist such a program.

What I don't believe that there exist a program that is a proper counter-example, that is, its halting is undecidable in ZFC, and it halts. Exactly because what I wrote earlier.

The problem is that you dont know if the checker that you use to detect if a program halts will itself halt on your given input or continue forever. But given a sound checker with some assumption, one can find non-halting programs which wont be detected by the checker using the diagonalization trick.

while (isFamousMathProblemWeDontKnowWhetherItHoldsForAllTheIntegers(n)) { n++ }

    if(going_to_halt)
        dont();

*: Trivial in a mathematician's sense

And obviously trivial to avoid recursion too.

Write a program that, for a positive integer, runs the Collatz process on it (if even: halve it; if odd, multiply by three and add one; repeat).

If the process results in a 1, move on to the next number and repeat.

If the process produces a number it produced previously, halt. (Worried you’ll need unbounded state for this part? Use tortoise+hare, it’s fine).

This program halts if and only if the Collatz conjecture is false.

Now, the Collatz conjecture might not be unprovable. But for now no human can tell you whether or not that program halts.

I ended up using it to compare single core performance on any windows machine, because the timestamped logging was deterministic. Rewrote it in python and still use it these days.

For example, it’s trivial to write a program that searches by brute force for a cycle that would disprove the Collatz conjecture, and halts if it finds one. No human knows whether that program will halt.

IMO, in biological systems the explore vs exploit tradeoff is pretty analogous to the halting problem. There doesn’t seem to be an optimal general “solution” to it.

Once an organism is very familiar with its environment it’ll approximate near-optimal trade offs, which would suggest it’s just using heuristics.

That’s a common misconception. The brain is not like a computer. The brain can’t store and execute programs.

Can't it? The brain is absolutely capable of emulating a turing complete system such as running a program.

I mean, sure you can reason about a portion of code on your screen but there is no way you could emulate anything without some visual support.

There is no way your short term memory could store the program and the variables. The human brain can barely remember 5 to 10 words for several seconds and you have no control on your long term memory.

So yes your brain can somehow emulate a computer if you give him a pen, a sheet of paper paper, time, and a lot of sugar. But that’s not because it’s functioning like a computer but rather because you learnt how a computer work.

https://github.com/MineDojo/Voyager/blob/main/voyager/contro...

This function keeps a global count of how many times it's been called to mine a block that doesn't exist nearby, and will warn the bot that it should try exploring first instead.

There's nothing necessarily wrong with all that - it's an important research question to understand how much hand-holding the agent needs to be able to do these sorts of tasks. But readers should be aware that it's hardly dropping an agent into the world and leaving it to its own devices.