I agree, there is a current technical limit in the token context but that's a limit in practice, not in theory.

There are plenty of wrapper tools around LLMs that cleverly use the token window to keep a longer "state of mind", overall strategies currently executing etc. With varying degrees of success, I should say... but still, it's kind of analogous to a human executing a strategy with intentions.