The problem with what you're saying is: There's a lot of data on the Internet, but it's still finite. If you divide a finite number - however big - by infinity, you still get zero. To get something like GPT, you need to generalise to infinitely many possibilities using only a finite amount of data. Markov Chains are too simple an algorithm to produce these results.

I think your comment shows what's called the "AI Effect", where a Black Swan event in AI research - like AlphaZero or GPT - is dismissed as predictable purely based on hindsight - even though the results surprised most experts.