I think it was scalability to languages, dialects, idioms, etc. Super easy to have high quality American English with a few commands. Much harder to support any language, any syntax, any accent. The brute force optimizations just don't scale.

Modern ML and embeddings models are the discontinuity that was needed to get from "massively complex hack that can't scale" to "even more complex but principled approach that scales pretty well".