We've basically been in a continuous bull market over the past 11 years; it's not a representative sample. Nor is any period of 10-20 years nearly long enough to tell you much about long term stock market returns. Plus, valuations (ie P/E or P/B) are significantly inflated currently compared to the past. Since valuations can't inflate forever, future market gains over the long term are expected to be lower than past.
In addition, we're talking about a real withdrawal rate; a 4% real rate of withdrawal will be approximately a 6% nominal rate assuming inflation sticks around 2%. It's very unlikely you're going to maintain that from a balanced portfolio over the long term without depleting principle at all. Might be possible with an all-stock portfolio if you get lucky, but significant chance of failure if you get a poor sequence of returns.
In addition, we're talking about a real withdrawal rate; a 4% real rate of withdrawal will be approximately a 6% nominal rate assuming inflation sticks around 2%. It's very unlikely you're going to maintain that from a balanced portfolio over the long term without depleting principle at all. Might be possible with an all-stock portfolio if you get lucky, but significant chance of failure if you get a poor sequence of returns.