Even so, I wonder if you could get better speed for, say, Dolphin, by pausing to do as much statically as possible (perhaps going all the way through LLVM's optimizations) rather than being in a hurry to do everything dynamically without pausing.
I wanted to experiment with this idea at some point (without pausing - using a background thread that does optimization and patches code with the optimized version when it's done) but our current JITCache is just not fit for this. We perform JIT compilation per-basic block, which does not allow for a lot of optimization compared to per-function (following all direct branches as long as possible). Changing that would've required too much time and I got bored of it :(
