That probably depends heavily on the field, there is certainly a huge difference whether you are trying to better understand established physics or whether you are trying to come up with a new theory in an area where we are lacking experimental data.

Well, what I mean is that (to give you perhaps an extreme example) classical mechanics, while not "lacking experimental data," has for the last 200 years been effectively considered part of (applied) mathematics (known as "analytical mechanics"). Thermodynamics, electromagnetism, quantum field theory, general relativity - they all seem to be purely mathematical now.

Then I probably misunderstood you, I thought you were talking about how closely the math is tied to the actual world, say all the reformulations of classical mechanics as opposed to some supersymmetric theory.

In the end physics is applied mathematics, once you have a theory that describes your observations, you can just study the mathematical structure and look for interesting things, only occasionally going back to the lab to verify your predictions, improving confidence that you picked the correct model.