The word you're looking for is propellant. In chemical rockets, the fuel and propellant are the same thing (to be precise, the combustion byproducts of the former comprise the latter). In nuclear fusion and ion thrusters, they're different. In all cases, Newton's second law means that you are using up propellant to generate an equal and opposite reaction in the direction you want to go, and thus the amount of "going" (delta-v) you can do is limited.

"Propellant" - Thank you! I knew it wasn't technically fuel, but it's still being expelled (as you mention, if you don't expel mass with a different velocity relative to the rocket, then you won't be able to accelerate a rocket)

Yah, I should have deleted my comment rather than editing it.