I assume you're talking about solar-powered ion engines, as solar panels cannot on their own move a spacecraft appreciably.

The big difference there is thrust. Ion engines are efficient but have very low thrust, so it can take months to change your orbit. Chemical and nuclear propulsion is high thrust, so we're talking minutes instead. Nuclear has higher propellant (i.e. mass) efficiency than chemical as well.