Let him go on good terms. If he stays with the understanding he can do various side projects on company time you are both compromising. You get half a developer and he still has to do tasks he doesn't want to do. This is a recipe for resentment.

In my experience, every time someone leaves another person blossoms and steps up. I've seen "irreplaceable" developers come and go, they were all replaced.