As a member of the encrypt-everything clique I have to say, though Moxie has a point, Gnupg is infinitely better than alternative of nothing at all. Until someone comes up with a solution that preserves its power and flexibility and marries it with a functional UX -- I shall keep using it
I'd recommend going base32 instead of base64 (case sensitivity won't screw anything up) and not padding with = characters. I know not padding violates RFC 4648 but I don't know anyone who's used to a = in their email address.
Base58 it is, then (no non-alphanumeric characters so you can double click and it will select the whole string, no similarly-looking characters you could mistype (1 and l, 0 and O...))
I like the idea, but you'd be amazed at how many databases only allow 30 character email addresses, or javascript email validators that don't recognize "+" as a legal character.