They forgot the most obvious one: your local CD or record store. Sure, Tower Records is gone and Borders and Best Buy have crappy selections, but if you live in a college town or a big city chances are you'll be able to find a good place. I never buy music online anymore.
Oh, and if you're worried about buying from the RIAA, just look on the back of the CD. It will almost always give the name of the record label.
Huge repository of midi files that has been extended to include a variety of live recordings. Quasi-free (like 5 files per month), and a $25/yr subscription gives access to pretty much everything. No advertisements at all (not counting self-promotion).
I gave a class presentation on this a couple weeks ago. I actually made the page in class because the group presenting before us had a web page so I figured we should make one too. The whole process took about fifteen minutes on Jottit, and the prof. loved it. I'm now a huge Jottit fan.
eMusic, of course. It rules. Cheap and all the best current bands (if your favorite band is not included, perhaps it's time to re-assess your taste in music).
Oh, and if you're worried about buying from the RIAA, just look on the back of the CD. It will almost always give the name of the record label.