You're missing his point about flagging. Flagging is for trolls, spam, etc. It isn't a 'downvote'. He wants downvoting to be the opposite of upvoting, meaning 'I don't like this' or 'I disagree with this'.
As Matt himself says, it is a nuance that is lost on most.
Flagging is also for things that are off-topic. If he's sick of 20 TSA stories every day, I don't think the flag is the wrong option. I personally flag things I see that are merely political.
When you use it, it moves posts down, which is exactly the opposite of upvoting! The nuance is lost on me. I don't see why we would be provided with something that acts exactly like a downvote would act, and then expected to use it differently than a downvote.
I also draw no distinction between "spam" and "a link to someone's blog about how they were patted down at the airport." I would just as soon read about free Viagra and watches.
Yes, and a Vice Grips is not a very good wrench, unless you are stuck on the side of the road and have nothing else. Is this nuance or unnecessary idealism? :)
As Matt himself says, it is a nuance that is lost on most.