Maybe it is because I've been programming in for something like 24 years, but I don't find them annoying to type at all; Also, maybe I'm just slow, but my typing speed is hardly the bottleneck for me in writing code; I doubt it would be even if I typed at like 20 wpm (about 95 wpm for me, and I don't touchtype). And then lastly, you can always remap your keyboard to make {} non-shift characters and/or use a language parsing IDE/editor that auto-inserts braces.
The common user don't really uses them, therefore they aren't even mapped in eu keyboards. Clearly the solution is to change the mapping, but as you can imagine, many people still need to look at the keyboard when typing... So how do you fix that?
I can teach my wife to code python, but if she has to type alt-gr 123 for every curly brace in JS, she will be tired quite soon.