You may not be swayed by the design of sites but the majority of people are. A big mistake people make when approaching design is thinking their attitudes apply to "most people". It takes training and experience to really get a feel for how "the average person" will react to something.
The "average" person is confused by things that are new and different. Making them learn a thousand different UIs just so "designers" can show off their artistic inabilities is one of the worst crimes perpetrated by the web.
