But isn't the neural network (like the algorithm described in the article) a mathematical concept? Are the computer-based neural networks so closely modeled after biological equivalents that they would inherit such an optimization?
(As you can tell, I know nearly nothing about neural networks, but curious to learn more...)
Digital circuits? I mean, it is just some matrix multiplies and a nonlinearity (then stack to the moon). No "circuitry" is really involved until you get into recurrent networks, and even then that is just feedback. Not quite sure what you mean here.
There have been experiments trying to encode information the way the brain does, they just haven't worked very well (or at least not as well).
There is equivalency with PCA (well ZCA, a modified form of PCA) in cat and monkey brains, and likely others as well. See Sejnowski and Bell in [1]
Also, PCA is an affine transform, so there is no reason it couldn't be incorporated/learned by the net itself. In fact, I think most nets these days eschew PCA/ZCA when they have sufficient data support.
To clarify for others, this type of neural network has nothing to do with the brain. A neural network is really a "universal function approximator", and I actually prefer to call it as such. Our goal is to learn the best possible mapping of input -> label, through whatever means necessary. It turns out that learning hierarchies of features helps from both a learning aspect and a computational point of view. But a sufficiently wide single layer could do the same thing in theory.
I have no idea. But I'd wager that cause and effect in natural systems has more to do with visual systems adapting to shapes than shapes of entire animals adapting to other species' visual systems.
But isn't the neural network (like the algorithm described in the article) a mathematical concept?
It is. It's a family of models that (a) can be expressed compactly using linear algebra and (b) can represent a large class of mathematical functions. (In fact, the family of neural nets can learn all continuous functions.) "Learning" is typically some variety of gradient descent (on the "error surface" defined by the training set) in the space of parameters.
Are the computer-based neural networks so closely modeled after biological equivalents that they would inherit such an optimization?
In my opinion, no. Biological neural networks are, in fact, a lot more complicated. They have time behavior, changing network topologies, and chemical influences (neurotransmitters) that play a major role. There's still a lot we don't know about them.
I disagree with GP's contention. While the biological neural network was an inspiration for this class of mathematical models, and convolutional behavior (in ANNs, a regularization, or mechanism for reducing the dimensionality of the parameter space, sacrificing training-set performance but often improving test-set performance) may be used in our visual cortex, but artificial neural nets are quite different and, mathematically, most varieties are quite simple.
(As you can tell, I know nearly nothing about neural networks, but curious to learn more...)