100% agree. When we explicitly segment and compose AI components, we are removing the ability for them to learn their own pathways between the components. We've been proven time and time again the bitter lesson[1]: that throwing a ton of data and compute at a model yields better results than what we could come up with.
That said, we can still isolate and modify parts of a network, and combine models trained for different tasks. But you need to break things down into components after the fact, instead of beforehand, in order to get the benefits of learning via scale of data + compute.
That said, we can still isolate and modify parts of a network, and combine models trained for different tasks. But you need to break things down into components after the fact, instead of beforehand, in order to get the benefits of learning via scale of data + compute.
[1]: http://www.incompleteideas.net/IncIdeas/BitterLesson.html