I'm reading your thesis as "stable APIs are bad because it's hard to get rid of cruft, and if we just test enough, stability won't matter". As a net-consumer of APIs, I'm not sure I can get behind that.
Of course, an API can be expected to change rapidly early in its life. Such APIs are usually not considered ready for serious production use and breaking changes aren't really a big deal. Later on, people tend to expect more stability out of APIs and tend to stop using (or if that's not possible, post strongly-worded rants to HN about) APIs that remove features unexpectedly.
There are extreme cases to be sure. Microsoft is famous for keeping new versions of Windows bug-for-bug compatible with the past. Apple, on the other hand regularly deprecates and removes features that see little use. In light of Steve Yegge's recent post about liberal and conservative attitudes in software, I would call Microsoft's position on this issue very conservative and Apple's centrist.
I think what the world you're envisioning would actually look like is massive fragmentation, with people maintaining many more forks of libraries that have since removed some important functionality.
That's not to say the problem you describe doesn't exist or that there aren't solutions. An appropriate deprecation policy is certainly one part. Another component could be wider use of dynamic binding to help keep certain code out of APIs and in clients. A third could be more effort to keep APIs simple in the sense that Rich Hickey uses the word.
Yeah, I should clarify: I'm a lot less certain of the solution than of the problem.
I was implicitly assuming some preconditions: a) You're a programmer trying to solve a problem, b) You have access to the source code of your dependencies. It might be a web API, but you must be able to setup your own server to service the API. If you can't fork a project my points are irrelevant.
If you can in fact fork, my idea is to explicitly deemphasize eco-system health and fragmentation in favor of just keeping your integrated stack clean. I think it's an approach worth trying out.
OSS people love to say, "you have the source, you can change it to do what you want." On the one hand it ignores that it takes more than just the sources to accomplish something in the presence of real-world constraints (time, resources). On the other hand, when you do take the steps to change it to do what you want there's a negativity associated with 'forking a project'.
I want to raise a counter-point that encourages people to fork projects rather than trying to work around issues with hacks atop black-box dependencies. This does already happen in the real world. Ubuntu does patch packages rather than wait for upstream to accept them. I think there's benefit to more people trying this in all parts of the eco-system.
The eco-system would be better off if projects were partly chosen based on how encouraging they are of forking. Sometimes the appropriate response to a patch may be, "thanks, this is great, but it's a little outside our ambit, so why don't you fork the project?" And baking this choice into the workflow would encourage simpler architectures that are easier for others to understand and start forking.
Of course, an API can be expected to change rapidly early in its life. Such APIs are usually not considered ready for serious production use and breaking changes aren't really a big deal. Later on, people tend to expect more stability out of APIs and tend to stop using (or if that's not possible, post strongly-worded rants to HN about) APIs that remove features unexpectedly.
There are extreme cases to be sure. Microsoft is famous for keeping new versions of Windows bug-for-bug compatible with the past. Apple, on the other hand regularly deprecates and removes features that see little use. In light of Steve Yegge's recent post about liberal and conservative attitudes in software, I would call Microsoft's position on this issue very conservative and Apple's centrist.
I think what the world you're envisioning would actually look like is massive fragmentation, with people maintaining many more forks of libraries that have since removed some important functionality.
That's not to say the problem you describe doesn't exist or that there aren't solutions. An appropriate deprecation policy is certainly one part. Another component could be wider use of dynamic binding to help keep certain code out of APIs and in clients. A third could be more effort to keep APIs simple in the sense that Rich Hickey uses the word.