I remember reading in Scientific American (wow, decades ago) an early article on A.I. using a, then new, trained neural network. There was a small comment at the end of the article where the researcher noted that as they "killed" (destroyed somehow?) the neural network, they started seeing early patterns that it had trained on output.
That sounds a lot like the way HAL-9000 regresses to its earliest programming (reciting its "birth" date; singing an old song) when Dave is shutting it down at the end of 2001: A Space Odyssey.
