You're only supposed to testify as to things about which you have first hand knowledge, and you aren't supposed to speculate. So if you don't remember something clearly you should not say it.
How does that relate to the privileged communication stuff? It seems like there's a lot he recalls, but not outside privileged communication, so he says "I don't recall?"