Isn't there continuity there? you're in the same body, which has to have some bearing on a consciousness.

for example, my knees are shot. part of being me is having to put up with that, which causes a sort of pattern in my experience. were I to get amnesia, those patterns would persist.