In my high school biology class, Mr. Glover floated the theory that Deja Vu was a time-lag problem--you saw an image, but the chemical symbol of the stimulus reached your interpretive centers from one eye an instant before the other one, so that you saw it twice, and when the second signal came in, you thought, "Hey, I've been here before."
An interesting theory, but it turns out The Wayne (as we called him then--God, the wit of age fifteen) was wrong. We don't store memories as sights, sounds, smells, or sensations, we store them as chemical signals, and remembering them is the process of translating those signals back into things our brain can interpret as the original stimuli. Apparently, we occasionally lose the subtleties of those signals and get confused when the new chemical translation looks exactly like an old one.
Memories feel a certain way, even the ones that aren't particularly happy or sad. That feeling, that sort of, "huh..." is there the instant before the translation of that memory happens and the experience unfolds back into its sensory components. Deja Vu seems to occur at this same level. The titular feeling comes first, a feeling coming from the brain that says, "I'm remembering this." So while it seems like it's a fault of memory, it could just as easily be a fault of encoding the incoming stimuli--and then the brain is indeed getting the same signal from two different places--one within, and one without.