I believe that happiness wears out in the effort made to recapture it; that nothing is more fatal to happiness than the remembrance of happiness.
~ André Gide, The Immoralist
If the “effort made to recapture it” involves, say, rearranging the furniture, wearing the same clothes, and repeating the same words and gestures, then, yes. But to merely remember a happy moment takes no effort at all. Joys are like children: you don’t wear away your affection by thinking of them or love one less for having another.