For me, personally, I recalled things whenever there were trigger words or things told to me about them.

[ Thanks, Enoshima... ]

Here, though, the memory tree does trigger some of my memories back, but sometimes I have dreams about them too, which does it. I haven't experienced anything like that, but that might be because how I lost my memories is different than other people's.