Learning and Reasoning with Visual Correspondence in Time