We may not have seen his encounter with the "future Federation" reminding him of the "temporal prime directive"
That said, can you remember your work mates from half of a life ago?
Human memory is good, but a life of adventure and an over abundance of scotch can give one "Swiss cheese memory" ala quantum leap