He can make those assumptions about time becasue many of the elements that we are made of and that contribute to our existence come from other galaxies and parts of the universe far from this one. There is a cosmic dust surrounding us that isn't from here and it's assumed it comes the collapses of stars.
Some day when our star burns out, if our galaxy has collapsed on itself by then, our DNA and everything that made us possible will be floating around space. Perhaps, some of it will land on a planet capable of sustaining life and our molecules will take hold and start anew.
It could take millions of years of floating around before we find a place though.