If time is infinite, the probability of exactly the same collection of atoms coming together in another you rises. If time is finite, then not so much.
Really the problem is mistaking the conservation of mass/energy with the conservation of an object. If I take a log and burn it, clearly the log is destroyed, while at the same time not violating any principles. Now if I ask "will that log, in its exact, pre-burnt form, appear again?" I think then people will look at me as being daft, more so than they already do…