Re: But we do already know that it annihilates when it comes into contact with matter.
Yes. And given this fact, you also need to conclude that matter and anti-matter didn't come into contact with each other, at the moment of their creation.
Because this is the logical implication of this known fact and the fact that matter and anti-matter are normally created in equal proportions according to physics theories and equations.
Which means that the Big Bang explosion must've happened in such a way that matter and anti-matter flew in opposite directions with opposite spins. Because that's the only way they could avoid coming into contact with each other.
I'd say this one known fact tells you a lot about how the Big Bang explosion must've happened.