Re: Almost unsolvable problem.
Even if you have all the code, and reviewed all the code, you are still compiling it with a compiler in binary form.
In the real world, this is not actually seriously a problem. It can be defeated in theory....
Countering "Trusting Trust"
...and I actually don't think a compiler exists that has enough swiss army knife functionality to look out for a few dozens programs just to put backdoors into the crypto parts unseen. Ken Thomson's initial idea was to finagle the lowly "login" program, which sounds feasible. Finagling GPG etc. via that method sounds like it needs a AI module in the package.