Well... let's not rewrite history shall we?
Wernher von Braun was a commissioned SS officer and a member of the Nazi party, so when you say "but was not in any sense a Nazi" then you're just plain wrong.
Whether he believed in the SS or Nazi party aims is a different matter, even if he didn't believe in the politicts at the very least he colaborated, he didn't just "go along", his work or the V2 was critical to it's success, when the bombs slamed into the cities of London and Paris did he cheer or hold his head in his hands and cry "what have I done?", his work is no more morally sound than the bombing of Dresden (tartgetting civillian targets).
>>von Braun found himself jailed for a period for talking too openly about his real desires for his rockets.
As I said, lets not rewrite history, in March 1944 von Braun was arrested by the Gestapo with the "alleged" crime that his main interest was space travel, the second accusation was that he was going to escape with the plans for the A4 (became the V2), in "The Rocket and the Reich" Michael Neufeld implies that the accusations were made up by Himler because von Braun refused to leave the german army rocket program and work for the SS instead, von Braun in fact had no plans to defect and had an opportunity to stop working on these weapons of destruction targetted at civilians (which he didn't take).
Remember (please do not ignore the fact) that he used slave labour at Mittelwerk (the Nazi underground rocket facility), Arthur Rudolph (who also worked on Saturn V) had to leave the US, and in fact gave up his US citizenship rather than face war crimes for similar actions, hundreds of people died there.
The V2 rocket program was built on suffering, to cause suffering
The US space program was built on the V2 rocket program
You can't really blame the man for doing what he did, he was very smart and made huge technical advantages, I'm sure that building killing machines and then changing sides to work for the US is all about following his dreams and staying alive in comfort, however, if he tried to keep people alive from concentration camps (Dora?) and sabotaged the killing machines then he would be a hero, Schindler had the same choice, but Schindler chose a much braver path and was a hero, von Braun only cared about the technology and himself. Oppenheimer struggled with the fact that he was building killing machines ("I am become death, the destroyer of worlds") what did von Braun ever do or say? if he is guilty of anything it's inaction and no remorse.