"Just look at how the Challenger disaster happened"
You mean beancounters decided to launch outside allowed parameters because delaying the launch would have been costly?
The O-ring issue was also known - and yes, it was luck that less cold rings would deform and stop any hot gas leak - and still, wasn't NASA planning to reuse the same boosters for its Ares launchers? And SLS as well?
What about having to build pieces taking into account the need to make them all across USA for political reasons as each Senator wants a share to buy votes, instead of having plants closer to the launching pads to simplify construction and assembling?
But you're right - Shuttle were the firsts of such kind of vehicles and weren't "perfect" enough. The question is why instead of improving the Shuttle designs they went back to Apollo 1.1. It looks only the Air Force improved the Shuttle design - and is keeping it secret.