Dinosaurs didn't have a space program and look what happened to them.
They're still around. There's bloody billions of the things. Some of them - such as the family Corvidae - are damned near as smart as we are.
It isn't asteroids we should be worried about, it's the slow death of the Sun. It might sound stupid - the Sun is supposed to have a few billion years left - but it's not. There are maybe a billion years of habitability left on Earth. Long before the Sun expands into a red giant and blasts the atmosphere off this rock it will have been completely sterilized. Several estimates have the planet able to sustain truly complex, intelligent life for 300-400 million years at best.
In geological context, that isn't all that long. That's one more mass extinction cycle. Or one more "largeish quantities of Algea into fossil fuels" cycle, to be more precise. It's this latter that matters.
You see, the biggest issue is that fossil fuels are of finite supply. They represent the densest chemical energy source we know of and they are an absolute bitch to manufacture synthetically. Simply put: if you want to put a rocket into space fossil fuels represent the most efficient method we know of.
Once fossil fuels are gone humanity will be facing a pretty big energy crisis. How will we generate the kind of power necessary to run our ever-increasingly-power-consuming society, let alone blast payloads into orbit?
Going out into space with enough people and equipment to begin a space-faring civilization is going to take sending up a lot of material. The geopolitical realities of our species make a continent-long rail gun highly unlikely, we're hundreds (if not thousands) of years away from orbital tether technology and an energy-starved future humanity isn't going to be all that inclined to spare the power necessary for the electrolysis of $stupid quantities of hydrogen or the purification of $stupid quantities of aluminum.
(Don't say "nuclear will save us" because A) people are dumb, stupid, panicky animals and I doubt they'll ever be able to overcome the conditioning of "OMG RADIATION" so they'll cut off their nose to spite their face on that regard. B) Even if we did suddenly start to grasp basic science and accept that fission is a safe form of energy there simply isn't enough fuel to sustain our society at present technology levels matching projected demands for all that long.)
If we are ever going to get out into space with enough people and enough equipment to master the stars the time is now. Now while energy is cheap. When it can be pulled out of the ground in ready-to-use form, densely packed and easy to implement. This time will not come again for our species.
By the time there is enough fossil fuel recreated by natural processes on Earth humanity will be dead and gone and buried. Dusts of the ages and extinct a long, long time ago. Will earth birth another space-capable race in time to use that fuel? The chances aren't good...and there's really only the one more chance after us.
It is comforting to think that the Universe is teeming with life; that even our own small galaxy abounds with multiple life-bearing planets. It is comforting to believe in this because it removes from us the burden of seeding the Universe with the only known life to exist: Earth's.
Humanity will die. Earth will die. Eventually, even our Sun will die...and none of that is all that far away. Personally, I think it would be a cosmically reprehensible shame if life itself died with our planet. It would be an unimaginable tragedy if it was our own shortsightedness that meant that the only species known to have ever existed to be capable of spreading this fluke of chemistry to the stars failed to do so.
I believe that the only purpose of life is the continuation of life itself. It isn't about the continuation of our lineage, or even our species. The stakes are bigger than that. It is about ensuring that life manages to outlast the stars themselves; that the Universe is given purpose beyond mere existence by the fact that life still exists to experience it's wonders.
Our time is now. It may never come again. If Earth is, in truth, the only place in the universe where life arose - and until we have solid confirmation it exists elsewhere we must assume this to be true - then we must take advantage of what we have and act. Hie thee to the stars, earthkin; we have a duty to the universe itself to spread the seeds of experience before the brief candle of our existence is blown out.