1: It's not ours
I fail to understand this point. None of the oil in the middle east is ours either. Until we buy it. Why would we expect to get electricity for free?
2: there's a lot of potential demand nearby
Not so much that demand would outstrip supply. The actual land required to provide sufficient solar energy for the entire world is actually very small.
3: The engineering of the transmission system alone would be the largest civil engineering project ever attempted by at least an order of magnitude
Most of it already exists. Read up on the Synchronous grid of Continental Europe.
And I guess what you're thinking about is the European Super Grid.