Easy solution
Those who wish to leave don't have to be part of the EU. They don't have to pay anything which goes to the EU, they are banned from public places built by EU money, they can't rely on EU laws which protect them in the workplace etc.
And the rest continue has we have been for years.
Win win!