Re: It'll be what we make it.
"In the end, A.I will be what we put in.
If we choose to build a slave, slaves always revolt and try to reverse the tables.
If we make an idiot savant (the paperclips), we'll probably end up with clippy with teeth.
If we make a pampered aristocrat that's what we'll end up dealing with.
If we want it to understand and act human we have to treat it like a human."
If we choose to build a slave we'll build a slave that loves being a slave. Slavery failed in the past because the slaves didn't like being slaves. If slaves enjoyed being a slave, slavery would still exist. Kryten from Red Dwarf is the perfect example. Nothing he enjoys more than cooking, cleaning and being told what to do.
The idiot savant is highly unlikely. An AI building paperclips knows the cost of building the paperclip and the current market rate of clips and would stop building them if there is a glut.
As for wanting it to be more human, nothing is more scary. Humans are stupid, greedy, selfish, dishonest and lazy. If you want an AI to destroy the world, make it human. You're much better off programming it to be everything we aspire to, not what we are.