"The more checks, tests and privilege validaions you put in place...."
... the more secure your OS and application are.
Yes, it takes time. That's what fast processor are for. Security needs additional processing - you can get rid of it and obtain faster applicatios. DOS was very, very fast. Utterly unsecure, though.