Well, it stands to reason...

If you have to surrender your keys, but not a safe combination, then if the password is in his memory, he shouldn't have to surrender it. If it's a physical object, then sure, surrender it. How hard is that?