Well yes, but I would keep the complexity down first
Every line of code is a potential security critical bug. And with Rowhammer we have seen that isolation mechanisms are often not effective.
I'd personally go for simpler systems which abstract away critical functions on dedicated hardware. That way no glitch of your RAM chip can give you a key you shouldn't have. Obviously those dedicated pieces of hardware would need to be open and ideally simple enough to be audited many times over.
@TheReg: Could you please also link to the _real_ offical video, not the copy on Youtube https://media.ccc.de/v/32c3-7352-towards_reasonably_trustworthy_x86_laptops#video There's also a way to download/torrent the high resolution files, as well as tons of other videos. Most talks which are German are translated into English, some even into Swiss German.