Worth considering the use of an 'active-active' system where two duplicated systems are both active with delta's applied between them to keep the databases in sync,
Quite common in the payments industry.
If you lose one system at least 50% of your terminals/access points are still working and your database is fully intact and up-to-date without needing any manual intervention.
You can then manually (or automatically) swap the remaining connections to the still running site.
You will still need well documented procedures and processes but it can take some of the panic out of recovering,