Feeds

back to article How many apps does it take to back up your data?

What is the better approach to backups: a single service that can back up everything on your network or a collection of applications for backing up different items? Over the course of my career I have been on both sides of this argument and I am still not convinced either is right. Now a pending network upgrade has forced me to …

COMMENTS

This topic is closed for new posts.

Speaking as a humble home user

So take this fwiw.

I split backup into two main areas. There is backup of the system (os) and there is backup of data. I always put my system on C: and Data on D: drives. That I have done for years, even on old laptops with one small drive.

System backup is achieve by imaging with programs such as True Image, Paragon, Macrium Reflect, DriveImage XML, and my current favourite - an old copy os Easus ToDo (3.5) - it is much better than version 4. Sad to see them stopping the free versions, but this seems to keep on working. It's fast, has the features I need and is proven to restore systems via bootable Live CD. On more than one occasion.

Data backup is achieved either by imaging again, if it is what I call a milestone event - say getting ready to put in a new hd or when the system reaches end of life. But this has drawbacks for data - much better to just do a standard copy - this way there is no need for requisite installed programs to 'read' the data, so it travels well, and there is no need to take the time to mount a disk.

Lately I have been getting into virtualisation, and though it does take a while to get your head around, it has obvious benefits relating to backing up. Backing up the Virtual Machines themselves though is a bit of a nightmare I have found, at least with my VM of choice - VirtualBox, but it is possible to grow and shrink disks and copy them, as well as clone them, with a bit of effort. This is one area, even as a humble home user that is going to be getting a lot more attention from me. VMs are the way forward.

So, backing up the system is fairly trivial, and as long as you are dilligent with the data side of things, there should not be too many tears in a crisis. I don't have an automated or scripted plan. This is sorely lacking and irresponsible on my part. The data that I do is priceless. In a day or two between backing up (or weeks sometimes), I generate a massive amount of unique irreplaceable data - artwork and music. Stuff that once gone is gone forever. I have a strange relationship with it - I love my work as some love children. Yet I risk losing significant amounts of it by not having a system in place. I know this risk and accept it. I also plan to change it when my head is not so frazzled with it all and I can devote some time to a strategy that works for me.

I also am too poor to be able to afford hard disks to back the stuff up. I have stuff backed up to just one disk, which is not a backup by any means. I have a lot of stuff mirrored on two disks, but not everything. This I aim to rectify soon as well. I have spent far too much money on software and supporting developers - very often giving donations where possible.

I am not as irresponsible and reckless as a lot of people that I do computer stuff for though. I mention backup to them and they just see it as another thing they have to shell out for or spend time on. I explain to them in the simplest terms, but to no avail. I rarely labor the point these days with them.

Anyway, just a little insight into the habits of a home user.

3
0
Silver badge

Re: Speaking as a humble home user

"There are two types of people:

Those that make backups.

And those that have yet to lose irreplaceable data."

"You don't convince family members to take periodic backups. Repeated, tragic data loss convinces family members to take periodic backups. Same as everyone else."

Anyway, when I bought the hard disk for my current system, I bought two. One went into the system, and one gets stuck into the external SATA toaster dock on the 1st & 15th of the month. I boot to single-user and run a grub command and an rsync script. (and yes, my VirtualBox vdi files take FOREVER)

Things go casters-up, and I can slot the backup drive into just about anything that takes SATA and Linux will boot. I've tested it too. It's gotten more complicated since OS is on SDD and my user dirs are on rust, but it still fits on the backup drive, and I can do w/o a profile for root until I edit fstab.

For my Android phone, I wrote an app that runs rsync on /data/data and /sdcard/Android/data to my Linux box. Minimal UI with "GO" & "EXIT" buttons, and the output of the rsync script.

0
0

Re: Speaking as a humble home user

Yeah, it's true what you say.

They think they lost data, but had me come along and rescue it.

They think next time will be the same.

I tried to explain that there may come a time when I won't be able to help.

I pray they never learn the lesson. Because everyone has some precious data to lose, that would impact their life.

So many sailing so close to the wind. I know the risks and accept them, as brutal as they are. They have been told the risks and refuse to accept them. Some difference.

1
0
Silver badge

Re: Speaking as a humble home user

backup can mean many things to many people. Backup for loss of data, backup for accidental loss of data, backup for fault tolerance, backup for archive.

Each scenario needs a different approach. Mirror your SAN and you have a "backup". Plug a 2TB USB drive and dedicate to windows backup and you have a rough and ready (bare metal if necessary!) backup (I actually moved a physical to hyper-v this way just to see if it worked and it did!). Since they are effectively free there isn't much of an excuse not to with small setups (in addition to normal backups). Backup Assist is a nice cheap backup program that works well for small businesses (will backup exchange mailboxes and sql servers plus works bare metal). Hell, script an xcopy bat to a mapped NAS drive and that can be a backup.

No one size fits all, depends entirely what you want.

1
0

Re: Speaking as a humble home user

As another home user, I run a similar setup: disk images of the OS drive, and file copies for data run periodically.

However, having been stung by data becoming corrupted without my knowledge, and then naturally having this corrupted data being itself backed up, I had to come up with something a bit stronger.

My current data backup system, lashed together by scripts, is:

1) Copy all files to be backed up to the backup disk (a USB attached hard disk) into a Mirror folder. I use FreeFileSync for this so I can set up inclusion and exclusion rules, and it can use VSS to copy files that are in use.

2) Create a text file containing the ACLS of all the backed up files, in that Mirror folder (optional, but in my case, some of the ACL permissions are important and would be a pain to reconstruct). Couldn't find a decent tool for this, had to write one.

3) Create a checksum file containing the checksums of everything in the Mirror folder (md5deep can do this)

4) Create a new timestamped history folder. For every file in Mirror, check the previous history folder (created last time the backup was run) for the same file. If the same file, with the same modified date, exists there, then create a hardlink to it in the new history folder. If the file isn't there, or has a different modified date, copy the file from Mirror into the new history folder. (A tool called ln <http://schinagl.priv.at/nt/ln/ln.html> can do this in Delorean Copy mode)

5) Check the checksums file against the files in the new history folder. Any mismatches indicate that a file has changed content without changing modified date, and therefore an indication of possible corruption. This is reported, then there are three copies to check by hand - the one in the history folder from previous backups, the one in the mirror folder from the current backup, and the one on the live installation that was backed up. At least one of them is probably corrupted, but it should be possible to find and restore one that isn't.

Before doing any restore, the checksums can again be verified to guard against the backed up data having become corrupted since it was backed up (only the current versions of backed up data are verified as part of the normal backup procedure).

Of course, the disadvantage of this is that it is slow, as it always has to copy all the data. However, if you don't actually copy the data, and only assume that it's still the same because it isn't supposed to have been modified, how would you know?

I have, every so often, looked to see if I could find backup software that would do the job in a less home-brew fashion, but nothing I've found yet can do the job.

1
0

Re: Speaking as a humble home user

@AlexV: "Of course, the disadvantage of this is that it is slow, as it always has to copy all the data. However, if you don't actually copy the data, and only assume that it's still the same because it isn't supposed to have been modified, how would you know?"

You could use a program such as rsync, which will (by default) checksum files to determine if contents have changed and delta-copy the differences to the destination, so this protects against bit-rot on the destination side, but your checksum datafile would help you find bit-rot on source side. Of course, you could just use a checksumming filesystem such as zfs or btrfs and not have to worry about it in the first place...

1
0
Anonymous Coward

"........continuous data protection a far more realistic goal. The idea is that every bit written by a system would be backed up in (or nearly) real time."

Continuous backups don't help if you need specific, timed snapshots as at end-of-month, end-of-quarter, end-of-year. In these types of backup, you want to stop transaction flow and data updates so that some set of financial and other databases can be copied as a synchronized set. Does the continuous backup software (or service) actually allow for this?

0
0
Gold badge

Usually, yes.

2
0

Heck, Windows has had Volume Shadow Copy built in for well over a decade now, so the software doesn't even need to know, just ask the OS to manage it.

It's a similar situation for POSIX, just ask the OS to manage it for you and copy the blocks you are fed.

0
0
K
Bronze badge

The problem here is

Showing the Finance dept. the price tag of a backup solution the usual response is - "how much?! fugging' ell!". What they fail to appreciate is its about the only software you need to be 100% effective.. hence its worth investing in.

I recently ran into a problem, I had £12k to buy enough licensing to backup 10 ESX hosts at 2 different sites (with 2way replication for DR) and 8TB of backup storage at each.. Upon speaking to Veeam, they wanted £10k-12k alone for just the software - I was pretty much screwed as there was no way I could deliver a workable backup solution. Fortunately Dell and Unitrends came to the rescue, I can't disclose actual costs, but suffice to say it was their end of year and I got 2xDR4100's each with 9TB, and Unitrends licensing to cover all the servers and it came within budget..

0
0
Bronze badge

Re: The problem here is

Sack the finance department.

A good finance department is a little like Loren Dean's character in Apollo 13 where he is talking about remaining power saying "I'm telling you what we have." In other words, cash is a scarce resource and those guys and girls should be trying to get together whatever management decides it needs. If the answer is "You can't spend that because management believes the cash is better spent on X" then the finance department is doing a better job. Saying "If you can convince management to do this we'll arrange the finance" is actually the very purpose it exists. An FD should be making sure the business has all the finance it requires, when it requires it. (S)he should be part of the decision making process, but not the arbiter. How can an accountant know the relative merits of a backup system or a marketing campaign if both want the same cash? They can't, but management can discuss it and which one should be done. If they call backup but lose the work, that's the risk they take. If they do the marketing and suffer a fatal data loss event, that's the risk they take. They are in charge, they take the risk.

TL;DR - finance departments should not make spending decisions, management should.

1
0
Silver badge

Re: The problem here is

I sum it up: the finance dept. should NEVER be running your company.

Ask Compaq how that worked out when the CFO took over the company.

Oh wait, they're gone.

1
0
K
Bronze badge

Re: The problem here is

Naive - Sure you can ruffle feathers and get the IT project approved over a marketing campaign, but what if a company cannot function without both and there simply is not budget to completely fund them - it requires a compromise.

Also, I make the decisions on what we spend - but at the end of the day the Finance team balance the books, as such have a major say on what budget is available.

0
0
Gold badge

Re: The problem here is

Then it becomes a question of risk tolerance; yours and theirs...and how good your contract is for when it inevitably explodes. Where does the finger of blame point is ultimately the question.

0
0
Silver badge

Re: the Finance team [..] have a major say on what budget is available

That they do, what people are saying here is that they should not have a say on HOW it is used.

There are enough examples of what happens to a company when accountants take over that this should not even be a worthy discussion any more.

0
0
K
Bronze badge

Re: The problem here is

Absolutely agree. Risk needs to be clearly spelled out and to ensure everybody understands them. Can't do any more than that :)

0
0
Anonymous Coward

Oh boy... I've had to do one (just one) rather large restore, from a combination of tape and disk. It still haunts me in my dreams from time to time, and it's been over 3 years ago. Issue : huge power failure. UPS's went titsup, SAN as well. It took us 72 hours to get everything (AD, a couple of Exchange servers, SQL servers, numerous application servers, file servers, iSeries,...) back online. Most services (not data though) were restored within 8 hours though. And most data was restored by talking softly to the SAN when needed, and giving it a kick when it wouldn't listen. Nerve-wrecking (check spelling Trevor !) indeed.

When afterwards we went to management to discuss RPO/RTO they didn't really care that much anymore. It was fixed.

Nowadays we use Veeam to handle Windows servers (we're on vSphere for 90% of servers) with replication to another site, and tape backup to a remote site for iSeries. RPO is still 24 hours though, RTO best effort. Selling decent backup procedure/solutions to management is bloody hard, even when you've looked disaster in the face and kicked its ass.

0
0
Silver badge
Boffin

Eggs in one basket

"What happens if the only guy in the company who knows how to make the thing work gets hit by a bus?"

Most often the guy who knows how everything works gets fired or quits in disgust.

I've found a 3 point policy works best for backups.

1. keep it simple

2. do it religiously

3. test often

2
0
Silver badge

Re: Eggs in one basket

documentation is usually the first casualty in a department, closely followed by testing. Possibly two of the most vital components in a disaster recovery plan.

1
0
Silver badge

That's because managers are most often not interested in making sure a solution works, they just want to be able to say that they have a solution.

Until everything goes pear-shaped that is, then it is always the grunt's fault that nothing worked and he's the one getting fired.

0
0
Anonymous Coward

Reconstruct, don't just restore

Your backup administrator fundamentally needs to decide whether they are backing up state or reconstruct-ability. There are many tools to backup state, but they tend to evolve into fragile silos: you have your database backup, say, and then your web server farm, each has its own backup tools and saved-backup format, and you can't say that service is back until all parts of all silos have been restored. And thrice woe unto you if you missed out something that should have gone into your silo at the time of the last backup. (You mean the special sauce config wasn't in the Registry? And the app doesn't start without it? Err.....)

Reconstructability is a different approach. Supposing your systems died and your backup tapes vaporized. How would you rebuild everything? Maybe you would extract data from a payroll system, or your web designer's CMS, massage it, and then squirt it into a database. These are the steps that need to be documented and automated. How will you rebuild the database in the first place? Maybe you'll use Chef or cfengine or somesuch. The point is not to attempt to capture state but to actually recreate what you need to get up and running.

There probably isn't a 100% correct answer for all scenarios, but over the years I have become more convinced that reconstructability is much more important than state backup, and has many positive benefits outside of backup/recovery besides.

2
0
Gold badge

Re: Reconstruct, don't just restore

Down that road lies Puppet. "Infrastructure as code."

0
0
Bronze badge

Management don't like IT in the loop

I was an IT Admin for a small company in London. They had a NT server with a poorly maintained RAID5 array and tape backup system that stank (when I joined that is). I nursed it along and tried several times bringing up the subject. I was never invited to management meetings. One day one of the disks died.

I thought we was OK because the tapes were there if another died during a rebuild. Spent all night fixing the array. I then spent some time checking the tape backups only to find most did not even verify and there was a problem with the backup regime itself. The entire company was hanging over a cliff holding on by the skin of its teeth. If any other disk had died during the rebuild it would have been gone.

I put all of this before management in written form. It was ignored.

Several months later due to poor health and stress I was made redundant. My last words leaving that company was 'sort your IT out'.

I tried searching for the name recently. Either they changed it or went under.

Management can be pretty thick when it comes to IT needs.

0
0

Depends on size and usage

For small businesses without 24 hour operation - full image backup overnight (or at weekend) is probably the best option as it is certain to have a consistent state of all applications.

For small to medium businesses with 24 hour operation but small number of servers, full image backup using one of the disk snapshot products is probably the best - note there must be sufficient spare disk capacity to handle the requirements of the snapshot software. Depending on the business there might also be a need for transaction data to be copied offsite if loss of data since the last backup cannot be tolerated or recovered.

For large businesses with a number of servers that need to be kept in step - this is where the problems occur - there is no "one size fits all" solution. Full mirroring at a remote site MAY be possible but expensive. (Dedicated fibre links between the sites would be required along with duplicate disk arrays etc.) If the site is big enough then having the DR system onsite but far enough away to be safe may be possible (but again expensive). Trying to get a consistent snapshot across multiple active servers tends to be DIFFICULT (understatement). If a person (or team) is given the job of designing the backup regime in this case then the first thing to do is to try to find out what data is fairly static and what changes frequently. Then find out from the business teams how important consistency is - for some data it will be critical - for others not so important.. Decisions need to be made on what should be done if the backup system fails (e.g. Mr JCB digs up the offsite fibre link).

If backing up to "the cloud" is considered then legal problems may occur unless the data is heavily encrypted due to the collision between the EU data protection directive and the US government slurping up the data that is sent over the internet or stored on a US cloud server. Cloud storage is also expensive when the network costs are added to the storage costs. (Just to backup or restore a single full 1TB disk in a 10 hour overnight backup window requires an uncontended 230Mbit/sec link !!!)

0
0

Product placement much...

Trevor - how much did Asigra pay for the skyscraper ads on this page?

0
0
Gold badge

Re: Product placement much...

No idea; you'd have to ask sales. I suspect a pretty penny was exchanged, but then, there's nothing for free, eh?

0
0

I do not believe that a single backup application is the best because it does not not know the specific gritty application details however a general purpose backup application is great for all of those miscellaneous applications. Use an OS specific backup app to backup the OS because there seems to be hidden files and files with extra "this file cannot be read". Database have to have their own backup software to ensure transactionally correct backups.

Oh yea, also make a separate small "in case of emergency" backup which can be used to create a minimal system to start the recovery process.

A multiapp method is way more work but the single applications do not seem to have the documentation that I would like see and understand that the application provider has thoroughly tested the app. For example try to get a list of files which the backup application does not backup.

0
0

If you are using only one app

Don't use backup exec 2010! I was platespinning our ISA box to a Xenserver VM when I got distracted and uninstalled ISA from the live VM. Thought ah tits but oh well it was backup up last night. Did a full recovery and half the ISA files were missing! Not one error/ notification on BE2010. Luckily could export some config xmls and got it back in 90mins. Before I left the CFO must of been high and asked for SAN>SAN rep at our other site. Other guys in the biz I know have not been so lucky. Especially after getting a new Farm of DL380s after 2 years as the last twat bought ones that were gimped to 32GB ram per host!

0
0
This topic is closed for new posts.