Wednesday, 5 January 2011

The Lesson to Be Learned Is....

What is the lesson to be drawn from the following story?

Our main backup system is an EMC Avamar backup to disk solution. Instead of writing our backups to tape, we store them all on a large central disk solution. Net benefit, instead of backing up a select critical group of servers to 13 different tape systems, that would take over 24 hours to complete, we now backup all of our servers to one system in 4 hours or less.

However.

The Avamar system is not sized large enough to meet our long term archive requirements. So we still ship the data to tape once each month for long term retention needs. Here-in begins the source of my troubles.

Problem: Our 9 year old tape library finally gave up the ghost and died completely.
Solution: Purchase a new tape library.
Time to completion: 4 weeks (1 Month-end backup cycle missed)

Problem: New tape library will not complete a backup, dies after 4-6 hours into backup job. Can only do 1 troubleshooting test per day. After multiple failures, suspect old (unsupported) version of backup software on server.
Solution: Obtain latest version of software and renew support agreement. Do a fresh install on newer hardware.
Time to completion: 3 weeks (2 Month-end backup cycles missed)

Problem: New tape library still will not complete a backup, dies after 4-6 hours into backup job. Can only do 1 troubleshooting test per day. Suspect a bad drive or a bad interface.
Solution: Disable the suspect drive
Time to completion: 2 weeks (3 Month-end backup cycles missed)

Problem: Avamar client will not install on the new server, cannot export data to tape system.
Solution: Have a separate server act as the staging server.
Time to completion: 1/2 day

Problem: New Separate server has hardware issues and blue-screens regularly
Solution: Get another separate staging server.
Time to completion: 1/2 day

Problem: Don't know yet, but I'm waiting for the next one.

After all this what is the lesson to be learned?
.
.
.
.
.
.
.
Tape backup systems are the god-damned dinosaurs of storage and should be rendered extinct!

No comments: