In the past month, both of my Western Digital 1 TB external backup hard drives have failed on me. This was pretty disconcerting considering I bought two of them specifically so they could be redundant backups. I was a bit lazy and barely had enough time to transfer my data while the last drive was limping along.
I turns out there isn’t that much data on real-world hard drive failure rates. When searching, many reference this post by online backup site BackBlaze which has owned over 25,000 hard drives (also see their recent update). In both posts, Seagate drives had much higher failure rates than Hitachi or Western Digital. However, other tech experts point out the flaws in drawing that conclusion. Brands don’t matter as much when they keep merging with each other anyway.
Really, the only conclusions I would confidently draw are (1) don’t buy specific “low power consumption” models of Seagate hard drives and (2) expect hard drives to fail on you. Imagine how little can go wrong in order to fit 4 terabytes of information on a little spinning disk. This Backblaze post is the one you should actually read. It cites other studies that say hard drives fail much more often than the MTBF (Mean Time Between Failures) statistics put out by manufacturers, and it is hard to predict impending failures even with regular hard drive checks. Here is a chart showing that hard drives tend to fail either within the first year, or after 3 years.
Here is a chart of survival rates over time.
That’s roughly 1 in 5 drives failing within 4 years, and maybe half will fail within 6 years. That starts to make a service like Crashplan at $60 a year more palatable.
In the end, Backblaze still buys Seagate drives because their cheaper price more than offsets any expected unreliability. Perhaps that is what consumers should do as well. Forget brands, buy whatever is cheapest, keep redundant backups in expectation of failure, and save your receipts. Of course, hard drive warranties keep getting shorter and are now usually only a year long (perhaps an admission of their unreliability?). Here’s my plan:
- Purchase with an American Express that offers their Extended Warranty which will double the manufacturer’s warranty, up to one year.
- Purchase from Costco whatever is on sale, due to their generous return policy. While even 90 days is relatively good, but I also do not see hard drives included explicitly in their 90-day electronics return policy – “We guarantee your satisfaction on every product we sell with a full refund. The following must be returned within 90 days of purchase for a refund: televisions, projectors, computers, cameras, camcorders, touchscreen tablets, MP3 players and cellular phones.” Do accessories count?
- Keep my important documents and family pictures on multiple formats (cloud, flash drives, multiple hard drives).
The Best Credit Card Bonus Offers – November 2024
Big List of Free Stocks from Brokerage Apps
Best Interest Rates on Cash - November 2024
Free Credit Scores x 3 + Free Credit Monitoring
Best No Fee 0% APR Balance Transfer Offers
Little-Known Cellular Data Plans That Can Save Big Money
How To Haggle Your Cable or Direct TV Bill
Big List of Free Consumer Data Reports (Credit, Rent, Work)
Why are you recommending CrashPlan when you’re citing BackBlaze? BackBlaze is a great backup service and even better because they open up their research to help consumers!
I have no experience with either Backblaze or Crashplan, and am not recommending one over the other. But I’ve read many more favorable reviews for Crashplan. I’ve only read hard drive-related posts about Backblaze.
1 – I’m a tech guy, who has been sitting in front of computers 12 hours a day for 20+ years, and the “Expect Failure and Plan Accordingly” advice is of course the best advice anyone can give.
2 – That said true (physical) hard drive failures actually seem pretty rare to me. I have had around 50 hard drives in the last 20+ years and only had one true (physical) hard drive failure. In fact, I have a handful of 10+ year old drives that are still in use and working fine.
3 – In my experience far too often logical data corruption, crappy software, end user error, etc gets mislabeled inappropriately as “hard drive failure.” Heck, one of my 4-drive RAIDs (a box with 4 hard drives in it that work together) reported that each of the 4 drives in it had “died” at different points in the last 3 months. And all of those were false alarms. The drives were removed, run through endurance tests, low level reformatted, including writing and reading every sector on the entire disk multiple times, and everything checked out OK. The disks were each returned to service. False alarm “failures” like this seem common in many consumer-level RAID devices.
4 – Also buying better quality server-class hard drives can help a lot on the lifespan of drives. These days I always buy server-class drives. But some of those 10+ year old drives that I still have in service were regular consumer-class drives at the time.
5 – With things moving, slowly, to SSD (solid state disks) it would seems like we should see fewer and fewer “hard drive failures”. Except for the little publicized “write cycle” issue with SSDs in that they do actually die after writing to each address some limited number of times. (Only 2,000-3,000 times in present drives if I recall correctly.) And in fact the performance of SSD write-cycles has been getting WORSE with each new generation of drives over the last 10 years. As they increase the storage size, increase speed, and reduce cost, they in fact have continually decreased the write-cycles.
My WD drives are making some pretty loud squeaking and clicking noises. Is there a Mac utility that I can use to try and recover data or reformat them if even the System Finder can’t recognize them?
I’m also disappointed by the SSD information I am reading these days. My current SSD as primary drive is 3 years old now and counting.
Hi Jonathan,
Thanks for this info. It’s supports my findings and what I was told many yrs ago: The question is not if a hard drive will crash, it’s when.
I’ve always had multiple backup drives/raided/etc. But about 6 yrs ago, I put a NAS at my parents to be a remote backup in case of disaster (fire, water damage, etc). With a little one also, I do not want to lose any pics/videos/etc. Data is the most valuable part of a computer.
I don’t trust any cloud backup services though. So, if you’re utilizing cloud services, a remote backup nas may be overkill.
Hi. I can share some tips:
When buying multiple hard disks, try to get them from different batch numbers so that their failure rates should not be correlated (i.e. this could be why both your disks failed about the same time).
Also, use a RAID array with multiple hard disks: http://en.wikipedia.org/wiki/RAID, so even if one fails, you have some redundancy. I am thinking of getting a Synology NAS myself ($300 for 4 drives, haven’t pulled the trigger yet).
I also backup to Amazon AWS using rsync (I spent months cooking up a solution and that was the best). I setup a nightly script to turn on a “tiny” machine on AWS ($0.02/hour) and use rsync (http://en.wikipedia.org/wiki/Rsync) to synchronize my local disk with AWS. rsync is bandwidth efficient. For about 75GB of backup, it takes less than an hour to sync just the stuff that changed for that day.
Seems like you have concocted a good solution. Would you be willing to share more details? Thanks.
Sure. Sorry, I am going to use AWS acronyms for brevity, but you can find more info from Amazon: http://aws.amazon.com/.
I signed up for an AWS account and setup a “tiny” EC2 instance with an EBS of the desired size. The downside of using EBS is that there are no guarantees around failure rates, but that’s my nth copy anyway, so I am not too worried about it. The EC2 instance I setup runs Ubuntu Linux. I suppose you can use Windows, but Linux is free.
I use rsync as the synchronization tool because it’s bandwidth-efficient. On Linux, you just need to make sure SSH is on and the rsync tool is installed. No additional installation necessary.
Then, every night, my Raspberry Pi (which also runs Linux) wakes up the EC2 instance, uses rsync to talk to it and the data is copied over.
YMMV. Each person has a specific need, so I am only outlining the strategy. The devil is in the details.
I experimented with Amazon S3 and even mounted it using FUSE on Linux, but the number of ops required to check whether a file was there or not was just too many (in the millions), so I scrapped that plan.
I tried Google Drive too, but the problem is that rsync needs to read in the entire file to know if it changed, so if the storage is not local, you are using bandwidth.
There’s also http://www.s3rsync.com/, but they have a limit of 10GB and I am too lazy to split my data into neat chunks like this — I just backup everything.
There is a long running endurance test of SSD drives that implies they are not as wear prone as is commonly believed. You can see the latest report on the test here:
http://techreport.com/review/26523/the-ssd-endurance-experiment-casualties-on-the-way-to-a-petabyte
It may very well be that SSDs are more reliable than spinning disks in the end. Especially in that they seem to provide ample warning of their pending failure.
I’ve been using CrystalDiskInfo (http://download.cnet.com/CrystalDiskInfo/3000-2086_4-10832082.html) to check the health of my drives. Does anyone know if a utility like this is of any value in predicting failures?
Looks like that utility does support the SMART standard. SMART is a hard drive standard that monitors and reports drive health : http://en.wikipedia.org/wiki/S.M.A.R.T.
So yes, that utility *should* be able to help you predict failures. But I wouldn’t rely on that too heavily, you can get failures before any SMART warnings. SMART can sometimes warn you in advance so thats good but its not perfect.
Great post. I’ve been doing a few things to back up my personal files. Since you seem to be on a Mac as well, I’ll share:
1. Backup my Mac to external drive via Time Machine. And also back up my wife’s Mac and my Linux laptop to external drive connected to my Mac using Crashplan. (She doesn’t like Time Machine for some reason.) Both solutions are free. The Crashplan solution does require my Mac to be on at the same time as the other computers, so I’ve set the sleep time for my Mac to three hours to help that occur more frequently.
2. Backup all important files to cloud services, so if a fire wipes out all my computers and external drives, I should still have access to my most important files. I’ve been milking the free cloud options for all they’re worth, and have plenty of space for important photos, my Photoshop projects, encrypted collections of financial and tax records, writing, files for work, etc. My Dropbox gives me 7 Gb after all the referrals I’ve made. Google Drive 15 Gb. OneDrive 30 Gb. With Box.com I got a limited time offer a couple of years ago for 50 Gb free (but I think they only offer 10 Gb free normally). In total, over 100 Gb free. I mostly use OneDrive for my photos partly because of the large amount of free space and partly because it seems to have the most impressive interface for interacting with your photos.
3. I use Automator on the Mac (or you could do it with a Python script in any platform) to once a week automatically copy and replace a few important encrypted containers that I really would hate to lose to two of the cloud services as well as an external drive. If you’re interested in how to do this, Google “Automator Calendar Alarm.”
For me, it used to be easy – Seagate had the best price/performance ratio, and was extremely reliable. Then, something changed. After a few disc failures in a short period of time, I switched around, and now use Western Digital for the most part. The lowered warranty period from ALL manufacturers is a cause for worry with me, and I’ve started using cloud based backup for important files, much in the way Dan does (splitting stuff between different free options).
As for whether Costco considers a hard drive as being covered under the shorter return policy, I can’t say for sure. But I can tell you that many years ago, I purchased some sort of peripheral from Staples – I think it was a modem, but I can’t be sure. They had something like a 90 day warranty on most items, and only 30 days on a list which included “Computers”, much like the Costco list. Within the 90 days, a modem came out that was twice as fast for the same price, and I went to return the first one. Guy tried to tell me it was only covered for 30 days, because it was a “computer”. Manager stuck to the same line, not being convinced by my “an oil filter goes IN a car, but ISN’T a car” analogy.
Finally he said, “If you will admit that this isn’t covered under our 90 day policy, I’ll give you the refund.” Being the stubborn fellow I am, I said, “I will admit that YOU don’t think it is covered.” Which he accepted, giving me back my money.
And I didn’t shop there again for many years.
I have lost information way too many times because I forget to do manual backups, so its the cloud for me.