Back in 2012, I wrote a post titled Looking at External Disk Performance using USB 3.0 and eSATA with SSD, where I tested a number of external drive caddy’s with SSD’s that I had replaced and just had sitting around. Ultimately I started to use those SSD’s for storing information that I didn’t really need to have on my laptops and when they were full, they ended up in my desk drawer, where they have sat, unplugged and “safe” for the last 7-8 years. Or so I thought. With cheer competitions season in full swing, one of the things I love to do is shoot photos of my kids and their teammates competing, and storing RAW files that are 25-35MB per photo when you shoot 1000+ photos in a weekend across four different teams starts to take up a lot of space, so I figured I would pull out the old SSD’s and see what was on them that was worth keeping, delete what wasn’t and I could then move last years RAW files over to them and archive them for safe keeping. WRONG!!!  Of the four SSD’s I had stored data on, 100% of them had data loss due to a phenomenon known as bit rot. One of them wouldn’t even show up in Disk Manager in Windows and had to be low level formatted and reset using diskpart’s clean command due to partition table corruption.

TLDR;

You can’t write data to an SSD, disconnect it, store it in a “safe” location for years on end and expect the data to remain accessible.  SSD’s require power and routine usage to allow the onboard controller to check cells and perform reprogramming to reduce the likelihood of bit rot. Some relevant links on this:

• Bit Rot: How Hard Drives and SSDs Die Over Time

• Bit Rot: How to Avoid the slow death of hard drives and SSDs

• Bit Rot: What It Is and How To Stop It From Destroying Your Data

What is bit rot?

Quite simply put, bit rot is when the bits stored on a storage medium degrades over time to the point where they are no longer readable/reliable. In traditional hard disks, floppy disks, and even magnetic tapes used for backups, the bits lose their magnetic orientation.  In CD-R and DVD-R disks the storage medium is UV sensitive and prone to decay over time.  For solid state media, the NAND cells store data using electrical charges that can slowly leak electrons away over time.  Periodic refreshing/rewriting of the data is a method of being able to prevent bit rot on all forms of media, but suffice it to say that nothing lasts forever. 

What affects bit rot?

While doing some research on this issue I learned there are actually a huge number of variables that affect the rate of decay, with the most critical one being the storage medium being used first and foremost.  After that, the next most important variable is the storage conditions of the medium, with dark, cool, low humidity storage being the ideal condition for storing disks that hold backup data.  However, I live in Florida, which is anything but cool and low humidity, and to make matters worse, I moved for years ago and my desk, and my external SSD disks, spent a little over a month in a storage unit while I was between selling my house and closing on the purchase of my new place.  Not a good place to store them as the storage unit wasn’t climate controlled and I wasn’t really thinking about this as a consideration with everything else I was juggling at the time. 

Does this really matter?

As a data professional, I was actually surprised that I hadn’t encountered this previously, and found the readings online both fascinating, confusing, and contradictory.  There are plenty of sources that say it is a huge consideration, and then there are plenty of other sources that downplay the effects/likelihood of it occurring.  I recognize that my situation is actually an extreme exception to what most people would do and is not typical/normal usage of the devices.  I also had other backups of practically everything that was on the SSD’s on other media, DVD’s copies of photos (which I am now in the process of testing/rewriting), photo backups on Facebook and Instagram in Albums, along with copies on Amazon Glacier and Dropbox.  I am that paranoid about losing memories of my kids. Not only that, but I had jpeg files that had been rendered by Lightroom after touchup edits to the original RAW files that remained on my local machine because they are significantly smaller in size at about 3-5MB per image, so while it is a lossy format, I can still go back and get the images. 

Detecting bit rot

So how do you go about checking for and detecting bit rot in SSD devices?  One of the ways that I found mentioned multiple times was using a tool called HD Tune which can perform an Error Scan of the device.  Here is the results of a Quick Scan of one of my Samsung 840 EVO drives that is used every day in my M6700.  The quick scan only takes a few seconds even for 1TB of space when nothing is wrong. 

And here is a quick scan of one of the SSD’s that I left disconnected in my desk drawer for the last seven years:

I stopped it at two minutes and forty-seven seconds because it was so incredibly slow.  There is also a feature to do a full scan, which will take a long time even on a good device because it is reading every block of the storage to check for CRC/ECC failures during the read.  For my 1TB drives with no errors it took almost an hour to perform the full test, scanning at around 280MB/sec.  For a bad 250GB drive I gave up at six minutes.

This is the same exact device as the bad quick scan above and shows that you really need the full scan to verify that there isn’t bit rot happening.  The scanning speed zeros out when it encounters an error and has to retry the operation in an attempt to correct the problem. 

Does this mean the device is bad?

No, it doesn’t mean the actual SSD is bad.  In fact I have been able to use diskpart and the clean command to clear the partition table and then initialize the device with a new partition table and then format it and the HD Tune full error scan returns no damaged cells. 

The SMART controller also doesn’t report any issues with write failures of the cells, and the devices all have plenty of life left in them.  This unfortunately is just a case of bit rot due to poor storage conditions and not having the device powered up for an incredibly long period of time.

Lessons Learned

First and foremost, it is always important to have a backup of your data stored in a safe, cool, dark, low humidity place. Years ago  I switched from rotating HDD external drives to SSD’s because, well there’s no moving parts, so I wasn’t going to lose data because of a mechanical failure like a head crash due to vibration/impact or the drive refusing to spin up because the lubricant on the spindle hardening.  While I use DVD-R’s for backing up things, I have had enough scratched disks to know that those aren’t sufficient alone so I still keep offsite cloud based backups of things that matter to me, even if there is a higher cost associated with this.  What I didn’t know, and have now learned was that it’s important to also keep flash based storage not only safe but powered up so that it doesn’t suffer from bit rot.  What I don’t know still is whether just keeping the device powered up alone is sufficient or if it has to also be connected to the SATA bus as well.  There are lots of differences of opinions on this and contradictions if you read the articles.  For the foreseeable future I plan to continue to rely on multiple copies of everything on different storage mediums, formats and in the cloud as a means for backing up the precious moments with my kids that I will never be able to get back.  If you are a SQL Server professional, hopefully this makes you think about your backup storage and longevity.