I've been googling this all week and can't find anything definitive. May as well ask 'what's the best oil for my car' on a car forum!

I suspect I want to solve a problem I don't really have. It seems bit rot, while possible, is extremely unlikely (with exception for dye based media, eg burned blurays). It seems most data loss issues come from user error, hard ware failure, or memory errors (in that order from most likely to least likely), in addition, hard drives are pretty good at finding single bit issues and preventing them from becoming an issue. I was looking at getting a NAS specially to run ZFS to address the issue, but it looks like I'm probably better off just maintaining my backup program rather than adding complexity to my network/system.

Specific to me, all my data is on one computer that is always on already (windows 10 pro). No raid or anything fancy, but i copy everything to bit locked drive every night (it is mounted then unmounted specifically for the backup and not available any other time to prevent any inadvertent loss), then I do a weekly manual update to an external hard drive. I also do periodic dumps to bluray when I have enough to fill one (at 25gb, it only happens about twice a year), which essentially gives me 4 copies of anything important.

anyone have any better thoughts? do you run a big raid system and what is your most common source of data loss? anyone come across a real study on this and not just anecdotal commentary? Can't loose those wedding photos or my name would be mud!

 

Raid, redundant drivers, files or systems is just that, redundancy, it's not a backup for corrupted data so your backup strategy is a good one. ECC DRAM memory with server class hardware is a must have on my important systems.

https://www.cs.virginia.edu/~gurumurthi/papers/asplos15.pdf

 

Seems that "Bit Rot" may be caused by a termite infestation in your computer.

I've heard that the overuse of pest control chemicals may be create poison resistant insects and they are evolving from eating wood to feeding on plastics and metal.

 

Raid, redundant drivers, files or systems is just that, redundancy, it's not a backup for corrupted data so your backup strategy is a good one. ECC DRAM memory with server class hardware is a must have on my important systems.

https://www.cs.virginia.edu/~gurumurthi/papers/asplos15.pdf

thanks for the link! it seems using ecc ram would probably provide a bigger benefit than running a hard drive system with check sums (like ZFS) to protect against corruption of stored data.

Seems that "Bit Rot" may be caused by a termite infestation in your computer.

I've heard that the overuse of pest control chemicals may be create poison resistant insects and they are evolving from eating wood to feeding on plastics and metal.

haha, i did finally get rid of the ants at the back door! i'm sure they will be back next year though....

 

For large file (HD video) streaming systems I normally run XFS on top of a hardware raid (for ‘bit rot’ or disk failures) disk block system with true enterprise drives with at least one hot spare parity drive. Both are good but I've used XFS for ages without problems. If the ECC on the physical drive can't transparently correct the physical read error, the data is lost anyway and a read failure will be reported to the controller and OS for a possible recovery. CRCs are great but they don't correct errors. If you love your data, use ECC RAM.

 

Department of Redundancy Department here -- Establish the levels of exposure and importance.
> OS -- backup the environment with a utility that handles bootable backup.
> apps -- stash the latest version of installer on an external drive (or 2).
> data -- your generated code etc, COPY IT EVERYWHERE. There is NO other help.

I have all my code (40+years of uC code) on 4 drives And on my colleagues system (geographic diverse).
Simple concept: higher importance + uniqueness of origin = protect like your children (kinda same idea).