Question: In an effort to prevent some sources of disk rot due to magnetic flux over time, I want to read and rewrite all the data on a disk. ?It looks like there is a ‘dd’ command to do this. ?Specifically:
dd if=/dev/sda of=/dev/sda
I have some questions regarding this command, specifically:
Answer: Empirical approach. I decided to run a test. Short answer: the test didn’t show the procedure is unsafe.
Testbed
uname -a:
Linux foobar 3.2.0-4-amd64 #1 SMP Debian 3.2.86-1 x86_64 GNU/Linux
An excerpt from smartctl -a /dev/sdc:
Model Family: ?Western Digital Caviar Blue EIDEDevice Model: ?WDC WD5000AAKB-00H8A0Firmware Version: 05.04E05User Capacity: ?500,107,862,016 bytes [500 GB]Sector Size: ?512 bytes logical/physical
Also SMART shows the disk is healthy.
It was connected via USB bridge (to USB 2.0 hub). lsusb | grep SATA:
Bus 001 Device 003: ID 152d:2509 JMicron Technology Corp. / JMicron USA Technology Corp. JMS539 SuperSpeed SATA II 3.0G Bridge
One partition, almost no unallocated space. fdisk -l /dev/sdc:
Disk /dev/sdc: 465,8 GiB, 500107862016 bytes, 976773168 sectorsUnits: sectors of 1 * 512 = 512 bytesSector size (logical/physical): 512 bytes / 512 bytesI/O size (minimum/optimal): 512 bytes / 512 bytesDisklabel type: dosDisk identifier: 0x202b0b89Device ?Boot Start ?End ?Sectors ?Size Id Type/dev/sdc1 ?2048 976773167 976771120 465,8G 83 Linux
Healthy ext4 filesystem, unmounted. file -s /dev/sdc1:
/dev/sdc1: Linux rev 1.0 ext4 filesystem data, UUID= (extents) (large files) (huge files)
The filesystem usage was about 20% but:
- it had been extensively used, so the free space was not all zeros for sure;
- I wasn’t going to rely on the filesystem anyway, I calculated md5sum of the entire device (see below).
Testing
At first I saved the checksum of the entire device content:
md5sum -b /dev/sdc > sdc.before.md5# the throughput was about 24 MB/s
Actual dd-ing:
dd if=/dev/sdc of=/dev/sdc# took 800 minutes, 10,4 MB/sdd if=/dev/sdc of=/dev/sdc bs=128M conv=sync# 630 minutes, 13,2 MB/sdd if=/dev/sdc of=/dev/sdc bs=512 conv=sync# 795 minutes, 10,5 MB/s
After each dd I run sync, then md5sum -b /dev/sdc. All the checksums matched sdc.before.md5.
I also did like strace dd if=/dev/sdc of=/dev/sdc and it flooded my terminal with pairs of lines like:
read(0, “]203 ]2031 ]2032 ]2033 ]2034 ]203f ]203r ]20330 ”…, 512) = 512write(1, “]203 ]2031 ]2032 ]2033 ]2034 ]203f ]203r ]20330 ”…, 512) = 512
or (for the large bs)
read(0, “A260_4245316273321p2033312503022354303630233366345237303244fx10@=0221+”…, 134217728) = 134217728write(1, “A260_4245316273321p2033312503022354303630233366345237303244fx10@=0221+”…, 134217728) = 134217728
Conclusions and answers
?
?
The checksum remains the same and there is nothing filesystem-specific here. Overwriting data with the same data should be safe.
Note: this is true for magnetic drives which overwrite sectors directly. SSDs on the other hand (and I’m not sure if all of them) need to erase a sector (or multiple sectors at once) before it can be written again. The same logical sector may be written to a different physical “cell” etc. In case of power failure this may lead to data corruption (or may not, I know little about precautions in SSDs). You mentioned magnetic flux so I guess SSDs are out of scope anyway.
?
?
strace shows this is exactly what happens. Buffers may delay writes but it seems that every sector that is read will be rewritten sooner or later.
I can think of an “optimization” (in OS or in HDD firmware) that would compare a write request to cached reads and might discard the write as unnecessary if the data is already there. It could make your procedure pointless. However, it seems unlikely because:
- the match would rarely occur unless you deliberately want it to happen;
- the cache not large enough would make no difference for large bs values.
So if in doubt then just use large bs. Large bs is good for performance as well.
?
?
I agree with one of the comments:
?
It does rewrite each sector, so presumably that is the “cure” for the alleged bit rot.
Still, to confirm this in practice we need statistical analysis of many disks (some “rejuvenated”, some not) over a long time. My single test cannot answer this.