Why bother?

I like archiving. It gives me access to data that is no longer available on the internet. Like that obscure web page with the knowledge that would otherwise be lost to time. It also gives me access to data that was never there. Like that high-school video that everybody completely forgot about, and now you have the only copy left.

But how should the data be archived? On what physical medium? After all, there is no point in archiving anything if you won’t be able to read it back after a couple of years. Or how about after a decade? A quarter of a century?

You have a couple options here. I won’t go into details you can find yourself, and I’ll only present a short summary. Remember that we are talking about cold storage here. You want to keep archival media in a shoebox somewhere, without a burden of maintenance.

• HDD: Prone to mechanical failure. Sensitive to magnetic fields. Risk of long-term problems with interfaces – have fun connecting your old PATA or SCSI drive to a modern PC.
• SSD: Memory cells lose charge when left unpowered for extended amounts of time. The same risk of long-term connectivity problems as HDDs.
• Pendrives: Supposedly a bit more resilient than SSDs, but still not reliable in long-term.
• Tape drives: Not really suitable for consumer use due to availability and pricing. Tapes are sensitive to magnetic fields and must be stored in a controlled environment to achieve a rated lifetime. No random access to data is possible.
• Optical media: Data is burned in organic dyes, which degrade over time. The disc surface is easy to scratch, making otherwise intact data unreadable.
• Cloud storage: Lol.
• Remote data storage you actually pay for: Fine for an off-site backup. Respects your privacy. Cheap options do exist. BuyVM offers 1 TB for $60 per year, plus $20 for a KVM slice. BorgBase offers 1 TB for $80 per year.

The most convenient storage medium is the optical disc. It doesn’t need maintenance and is small in size (you could even say that it is compact). The CD-ROM format was introduced to the market in 1987, and you can still read it with a brand new drive manufactured in 2021, a whopping 35 years later.

But how reliable is it?

Great question! It so happens that I have a couple of old recordable CDs and DVDs lying around, so why not check if they are still readable?

The test method

How do you check if a CD has not gone bad? You may try to copy the data, but it will give you just a simple yes or no answer, with no nuance involved. And there is a lot of nuances here. As the medium degrades, the drive will have to work harder to recover the data. Ultimately no amount of trying to read again will help. We need a way to see this behavior.

When a disc is read, the drive will want to do it as fast as possible. If everything is in order, the speed can be maintained. However, if there are some problems with reading the data, the drive will have to slow the disc rotation speed, reread the same sectors, etc. This will result in a visible read speed reduction. We can graph this.

For this purpose, I have chosen the dvdisaster utility. It is meant to generate recovery data, but it also has a scan mode, which produces the graph we need. A typical chart of a perfect disc read does look as follows:

As you can see, the maximum speed is preserved across the whole disc. We can observe the actual read speed increase because when a disc rotates with a constant angular velocity, the linear speed of data moving under the laser head increases the further away we move from the center. A slight speed dip at the beginning of the read cycle can be attributed to the drive calibrating to the inserted medium. The green spiral on the right side would show us any unreadable sectors and their location.

Let’s now see how an unreliable disc read may look like.

Here the drive cannot attain the maximum speed until the 120 MB mark and reads the data just at the 8× speed. Then, right before the 400 MB mark, the drive had to retry reading some of the sectors. Note that despite these issues, all sectors were successfully read.

In the case of dual-layer DVDs, we will first see a speed increase and then, after the drive switches to reading the second layer in the middle of data, a speed decrease.

For reference, the 1× speed of CD is 150 KB/s, the 1× speed of DVD is 1.3 MB/s, and the 1× speed of Blu-ray is 4.5 MB/s.

The data set

I have checked 106 different recordable discs using the LG BH16NS55 Blu-ray drive with 1.05 firmware.

There are some caveats. The optical media I used were not specially preselected, prepared, or otherwise made with this test in mind. They were created for archival purposes across many years. I do not know what drive was used to burn them. I do not know the write speed at which any of them was burned. I may vaguely guess what software was used. I cannot determine if any read problems are caused by the degradation of the dye or the bad quality of the original burn process.

Do you need to keep the disc surface pristine?

I do not know if any of the discs were scratched, dusty, had fingerprints, or any other such thing. I’m too lazy to check that. It does not matter.

Comment	Disc surface	Read graph
This dual-layer DVD was recorded in 2008. The surface is dusty, and there are some small scratches and fingerprints. It reads without any problems.
The disc was forcibly pressed against a wooden table and moved around for some time. This produced a lot of scratches across the whole surface. There was no change in how it was read.
A deep scratch in the surface material was made with a key. The first layer was read at reduced speed. The second layer read process was unaffected. There were no read errors.
In a last-ditch effort, I employed sandpaper. It finally managed to make the disc unreadable.

Physical degradation

Deterioration of writable layer chemistry is just one of the possible ways for a disc to stop working. Sometimes other layers may give due to the passage of time. For example, the reflective layer started to peel off on one disc. However, the disc was still perfectly readable even though this damage encroached on the data area.

Results

Out of the 106 discs, 20 had fully recoverable problems. Unfortunately, the data was partially lost on 2 discs, and another 2 discs were completely unreadable. It’s hard to tell whether this is a result of the discs failing due to old age or if it was just a bad burn that was not verified.

Full results are available in the table below. The speed column describes the maximum rated write speed of the disc, but not necessarily the speed at which burn was made. The rating column gives a subjective grade to each disc. To receive at least one star, the disc must be completely readable.

Note that some discs may have 2 unreadable sectors at the end. This is an artifact of writing in the Track-At-Once mode and not a real problem. Refer to the dvdisaster manual for more information.

Drives matter

I was astonished by the results I got, as I thought most of my discs had already perished. This was due to my previous drive, Samsung’s SH-S203, which turned out to be a broken pile of shit. I remember that I have become so frustrated with it, with only two discs out of a ten-pack turning out to be usable on average, that I have stopped burning anything 10 years ago.

The table below shows how my current LG drive compares to the Samsung junk.

LG BH16NS55	Samsung SH-S203

The Samsung drive could not read half of the discs I tried. In the readable second half, not one disc could be read without problems. At the same time, the LG drive managed to perform perfect reads on those discs.

Epilogue

It turns out you don’t need to worry about the decay of the organic dye in optical media anymore. Instead, you can purchase M-DISCs, where the data is etched in a “rock-like” layer. Moreover, they are available in a 100 GB Blu-ray form, making them much more useful than yesteryear’s 8.5 GB dual-layer DVDs.

This is an interesting test of M-DISC resilience: http://www.microscopy-uk.org.uk/mag/artsep16/mol-mdisc-review.html.

And here’s the Lasagna Test: