The problem with tape is the size of data required to make it cost effective. A disk is more expensive than tape of the same capacity but a lot cheaper than an equivalent tape drive. Last time I did the maths, 50TiB of backups was about the point when tape became cheaper. The article tells me that the point is now 165 TiB. That’s actually a bit misleading because it assumes a constant cost for the tape drive, but if you have 165 TiB of data on LTO-7 then you’re at almost 30 tapes and so you start to enter the territory where you want an autoloader of some form, which pushes the break-even point to 345 TiB (from the nice calculator in the article). And at that point you now have almost 60 tapes, so you probably want a larger autoloader.
By the time you have 300+ TiB of storage, you’re also well into the category of wanting off-site storage, which changes things a bit. Tape does well in this regard because you can keep your tape library on site and ship tapes to an off-site storage facility. That all adds to your costs.
The other comparison point is cloud storage. Cloud providers provide very cheap archive storage, which has similar characteristics to tape: cheap, write-mostly storage, with very high (multi-hour) latency for read and have the off-site and append-only bits built into the offering from the start. That 345 TiB point costs around £7,000. That buys you about 788 TiB-years of archive storage in the cloud, so for the same price as the disks or tape solution you could make the whole thing (including the off-site storage) someone else’s problem for 2-3 years, at which point I’d expect prices to have come down.
I wonder if those cloud providers use tape for their archive storage products?
I don’t think any of them talk publicly about what they use (and I don’t interact with the Azure Storage team so I have no direct knowledge of anything that we do). Project Pelican’s publications talk about targeting workloads that would traditionally use tape. Project Silica is aiming at replacing whatever we use now with laser-etched glass.
The economics are quite different for the cloud. Most data written to archive storage is write-only. People use it for offline off-site backups and so you only ever need to read back if your primary storage fails and your on-site backups / redundancy fail. If you’ve got a local redundant storage system with snapshots then the kinds of failure that require you to go to archive storage are pretty catastrophic (e.g. your office burned down). If you’re using cool storage for a first tier of backups then you probably don’t need to go to archive storage for most of these. At a guess, 99% of data is never read back, but you don’t know which 1% is going to be needed. This means that something with a tape-like model of a very expensive reader and very cheap and durable storage units makes sense. Silica is aiming to use cheap sheets of glass for the long-term storage with a reader / writer that makes tape drives look cheap. The writer basically streams whatever’s written to it onto sheets of glass and they’re then stored very densely. The reader may have to wait for the right sheet of glass to be moved into the queue, and streams the contents off into hot / cool storage for remote access.
This kind of model doesn’t make sense for anyone except a handful of companies. The fixed cost of building this infrastructure is huge and so you need to be storing many petabytes of data for the fact that the marginal cost of an extra terabyte is tiny to matter. Big movie studios that want to archive all of their historical footage including the unedited bits are probably in that space, not many other companies.
Project Silica is the coolest and most sci-fi thing I’ve ever heard that seems like a fundamentally terrible idea. XD Quartz glass will last basically forever, but sounds quite expensive and bulky compared to, say, metal foil or some kind of plastic. Great for really archival stuff like library stacks, but doesn’t seem terribly appealing for essentially-disposable write-once media in a data center.
Glass microfiche used to be a very popular archival format (before it was replaced by plastic/gel microfiche, although the glass was more durable and continued to see use), so this sounds pretty practical to me. The media is inexpensive and there was equipment available in the ’70s to store it in a very compact way. Microfiche is really durable and very easy to handle, esp. compared to its competitor microfilm, so the library environment I worked in years ago preferred it since patrons were less likely to damage it. I can see it having the same advantages for automated handling.
The glass is very cheap. On the order of $25 for a single piece. I can’t remember how many TiB they’re putting on one of those, but it’s a lot cheaper than tape with the same capacity. It’s then expected to be able to last for 100 years or more without any data loss (tape has layers of magnetic material next to each other, so gradually fades).
Cloud archive storage is not essentially disposable. 99% of it is never read, but you never know which 99%. Long-term reliability is incredibly important.
I suppose the storage density is the question, yes. From one of the papers on the Project Silica website, “Glass: A New Media for a New Era?”:
In the first system we anticipate that in a volume equivalent to a DVD-disk we can write about 1 TB. The technology can potentially get to 360 TB.
If we choose a moderate number of say 100 TB/plate then that’s $0.25/TB, or even at 10 TB/plate then it still beats current tape. From the original article:
…a 12TB SATA drive costs around £18.00 per TB (at the time of writing), a LTO-8 tape that has the same capacity costs around £7.40 per TB (at the time of writing)
…Okay, they are aiming to cram a lot more data onto those things than I expected, I suppose. Unfortunately I don’t see any open publications on how well it’s going, the latest actual numbers I see are an Ars Technica number from 2019 about them saving a 75 GB movie onto one plate. Not exactly thrilling. I do hope they make it work though, quartz glass would make a great historical medium. If I recall correctly from The World Without Us, glass is one of the most environmentally-resistant things we make, and quartz glass is tougher.
If we choose a moderate number of say 100 TB/plate then that’s $0.25/TB, or even at 10 TB/plate then it still beats current tape. From the original article:
The $25 plates are a lot bigger than a DVD. I think they’re about 30cm square and don’t have a hole in the middle for a spindle. A DVD is a 12cm diameter circle. The hole is 1.5cm, I think the gap around it adds up to 3cm, so the area is (6^2 * pi) - (1.5^2 * pi) cm^w, or around 106cm^2. The size of a Silica glass slab is around 8.5x the size of a DVD, so 100 TB/DVD-equivalent-area they’d be just short of 1PB. At their target number that’d be at 3PB per 30cm x 30cm glass slab.
Unfortunately I don’t see any open publications on how well it’s going, the latest actual numbers
This kind of project tends not to publish until it’s either shipping in a product or it’s cancelled. The last update I saw from the project was pretty impressive. The same team is working on holographic storage for cool storage, assuming a future storage flash or other persistent RAM for hot, holographic storage for cool, and glass for archive storage.
Not sure if you factored this in, but tapes are suitable for cold storage whereas it is generally discouraged for hard drives. That brings down the cost if you consider it over decades, where hard drives would be constantly spinning (at least if you follow conventional advice).
Tape backups aren’t dead, they’re just pining!
Nah honestly I can see this being really useful for some workloads… Just not worth the effort for most users anymore.
The problem with tape is the size of data required to make it cost effective. A disk is more expensive than tape of the same capacity but a lot cheaper than an equivalent tape drive. Last time I did the maths, 50TiB of backups was about the point when tape became cheaper. The article tells me that the point is now 165 TiB. That’s actually a bit misleading because it assumes a constant cost for the tape drive, but if you have 165 TiB of data on LTO-7 then you’re at almost 30 tapes and so you start to enter the territory where you want an autoloader of some form, which pushes the break-even point to 345 TiB (from the nice calculator in the article). And at that point you now have almost 60 tapes, so you probably want a larger autoloader.
By the time you have 300+ TiB of storage, you’re also well into the category of wanting off-site storage, which changes things a bit. Tape does well in this regard because you can keep your tape library on site and ship tapes to an off-site storage facility. That all adds to your costs.
The other comparison point is cloud storage. Cloud providers provide very cheap archive storage, which has similar characteristics to tape: cheap, write-mostly storage, with very high (multi-hour) latency for read and have the off-site and append-only bits built into the offering from the start. That 345 TiB point costs around £7,000. That buys you about 788 TiB-years of archive storage in the cloud, so for the same price as the disks or tape solution you could make the whole thing (including the off-site storage) someone else’s problem for 2-3 years, at which point I’d expect prices to have come down.
Thank you for the excellent summary of the problem! …and for doing all the math for me that I was about to do myself. <3
I wonder if those cloud providers use tape for their archive storage products?
I don’t think any of them talk publicly about what they use (and I don’t interact with the Azure Storage team so I have no direct knowledge of anything that we do). Project Pelican’s publications talk about targeting workloads that would traditionally use tape. Project Silica is aiming at replacing whatever we use now with laser-etched glass.
The economics are quite different for the cloud. Most data written to archive storage is write-only. People use it for offline off-site backups and so you only ever need to read back if your primary storage fails and your on-site backups / redundancy fail. If you’ve got a local redundant storage system with snapshots then the kinds of failure that require you to go to archive storage are pretty catastrophic (e.g. your office burned down). If you’re using cool storage for a first tier of backups then you probably don’t need to go to archive storage for most of these. At a guess, 99% of data is never read back, but you don’t know which 1% is going to be needed. This means that something with a tape-like model of a very expensive reader and very cheap and durable storage units makes sense. Silica is aiming to use cheap sheets of glass for the long-term storage with a reader / writer that makes tape drives look cheap. The writer basically streams whatever’s written to it onto sheets of glass and they’re then stored very densely. The reader may have to wait for the right sheet of glass to be moved into the queue, and streams the contents off into hot / cool storage for remote access.
This kind of model doesn’t make sense for anyone except a handful of companies. The fixed cost of building this infrastructure is huge and so you need to be storing many petabytes of data for the fact that the marginal cost of an extra terabyte is tiny to matter. Big movie studios that want to archive all of their historical footage including the unedited bits are probably in that space, not many other companies.
Project Silica is the coolest and most sci-fi thing I’ve ever heard that seems like a fundamentally terrible idea. XD Quartz glass will last basically forever, but sounds quite expensive and bulky compared to, say, metal foil or some kind of plastic. Great for really archival stuff like library stacks, but doesn’t seem terribly appealing for essentially-disposable write-once media in a data center.
Glass microfiche used to be a very popular archival format (before it was replaced by plastic/gel microfiche, although the glass was more durable and continued to see use), so this sounds pretty practical to me. The media is inexpensive and there was equipment available in the ’70s to store it in a very compact way. Microfiche is really durable and very easy to handle, esp. compared to its competitor microfilm, so the library environment I worked in years ago preferred it since patrons were less likely to damage it. I can see it having the same advantages for automated handling.
That’s interesting, I didn’t know that. I’ve used the plastic microfiche before but never seen glass.
The glass is very cheap. On the order of $25 for a single piece. I can’t remember how many TiB they’re putting on one of those, but it’s a lot cheaper than tape with the same capacity. It’s then expected to be able to last for 100 years or more without any data loss (tape has layers of magnetic material next to each other, so gradually fades).
Cloud archive storage is not essentially disposable. 99% of it is never read, but you never know which 99%. Long-term reliability is incredibly important.
I suppose the storage density is the question, yes. From one of the papers on the Project Silica website, “Glass: A New Media for a New Era?”:
If we choose a moderate number of say 100 TB/plate then that’s $0.25/TB, or even at 10 TB/plate then it still beats current tape. From the original article:
…Okay, they are aiming to cram a lot more data onto those things than I expected, I suppose. Unfortunately I don’t see any open publications on how well it’s going, the latest actual numbers I see are an Ars Technica number from 2019 about them saving a 75 GB movie onto one plate. Not exactly thrilling. I do hope they make it work though, quartz glass would make a great historical medium. If I recall correctly from The World Without Us, glass is one of the most environmentally-resistant things we make, and quartz glass is tougher.
The $25 plates are a lot bigger than a DVD. I think they’re about 30cm square and don’t have a hole in the middle for a spindle. A DVD is a 12cm diameter circle. The hole is 1.5cm, I think the gap around it adds up to 3cm, so the area is (6^2 * pi) - (1.5^2 * pi) cm^w, or around 106cm^2. The size of a Silica glass slab is around 8.5x the size of a DVD, so 100 TB/DVD-equivalent-area they’d be just short of 1PB. At their target number that’d be at 3PB per 30cm x 30cm glass slab.
This kind of project tends not to publish until it’s either shipping in a product or it’s cancelled. The last update I saw from the project was pretty impressive. The same team is working on holographic storage for cool storage, assuming a future storage flash or other persistent RAM for hot, holographic storage for cool, and glass for archive storage.
Not sure if you factored this in, but tapes are suitable for cold storage whereas it is generally discouraged for hard drives. That brings down the cost if you consider it over decades, where hard drives would be constantly spinning (at least if you follow conventional advice).