This article is part of a series on the subject of edge computing that was made possible with funding from Intel. This coverage remains entirely independent, with no input from Intel on how this article was reported or written.
When it comes to storage drives, solid state drive (SSD) technology went mainstream in the last decade. Ever since, analysts have kept a watchful eye on the number of hard disk drive (HDD) and SSD units sold, along with performance efficiency and the price per gigabyte of each drive type. SSDs offered significantly better performance than HDDs, but they cost far more per gigabyte.
Analysts’ graphs indicated that at some point there would be parity, if not a crossover point, at which time SSDs would perhaps kill off HDDs.
The current move from Serial ATA (SATA) to the NVM Express (NVMe) interface, which today enables 5 times the performance, only sweetens the SSD proposition.
Hear that sound? That’s the death knell for hard drives … right?
Not so fast
Yes, hard drive shipments are in trouble, and their slower performance compared to SSD technology is largely to blame. Arguably the biggest bombshell to drop in this space came with 2018 financial disclosures from Nidec, which manufactures most of the world’s hard drive spindles. Nidec’s graph speaks for itself: During calendar year 2020, the industry is projected to move roughly half as many hard drives as it did in 2014.
Statista numbers echo Nidec, showing SSD units (360 million) outstripping HDD units (330 million) for the first time in 2021. These figures almost exactly match the same crossover and levels revealed by the IHS SSD and HDD Storage Market Tracker in July.
But take a closer look at that Nidec chart, and you’ll see why hard drive makers Seagate and Western Digital won’t be closing up shop anytime soon. Despite the rampant growth of SSDs in the enterprise market, datacenter hard drive shipments are actually growing.
Where data lives
Think back 10 years, when living room DVRs supplied far more video to consumers than streaming services did. (According to Nielsen, that particular crossover arrived in 2016.) When a show aired on TV, millions of people might record it for later viewing, resulting in millions of show copies on millions of hard drives. Today, streaming services render DVR unnecessary. Millions of users can pull millions of viewings from a relatively few copies distributed across global servers, requiring a tiny fraction of the hard drives previously required.
Video infamously consumes the lion’s share of stored bytes, but the same sort of consolidation and deduplication, if you will, plays out across the dataverse. Fewer people need to maintain content locally, especially in the consumer space, which is why Nidec shows the most HDD erosion from the consumer market. As consumers, we need a few things handled at maximum speed, like operating system files, game level loads, work-in-progress files, and so on, and a comparatively small amount of SSD storage is great for all of that. In short, the C: drive volume runs best and fastest on SSD.
The bulk of “cold” or archival data, as well as backup data, can live in quiet obscurity on attached USB hard drives. (Again, notice how Nidec shows stability in External shipments.) Alternatively, or in addition, all that cold data can sit in the cloud. Why not? It’s cheap, universally accessible, highly dependable, and often easier to share than local storage.
Put another way, those who are comfortable with external disk storage will keep using it, but those numbers will remain fairly flat. Customers using client systems will increasingly use SSDs for the terabyte or two of high-performance files and push everything else to the cloud. This applies in homes and offices alike, even for those who want the cheapest systems. Right now, $50 can buy you a roughly 500GB SSD or a 2TB hard drive on Newegg. For the needs of a PC user placing most data in the cloud or on attached USB storage, the SSD and HDD are practically price-equivalent, so why not buy the faster option?
Admittedly, the above DVR example is misleading. Even if that sort of video consolidation yields a 1,000:1 reduction in bytes consumed, the capacity savings is more than offset by other types of content exploding across the market. Consider surveillance video, which is quickly migrating from SD and HD resolutions to 4K. Consider the internet of things and its billions of devices feeding data into the cloud for analysis. Consider the rise of analytics, with the need to aggregate ever more and larger databases. Add in smart homes, smart factories, smart cities, and so on, and the need for high-capacity storage only continues to balloon.
Gartner noted in April 2019 that “continuous [HDD] unit-volume declines will be lucratively offset by richer mixes of higher-capacity HDDs in all markets.” Toshiba expanded on this prediction in a 2019 storage trends report by stating, “not only does the amount of data that we store continue to grow unabated, its growth is faster than predicted. The expectation had been that, while the proportion of data stored on flash and SSD increased, there would be a drop in the quantity of data stored on hard drives and magnetic tape. However, it is clear today that all three technologies continue to grow simply because there is so much data to be stored. In 2019, it can be assumed that 90% of the capacity for typical cloud computing applications will be realized with hard disks, with some possibly on magnetic tape, and only 10% will be implemented with SSD.”
According to Eddie Ramirez, senior director of product marketing and datacenter devices at Western Digital, the company is shipping roughly 500 petabytes (PB) of hard disk storage to enterprise datacenters annually, compared to 68PB on SSD — a greater than 7:1 disparity. The entire NAND flash (the storage media within SSDs) industry would have to massively increase its production to displace the hard disk, even if price-per-gigabyte comparisons were equal. That output capability does not currently exist.
Ramirez further indicated that HDDs now hover around 1.8 cents per gigabyte, while SSDs range from 13 to 15 cents per gigabyte — again yielding a 7 to 8 times gap between the two technologies. The common argument from the “HDDs are dead” camp is that Moore’s Law keeps pushing SSD storage densities, which benefit from semiconductor fabrication technologies, higher at a faster rate. Disk densities, they argue, can’t keep pace.
Western Digital demonstrated its 20TB datacenter hard drive last June. This marked a doubling of the 10TB drive that arrived four years ago. Ramirez said that WD has a 30TB drive in its sights, thanks to new technologies such as microwave- and heat-assisted recording, and that the company will achieve the 30TB capacity in a shorter time than it leapt from 10TB to 20TB. Naturally, these top-capacity drives will carry a cost-per-terabyte premium, but they will remain a fraction of what SSDs would cost at the same capacities.
Will the gap dramatically narrow between HDD and SSD in the datacenter, as it did in the consumer market? Some experts say it won’t anytime soon. According to Ramirez, expect that price-per-gigabyte delta between the two technologies to still hover around 6:1, even as far out as the year 2030. And even the slightly more critical Joseph Unsworth, research vice president at Gartner, maintains that the price/capacity gap will persist beyond the next decade.
So no, hard disk technology is far from dead. If anything, its capacity advantages are only becoming more critical in datacenters.
Significance for businesses
As cloud providers continue to pack in ever-higher storage capacities, economies of scale rule. Storage becomes cheaper, enabling businesses to store more and more types of data. Expect cloud providers to offer an increasing array of services around these repositories.
Cold, warm (intermediate or “nearline” speed), and hot (fast) storage tiers now suit various application types at escalating price levels. Analytics and AI services can compare data sets from today versus a decade earlier, for example, and extract the impacts of trends and tactics. Also expect various levels of protection for different data tiers, especially as organizations face increasing demands for longer retention periods. For example, five years or so ago, most organizations were expected to store surveillance footage for 30 to 90 days. Today, in Georgia, police body cam video must be retained for at least 180 days. California requires video evidence to be stored for one to two years.
Navigating the pricing fine print for storage services can be daunting, but the overarching idea remains: For the foreseeable future, hard disk technology will remain with us. It may have lost its performance luster (note the demise of speedy 10K and 15K models geared toward enterprises), but high capacity, reliability, and a low cost-per-gigabyte relative to SSDs will keep HDDs as the go-to storage option for datacenters, driven by businesses and consumers craving ever more services surrounding ever more data.
This is why those old predictions of SSD technology killing off HDD were so misleading. Certainly, SSDs delivered better total cost of ownership — when viewed from an IOPS-per-watt perspective. But that’s the wrong metric for a world struggling to keep up with double-digit data growth. Sometimes, size really is what matters, and SSDs won’t be catching up to HDDs on that front for many years to come.