Category: storage

HDD vendors sing a different tune, of course. Back in 2021, HDD vendor Seagate said the SSD most certainly would not kill disk drives. There’s a VAST vs Infinidat angle to it as well, with the former also stating disk drive IO limitations would cripple the use of larger disk drives in petabyte-scale data stores, with Infidat blasting back that it “must be joking.” Gartner has had a look in too, claiming that enterprise SSDs will hit 35 percent of HDD/SSD exabytes shipped by 2026 – though that would make Rosemarin’s 2028 cutoff unlikely. Pure recently stated SSDs would kill HDDs in a crossover event that would happen “soon.” Rosemarin, meanwhile, continued his argument: “Our CEO in many recent events has quoted that 3 percent of the world’s power is in datacenters. Roughly a third of that is storage. Almost all of that is spinning disk.

So if I can eliminate the spinning disk, and I can move to flash, and I can in essence reduce the power consumption by 80 or 90 percent while moving density by orders of magnitude in an environment where NAND pricing continues to fall, it’s all becoming evident that hard drives go away.” Are high electricity prices set to continue? “I think the UK’s power has gone up almost 5x recently. And here’s the thing … when they go up, they very seldom if ever come down … I’ve been asked many times do I think the cost of electricity will drop over time. And, frankly, while I wish it would and I do think there are technologies like nuclear that could help us over time. I think it’ll take us several years to get there. We’re already seeing countries putting quotas on electricity, and this is a really important one — we’ve already seen major hyperscalers such as one last summer who tried to enter Ireland [and] was told you can’t come here, we don’t have enough power for you. The next logical step from that is OK, so now if you’re a company and I start to say, well, we only have so much power, so I’m gonna give you X amount of kilowatts per X amount of employees, or I’m gonna give you X amount of kilowatts for X amount of revenue that you contribute to the GDP of the country or whatever metric is acceptable.”

The Gigabyte drive is currently listed on Amazon and Newegg, though the latter is currently sold out while the former is only available via a third-party marketplace seller — at a whopping $679.89 for the 2TB model. That’s almost certainly not the MSRP or a reflection of what MSRP might end up being once the drive becomes more widely available, which should happen in the coming month or two.

The other PCIe 5.0 M.2 SSD that’s now available is the Inland TD510 2TB, available at Microcenter for just $349.99 — assuming you have a Microcenter within driving distance. Inland is Microcenter’s own brand of drive, and while the cooler that comes with the SSD isn’t quite as large as the Aorus, it does feature a small fan for active cooling. Word is that the fan can be quite loud for something this small, so not a great feature in other words. Like the Aorus 10000, the Inland TD510 uses the Phison E26 controller and has the same 10,000 MB/s reads and 9,500 MB/s writes specification. Where Gigabyte doesn’t currently list random read/write speeds, the Microcenter page lists up to 1.5 million IOPS read and 1.25 million IOPS write for the Inland drive. Both drives also have an endurance rating of 1,400 TBW, with read/write power use of around 11W.

But someone’s doing something about it: the newly-founded Flickr Foundation, which has announced plans “to make sure Flickr will be preserved for future generations.” Or, as Popular Photography puts it, to stop photos “from suffering the same ill fate as our MySpace photos” — providing the example of important historical photos.

One particular collection their article notes is The Flickr Commons, “started back in 2008 as a collaborative effort with the Library of Congress to make publicly held photography collections readily available online for people seeking them out.”

It’s a massive, eclectic, fascinating archive that pulls images and content from around the world. This new organization hopes to integrate more partners and ensure that everything remains available and easily accessible…. If you’re not already familiar with The Commons, it’s a really fascinating online resource. It grants access to everything from historical portraits to scientific images and everything in between. It’s easy to get lost in the sheer volume of images available on the site, but Flickr relies on curators in order to bring notable images to the forefront and keep things organized and available.

With the establishment of the new foundation, Flickr hopes that it can keep this archive running to 2122 and beyond. It will doubtlessly add countless more images along the way.

Flickr is currently hiring a new archivist, according to their announcement (which also points out that the Flickr API was one of the first public APIs ever).

Among other things, it says that the foundation hopes to “investigate preservation strategies that could last for the next century,”

Residential homes are usually served by a single-phase power supply, and this project, on average, is expected to store and dispatch enough electricity to power more than 47,000 homes each year. Crimson Storage holds two long-term contracts with local utilities: a 200 MW/800 MWh 14-year and 10-month contract with Southern California Edison, and a 150 MW/ 600MWh 15-year contract with Pacific Gas and Electric.

“Hyperscale operators take a lot of factors into account when deciding where to locate their datacenter infrastructure,” said Synergy chief analyst John Dinsdale. “This includes availability of suitable real estate, cost and availability of power supply options, proximity to customers, the risk of natural disasters, local incentives and approvals processes, the ease of doing business and internal business dynamics, and this has inevitably led to some hyperscale hot spots.” Amazon in particular locates a large amount of its datacenter infrastructure in Northern Virginia, with Microsoft, Facebook, Google, ByteDance, and others also having a major presence, according to Synergy. The big three cloud providers — Amazon, Microsoft and Google — have the broadest hyperscale bit barn footprint, with each of these having over 130 datacenters of the 800 or so around the globe. When measured in datacenter capacity, the leading companies are Amazon, Google, Microsoft, Facebook, Alibaba and Tencent, according to Synergy.

According to Article 26 of the government’s Decree 53, the new rules go into effect October 1, 2022 — around seven weeks from the date of its announcement. However, foreign companies have an entire 12 months in which to comply — beginning when they receive instructions from the Minister of Public Security. The companies are then required to store the data in Vietnam for a minimum of 24 months. System logs will need to be stored for 12 months. After this grace period, authorities reserve the right to make sure affected companies are following the law through investigations and data collection requests, as well as content removal orders. Further reading: Vietnam To Make Apple Watch, MacBook For First Time Ever

As always, the issue with switching all systems to SSDs boils down to cost: Trendfocus Vice President John Chen tells us that replacing a 1TB HDD requires stepping down to a low-cost 256 GB SSD, which OEMs don’t consider to be enough capacity for most users. Conversely, stepping up to a 512 GB SSD would ‘break the budget’ for lower-end machines with a strict price limit. “The original cut-in date based on our discussions with OEMs was to be this year, but it has been pushed out to sometime next year (the second half, I believe, but not clear on the firm date),” Chen told Tom’s Hardware. “OEMs are trying to negotiate some level of push out (emerging market transition in 2024, or desktop transition in 2024), but things are still in flux.”

The majority of PCs in developed markets have already transitioned to SSDs for boot drives, but there are exceptions. Chen notes that it is possible that Microsoft could make some exceptions, but the firm predicts that dual-drive desktop PCs and gaming laptops with both an SSD for the boot drive and an HDD for bulk storage will be the only mass-market PCs with an HDD. […] It’s unclear what measures, if any, Microsoft would take with OEMs if they don’t comply with its wishes, and the company has decided not to comment on the matter. Trendfocus says the switchover will have implications for HDD demand next year.

As a demonstration of DirectStorage, though, Forspoken’s numbers are a mixed bag. On one hand, the examples Ono showcased clearly demonstrate DirectStorage loading scenes more quickly on the same hardware, compared to the legacy Win32 API — from 2.6 seconds to 2.2 seconds in one scene, and from 2.4 seconds to 1.9 seconds in another. Forspoken demonstrated performance improvements on older SATA-based SSDs as well, despite being marketed as a feature that will primarily benefit NVMe drives — dropping from 5.0 to 4.6 seconds in one scene, and from 4.1 to 3.4 seconds in another. Speed improvements for SATA SSDs have been limited for the better part of a decade now because the SATA interface itself (rather than the SSD controller or NAND flash chips) has been holding them back. So eking out any kind of measurable improvement for those drives is noteworthy.

On the other hand, Ono’s demo showed that game load time wasn’t improving as dramatically as the raw I/O speeds would suggest. On an NVMe SSD, I/O speeds increased from 2,862MB/s using Win32 to 4,829MB/s using DirectStorage — nearly a 70 percent increase. But the load time for the scene decreased from 2.1 to 1.9 seconds. That’s a decrease that wouldn’t be noticeable even if you were trying to notice it. The Forspoken demo ultimately showed that the speed of the storage you’re using still has a lot more to do with how quickly your games load than DirectStorage does. One scene that took 24.6 seconds to load using DirectStorage on an HDD took just 4.6 seconds to load on a SATA SSD and 2.2 seconds to load on an NVMe SSD. That’s a much larger gap than the one between Win32 and DirectStorage running on the same hardware.

One unexciting possibility is that the A13 Bionic chip used inside the Studio Display is literally the exact same A13 Bionic chip that was first shipped in the iPhone 11. As you might remember, the iPhone 11 came with 64GB of storage in its entry-level configuration, meaning Apple likely produced millions of A13 Bionic chips with 64GB of onboard storage. What do you think? Will Apple ever tap into the A13 Bionic chip and 64GB storage inside the Studio Display for something more interesting?

It is time to replace file systems with something better. A local object store that can effectively manage hundreds of millions of files and find things in seconds based on file type and/or tags attached is possible. File systems are usually free and come with your operating system, so there seems to be little incentive for someone to build a new system from scratch, but just like we needed the internet to come along and change everything we need a better data storage manager.
See Didgets for an example of what is possible.
In a Substack article, Didgets developer Andy Lawrence argues his system solves many of the problems associated with the antiquated file systems still in use today. “With Didgets, each record is only 64 bytes which means a table with 200 million records is less than 13GB total, which is much more manageable,” writes Lawrence. Didgets also has “a small field in its metadata record that tells whether the file is a photo or a document or a video or some other type,” helping to dramatically speed up searches.

Do you think it’s time to replace file systems with an alternative system, such as Didgets? Why or why not?