Category: storage

But someone’s doing something about it: the newly-founded Flickr Foundation, which has announced plans “to make sure Flickr will be preserved for future generations.” Or, as Popular Photography puts it, to stop photos “from suffering the same ill fate as our MySpace photos” — providing the example of important historical photos.

One particular collection their article notes is The Flickr Commons, “started back in 2008 as a collaborative effort with the Library of Congress to make publicly held photography collections readily available online for people seeking them out.”

It’s a massive, eclectic, fascinating archive that pulls images and content from around the world. This new organization hopes to integrate more partners and ensure that everything remains available and easily accessible…. If you’re not already familiar with The Commons, it’s a really fascinating online resource. It grants access to everything from historical portraits to scientific images and everything in between. It’s easy to get lost in the sheer volume of images available on the site, but Flickr relies on curators in order to bring notable images to the forefront and keep things organized and available.

With the establishment of the new foundation, Flickr hopes that it can keep this archive running to 2122 and beyond. It will doubtlessly add countless more images along the way.

Flickr is currently hiring a new archivist, according to their announcement (which also points out that the Flickr API was one of the first public APIs ever).

Among other things, it says that the foundation hopes to “investigate preservation strategies that could last for the next century,”

Residential homes are usually served by a single-phase power supply, and this project, on average, is expected to store and dispatch enough electricity to power more than 47,000 homes each year. Crimson Storage holds two long-term contracts with local utilities: a 200 MW/800 MWh 14-year and 10-month contract with Southern California Edison, and a 150 MW/ 600MWh 15-year contract with Pacific Gas and Electric.

“Hyperscale operators take a lot of factors into account when deciding where to locate their datacenter infrastructure,” said Synergy chief analyst John Dinsdale. “This includes availability of suitable real estate, cost and availability of power supply options, proximity to customers, the risk of natural disasters, local incentives and approvals processes, the ease of doing business and internal business dynamics, and this has inevitably led to some hyperscale hot spots.” Amazon in particular locates a large amount of its datacenter infrastructure in Northern Virginia, with Microsoft, Facebook, Google, ByteDance, and others also having a major presence, according to Synergy. The big three cloud providers — Amazon, Microsoft and Google — have the broadest hyperscale bit barn footprint, with each of these having over 130 datacenters of the 800 or so around the globe. When measured in datacenter capacity, the leading companies are Amazon, Google, Microsoft, Facebook, Alibaba and Tencent, according to Synergy.

According to Article 26 of the government’s Decree 53, the new rules go into effect October 1, 2022 — around seven weeks from the date of its announcement. However, foreign companies have an entire 12 months in which to comply — beginning when they receive instructions from the Minister of Public Security. The companies are then required to store the data in Vietnam for a minimum of 24 months. System logs will need to be stored for 12 months. After this grace period, authorities reserve the right to make sure affected companies are following the law through investigations and data collection requests, as well as content removal orders. Further reading: Vietnam To Make Apple Watch, MacBook For First Time Ever

As always, the issue with switching all systems to SSDs boils down to cost: Trendfocus Vice President John Chen tells us that replacing a 1TB HDD requires stepping down to a low-cost 256 GB SSD, which OEMs don’t consider to be enough capacity for most users. Conversely, stepping up to a 512 GB SSD would ‘break the budget’ for lower-end machines with a strict price limit. “The original cut-in date based on our discussions with OEMs was to be this year, but it has been pushed out to sometime next year (the second half, I believe, but not clear on the firm date),” Chen told Tom’s Hardware. “OEMs are trying to negotiate some level of push out (emerging market transition in 2024, or desktop transition in 2024), but things are still in flux.”

The majority of PCs in developed markets have already transitioned to SSDs for boot drives, but there are exceptions. Chen notes that it is possible that Microsoft could make some exceptions, but the firm predicts that dual-drive desktop PCs and gaming laptops with both an SSD for the boot drive and an HDD for bulk storage will be the only mass-market PCs with an HDD. […] It’s unclear what measures, if any, Microsoft would take with OEMs if they don’t comply with its wishes, and the company has decided not to comment on the matter. Trendfocus says the switchover will have implications for HDD demand next year.

As a demonstration of DirectStorage, though, Forspoken’s numbers are a mixed bag. On one hand, the examples Ono showcased clearly demonstrate DirectStorage loading scenes more quickly on the same hardware, compared to the legacy Win32 API — from 2.6 seconds to 2.2 seconds in one scene, and from 2.4 seconds to 1.9 seconds in another. Forspoken demonstrated performance improvements on older SATA-based SSDs as well, despite being marketed as a feature that will primarily benefit NVMe drives — dropping from 5.0 to 4.6 seconds in one scene, and from 4.1 to 3.4 seconds in another. Speed improvements for SATA SSDs have been limited for the better part of a decade now because the SATA interface itself (rather than the SSD controller or NAND flash chips) has been holding them back. So eking out any kind of measurable improvement for those drives is noteworthy.

On the other hand, Ono’s demo showed that game load time wasn’t improving as dramatically as the raw I/O speeds would suggest. On an NVMe SSD, I/O speeds increased from 2,862MB/s using Win32 to 4,829MB/s using DirectStorage — nearly a 70 percent increase. But the load time for the scene decreased from 2.1 to 1.9 seconds. That’s a decrease that wouldn’t be noticeable even if you were trying to notice it. The Forspoken demo ultimately showed that the speed of the storage you’re using still has a lot more to do with how quickly your games load than DirectStorage does. One scene that took 24.6 seconds to load using DirectStorage on an HDD took just 4.6 seconds to load on a SATA SSD and 2.2 seconds to load on an NVMe SSD. That’s a much larger gap than the one between Win32 and DirectStorage running on the same hardware.

One unexciting possibility is that the A13 Bionic chip used inside the Studio Display is literally the exact same A13 Bionic chip that was first shipped in the iPhone 11. As you might remember, the iPhone 11 came with 64GB of storage in its entry-level configuration, meaning Apple likely produced millions of A13 Bionic chips with 64GB of onboard storage. What do you think? Will Apple ever tap into the A13 Bionic chip and 64GB storage inside the Studio Display for something more interesting?

It is time to replace file systems with something better. A local object store that can effectively manage hundreds of millions of files and find things in seconds based on file type and/or tags attached is possible. File systems are usually free and come with your operating system, so there seems to be little incentive for someone to build a new system from scratch, but just like we needed the internet to come along and change everything we need a better data storage manager.
See Didgets for an example of what is possible.
In a Substack article, Didgets developer Andy Lawrence argues his system solves many of the problems associated with the antiquated file systems still in use today. “With Didgets, each record is only 64 bytes which means a table with 200 million records is less than 13GB total, which is much more manageable,” writes Lawrence. Didgets also has “a small field in its metadata record that tells whether the file is a photo or a document or a video or some other type,” helping to dramatically speed up searches.

Do you think it’s time to replace file systems with an alternative system, such as Didgets? Why or why not?

But then they realized “It would also be interesting to compare the weirdness on other OSes…”

Here are the two data points in front of me:

(1) Using the right click on properties for all of the top-level folders on the drive (including the so-called hidden folders), it quickly determined that there are a few hundred thousand files in those folders (and a few hundred thousand subfolders). That’s already ridiculous, but the expected par these days. The largest project I have on the machine only has about 3,000 files, and that one goes back many years… (My largest database only has about 5,000 records, but it’s just a few files.)

(2) However, I also decided to take a look with Microsoft’s malicious software removal tool and got a completely different answer. For twisted grins, I had invoked the full scan. It’s still running a day later and has already passed 10 million files. Really? The progress bar indicates about 80% finished? WTF?

Obviously there is some kind of disagreement about the nature of “file” here. I could only think of one crazy explanation, but my router answered “No, the computer is not checking all of the files on the Internet.” So I’ve already asked the specific question in three-letter form, but the broader question is about the explosive, perhaps even cancerous, “population growth” of files these days.

Maybe we can all solve this mystery together. So use the comments to share your own answers and insights.

How many files are on your computer?