Ask Slashdot: Is It Time To Replace File Systems?

DidgetMaster writes: Hard drive costs now hover around $20 per terabyte (TB). Drives bigger than 20TB are now available. Fast SSDs are more expensive, but the average user can now afford these in TB capacities as well. Yet, we are still using antiquated file systems that were designed decades ago when the biggest drives were much less than a single gigabyte (GB). Their oversized file records and slow directory traversal search algorithms make finding files on volumes that can hold more than 100 million files a nightmare. Rather than flexible tagging systems that could make searches quick and easy, they have things like “extended attributes” that are painfully slow to search on. Indexing services can be built on top of them, but these are not an integral part of the file system so they can be bypassed and become out of sync with the file system itself.

It is time to replace file systems with something better. A local object store that can effectively manage hundreds of millions of files and find things in seconds based on file type and/or tags attached is possible. File systems are usually free and come with your operating system, so there seems to be little incentive for someone to build a new system from scratch, but just like we needed the internet to come along and change everything we need a better data storage manager.
See Didgets for an example of what is possible.
In a Substack article, Didgets developer Andy Lawrence argues his system solves many of the problems associated with the antiquated file systems still in use today. “With Didgets, each record is only 64 bytes which means a table with 200 million records is less than 13GB total, which is much more manageable,” writes Lawrence. Didgets also has “a small field in its metadata record that tells whether the file is a photo or a document or a video or some other type,” helping to dramatically speed up searches.

Do you think it’s time to replace file systems with an alternative system, such as Didgets? Why or why not?

Read more of this story at Slashdot.

Inside ‘Project Tinman’: Peloton’s Plan To Conceal Rust In Its Exercise Bikes

Dubbed internally as “Project Tinman,” executives at Peloton worked to conceal a build-up of rust on some exercise machines (Warning: source may be paywalled; alternative source) that were sent to customers instead of returned to the manufacturer. “The project was first revealed in FT Magazine last week but eight current and former Peloton employees across four US states have provided further details on the operation,” reports the Financial Times. Here’s an excerpt from the report: They described the plan as a nationwide effort to avoid yet another costly recall just months after the company’s most tragic episode — the death of a child due to the design of its treadmill. Internal documents seen by the FT showed that Tinman’s “standard operating procedures” were for corrosion to be dealt with using a chemical solution called “rust converter,” which conceals corrosion by reacting “with the rust to form a black layer.” Employees said the scheme was called Tinman to avoid terms such as “rust” that executives decided were out of step with Peloton’s quality brand.

Insiders were also angered about enacting a plan that they argued cut across Peloton’s supposed focus on its users, who are called “members” to evoke a sense that buyers are more than customers and part of a broader community. Tinman also put a spotlight on the company’s quality control process versus meeting aggressive sales targets in the search for growth. Peloton said the issue affected at least 6,000 bikes and that 120 staff had undertaken “rigorous testing” on the devices to conclude the rust — which it described as “cosmetic oxidation” — had “no impact on a bike’s performance, quality, durability, reliability, or the overall member experience.”

Read more of this story at Slashdot.

Qualcomm Will Support AV1 Video Codec In 2023, Report Says

Protocol reports that Qualcomm will finally jump on the AV1 video codec bandwagon next year. Ars Technica reports: AV1 is the web’s next open, royalty-free video codec, and widespread adoption will require hardware support from the world’s chip vendors. Qualcomm’s 2022 flagship SoC, the Snapdragon 8 Gen 1 chip, doesn’t support AV1. Samsung’s Exynos 2200 managed to ship the video codec this year in international versions of the Galaxy S22, while the MediaTek Dimensity 1000 SoC has been shipping in phones for over a year now with AV1 support. Apple is a founding member of the AV1 Alliance, but its devices also don’t support the codec yet.

The report says Qualcomm’s “upcoming flagship Snapdragon mobile processor” — model number “SM8550” — will support AV1. That would probably be called the “Snapdragon 8 Gen 2” SoC, due out in 2023. Wide adoption of AV1 seems inevitable, though it is taking a while. The codec is a successor to Google’s VP8 and VP9 codecs and is being built by the Alliance for Open Media. The alliance’s lineup is a who’s who of tech companies, with founding members like Amazon, Apple, ARM, Facebook, Google, Intel, Microsoft, Mozilla, Netflix, Nvidia, and Samsung. Netflix and Google’s YouTube are both making AV1 support “a requirement” for future products that want to support either video service. That should motivate just about every hardware and software vendor out there to get the job done. Aside from being open source and royalty-free, the report notes that the newer AV1 codec also has the benefit of being 30% more efficient than H.265.

Read more of this story at Slashdot.

Microsoft Details ‘Planet-Scale’ AI Infrastructure Packing 100,000+ GPUs

Microsoft has revealed it operates a planet-scale distributed scheduling service for AI workloads that it has modestly dubbed “Singularity.” The Register reports: Described in a pre-press paper [PDF] co-authored by 26 Microsoft employees, Singularity’s aim is described as helping the software giant control costs by driving high utilization for deep learning workloads. Singularity achieves that goal with what the paper describes as a “novel workload-aware scheduler that can transparently preempt and elastically scale deep learning workloads to drive high utilization without impacting their correctness or performance, across a global fleet of AI accelerators (e.g., GPUs, FPGAs).”

The paper spends more time on the scheduler than on Singularity itself, but does offer some figures to depict the system’s architecture. An analysis of Singularity’s performance mentions a test run on Nvidia DGX-2 servers using a Xeon Platinum 8168 with two sockets of 20 cores each, eight V100 Model GPUs per server, 692GB of RAM, and networked over InfiniBand. With hundreds of thousands of GPUs in the Singularity fleet, plus FPGAs and possibly other accelerators, Microsoft has at least tens of thousands of such servers! The paper focuses on Singularity’s scaling tech and schedulers, which it asserts are its secret sauce because they reduce cost and increase reliability.

The software automatically decouples jobs from accelerator resources, which means when jobs scale up or down “we simply change the number of devices the workers are mapped to: this is completely transparent to the user, as the world-size (i.e. total number of workers) of the job remains the same regardless of the number of physical devices running the job.” That’s possible thanks to “a novel technique called replica splicing that makes it possible to time-slice multiple workers on the same device with negligible overhead, while enabling each worker to use the entire device memory.” […] “Singularity achieves a significant breakthrough in scheduling deep learning workloads, converting niche features such as elasticity into mainstream, always-on features that the scheduler can rely on for implementing stringent SLAs,” the paper concludes.

Read more of this story at Slashdot.

EU Accuses China of ‘Power Grab’ Over Smartphone Technology Licensing

The EU is taking China to the World Trade Organization for alleged patent infringements that are costing companies billions of euros, as part of what officials in Brussels claim is a “power grab” by Beijing [Editor’s note: the link may be paywalled; alternative source] to set smartphone technology licensing rates. Financial Times reports: Businesses, including Sweden’s Ericsson, Finland’s Nokia and Sharp of Japan, have lost money after China’s supreme court banned them from protecting their patents by securing licensing deals in foreign courts, the European Commission said. Chinese courts set licence fees at around half the market rate previously agreed between western technology providers and manufacturers such as Oppo, Xiaomi, ZTE and Huawei, it added.

The lower licensing fees set by Beijing deprive smartphone makers and other mobile telecommunications businesses of a crucial source of revenue to reinvest in research and development. “It is part of a global power grab by the Chinese government by legal means,” said a European Commission official. “It is a means to push Europe out.” Smartphone makers have agreed global standards for telecommunications networks. In return, technology manufacturers must license their patents to others. If they cannot agree on a price, they go to court to set it. Chinese courts generally set prices at half the level of those in the west, meaning their companies pay less for the technology from overseas providers. In August 2020, China’s Supreme People’s Court decided that Chinese courts can impose “anti-suit injunctions,” which forbid a company taking a case to a court outside the country. Those that do are liable for a â147,000 daily fine and the judgments of courts elsewhere are ignored.

Read more of this story at Slashdot.

Could Texas Avoid Blackouts With Renewable Energy?

“Around this time last year, millions of Texans were shivering without power during one of the coldest spells to hit the central United States,” remembers the Washington Post. “For five days, blackouts prevented people from heating their homes, cooking or even sleeping. More than 200 people died in what is considered the nation’s costliest winter storm on record, amounting to $24 billion in damages.

“Twelve months later, the state’s electrical grid, while improved, is still vulnerable to weather-induced power outages.”

“If we got another storm this year, like Uri in 2021, the grid would go down again,” said Andrew Dessler, a professor of atmospheric sciences at Texas A&M University. “This is still a huge risk for us.”

Now, a recent study shows that electricity blackouts can be avoided across the nation — perhaps even during intense weather events — by switching to 100 percent clean and renewable energy, such as solar, wind and water energy. “Technically and economically, we have 95 percent of the technologies we need to transition everything today,” said Mark Jacobson, lead author of the paper and professor of civil and environmental engineering at Stanford University. Wind, water and solar already account for about one fifth of the nation’s electricity, although a full transition in many areas is slow.

The study showed a switch to renewables would also lower energy requirements, reduce consumer costs, create millions of new jobs and improve people’s health….

The team found the actual energy demand decreased significantly by simply shifting to renewable resources, which are more efficient. For the entire United States, total end-use energy demand decreased by around 57 percent. Per capita household annual energy costs were around 63 percent less than a “business as usual” scenario…. In Texas, a complete green transition would reduce the annual average end-use power demand by 56 percent. It also reduces peak loads, or the highest amount of energy one draws from the grid at a time. Jacobson said many homes would also have their own storage and wouldn’t need to rely on the grid as much.
The team also found interconnecting electrical grids from different geographic regions can make the power system more reliable and reduce costs. Larger regions are more likely to have the wind blowing, the sun shining or hydroelectric power running somewhere else, which may be able to help fill any supply gaps. “The intermittency of renewable energy declines as you look at larger and larger areas,” said Dessler. “If it’s not windy in Texas, it could be windy in Iowa. In that case, they could be overproducing power and they could be shipping some of their extra power to us.” The study stated costs per unit energy in Texas are 27 percent lower when interconnected with the Midwest grid than when isolated, as it currently is.

Interestingly, long-duration batteries aren’t important for grid stability. the team found, since our current 4-hour batteries can just be connected for longer-term storage. Professor Dessler tells the Post we should think of “renewables” as a system which includes storage technology and easily-dispatchable energy solutions.

And the Post adds that a grid powered by renewables “would also produce cleaner air, which could reduce pollution-related deaths by 53,000 people per year and reduce pollution-related illnesses for millions of people in 2050.”

Read more of this story at Slashdot.

After Blackouts, Texas Became a Top State for New Solar Installations as Thousands Install Microgrids

“Thousands of Texans who have turned to solar power and battery storage, creating so-called microgrids, as a solution to blackouts,” reports the Houston Chronicle.

“With a venture creating the same little power plants for apartment buildings, Texas has become a national leader in residential solar power installations.”

From 2019 to 2020, small-scale solar capacity in Texas grew by 63 percent, to 1,093 megawatts from 670 megawatts, according to the Energy Information Administration. In the first three quarters of 2021, another 250 megawatts of residential solar were installed in the state, according to the Solar Energy Industries Association. In last year’s third quarter alone, Texas ranked second behind California in the amount of power from new installations during the period, the industry’s Washington, D.C. trade group said.

Surging demand for residential solar power in Texas after the February 2021 freeze put pressure on installers to keep up, said Abigail Hopper, president and CEO of the association. The race to buy new rooftop panels has slowed some, she said, but Texas remains among the top three states for new installations. And the shrinking price of solar cells will help support its growing popularity, Hopper said.

“I think as more and more Americans really struggle with the impact of severe weather — everything from fires, the cold, hurricanes, droughts — and see the impacts on power and power outages, you’re going to continue to see folks looking for resiliency,” Hopper said.

Read more of this story at Slashdot.