Researchers Upend AI Status Quo By Eliminating Matrix Multiplication In LLMs

Researchers from UC Santa Cruz, UC Davis, LuxiTech, and Soochow University have developed a new method to run AI language models more efficiently by eliminating matrix multiplication, potentially reducing the environmental impact and operational costs of AI systems. Ars Technica’s Benj Edwards reports: Matrix multiplication (often abbreviated to “MatMul”) is at the center of most neural network computational tasks today, and GPUs are particularly good at executing the math quickly because they can perform large numbers of multiplication operations in parallel. […] In the new paper, titled “Scalable MatMul-free Language Modeling,” the researchers describe creating a custom 2.7 billion parameter model without using MatMul that features similar performance to conventional large language models (LLMs). They also demonstrate running a 1.3 billion parameter model at 23.8 tokens per second on a GPU that was accelerated by a custom-programmed FPGA chip that uses about 13 watts of power (not counting the GPU’s power draw). The implication is that a more efficient FPGA “paves the way for the development of more efficient and hardware-friendly architectures,” they write.

The paper doesn’t provide power estimates for conventional LLMs, but this post from UC Santa Cruz estimates about 700 watts for a conventional model. However, in our experience, you can run a 2.7B parameter version of Llama 2 competently on a home PC with an RTX 3060 (that uses about 200 watts peak) powered by a 500-watt power supply. So, if you could theoretically completely run an LLM in only 13 watts on an FPGA (without a GPU), that would be a 38-fold decrease in power usage. The technique has not yet been peer-reviewed, but the researchers — Rui-Jie Zhu, Yu Zhang, Ethan Sifferman, Tyler Sheaves, Yiqiao Wang, Dustin Richmond, Peng Zhou, and Jason Eshraghian — claim that their work challenges the prevailing paradigm that matrix multiplication operations are indispensable for building high-performing language models. They argue that their approach could make large language models more accessible, efficient, and sustainable, particularly for deployment on resource-constrained hardware like smartphones. […]

The researchers say that scaling laws observed in their experiments suggest that the MatMul-free LM may also outperform traditional LLMs at very large scales. The researchers project that their approach could theoretically intersect with and surpass the performance of standard LLMs at scales around 10^23 FLOPS, which is roughly equivalent to the training compute required for models like Meta’s Llama-3 8B or Llama-2 70B. However, the authors note that their work has limitations. The MatMul-free LM has not been tested on extremely large-scale models (e.g., 100 billion-plus parameters) due to computational constraints. They call for institutions with larger resources to invest in scaling up and further developing this lightweight approach to language modeling.

Read more of this story at Slashdot.

Waymo’s Autonomous Ride-Hailing Service Now Available To All In San Francisco

An anonymous reader quotes a report from Reuters: Alphabet’s Waymo said on Tuesday its autonomous ride-hailing service, Waymo One, is now available to everyone in San Francisco, nearly four years after a similar move in Phoenix, Arizona. Driverless vehicles are expected to drive commercial success for automakers even as regulatory scrutiny remains tight amid concerns of investors about growing investments in the nascent technology. Waymo had started a test service with its research-focused program in San Francisco in 2021, which included an autonomous specialist on board for all rides at that time, as it looked to commercialize the technology.

The company said that about 300,000 people had signed up to ride with Waymo since it first opened a waitlist in the city, signaling strong demand. Now with open access, anyone can request a ride on its app. The company had opened access to everyone in Phoenix, Arizona without a waitlist in 2020. Mountain View, California-based Waymo is a self-driving technology pioneer, which started its first U.S. driverless taxi service in 2020 over a decade after it was born in 2009 as a project inside Google. In March, the company received approval from the California Public Utilities Commission (CPUC) to start its Waymo One in Los Angeles and some cities near San Francisco.

Read more of this story at Slashdot.

Meta Is Tagging Real Photos As ‘Made With AI,’ Says Photographers

Since May, Meta has been labeling photos created with AI tools on its social networks to help users better identify the content they’re consuming. However, as TechCrunch’s Ivan Mehta reports, this approach has faced criticism as many photos not created using AI tools have been incorrectly labeled, prompting Meta to reevaluate its labeling strategy to better reflect the actual use of AI in images. From the report: There are plenty of examples of Meta automatically attaching the label to photos that were not created through AI. For example, this photo of Kolkata Knight Riders winning the Indian Premier League Cricket tournament. Notably, the label is only visible on the mobile apps and not on the web. Plenty of other photographers have raised concerns over their images having been wrongly tagged with the “Made with AI” label. Their point is that simply editing a photo with a tool should not be subject to the label.

Former White House photographer Pete Souza said in an Instagram post that one of his photos was tagged with the new label. Souza told TechCrunch in an email that Adobe changed how its cropping tool works and you have to “flatten the image” before saving it as a JPEG image. He suspects that this action has triggered Meta’s algorithm to attach this label. “What’s annoying is that the post forced me to include the ‘Made with AI’ even though I unchecked it,” Souza told TechCrunch.

Meta would not answer on the record to TechCrunch’s questions about Souza’s experience or other photographers’ posts who said their posts were incorrectly tagged. However, after publishing of the story, Meta said the company is evaluating its approach to indicate labels reflect the amount of AI used in an image. “Our intent has always been to help people know when they see content that has been made with AI. We are taking into account recent feedback and continue to evaluate our approach so that our labels reflect the amount of AI used in an image,” a Meta spokesperson told TechCrunch. “For now, Meta provides no separate labels to indicate if a photographer used a tool to clean up their photo, or used AI to create it,” notes TechCrunch. “For users, it might be hard to understand how much AI was involved in a photo.”

“Meta’s label specifies that ‘Generative AI may have been used to create or edit content in this post’ — but only if you tap on the label. Despite this approach, there are plenty of photos on Meta’s platforms that are clearly AI-generated, and Meta’s algorithm hasn’t labeled them.”

Read more of this story at Slashdot.

Julian Assange Reaches Plea Deal With US, Allowing Him To Go Free

WikiLeaks founder Julian Assange has agreed to a plea deal with the U.S. Justice Department over his alleged role in one of the largest U.S. government breaches of classified material. As a result, he will avoid imprisonment in the United States. CNN reports: Under the terms of the new agreement (PDF), Justice Department prosecutors will seek a 62-month sentence — which is equal to the amount of time Assange has served in a high-security prison in London while he fought extradition to the US. The plea deal would credit that time served, allowing Assange to immediately return to Australia, his native country. The plea deal must still be approved by a federal judge.

Assange had faced 18 counts from a 2019 indictment for his alleged role in the breach that carried a max of up to 175 years in prison, though he was unlikely to be sentenced to that time in full. Assange was being pursued by US authorities for publishing confidential military records supplied by former Army intelligence analyst Chelsea Manning in 2010 and 2011. US officials alleged that Assange goaded Manning into obtaining thousands of pages of unfiltered US diplomatic cables that potentially endangered confidential sources, Iraq war-related significant activity reports and information related to Guantanamo Bay detainees.

Read more of this story at Slashdot.

Windows 11 is Now Automatically Enabling OneDrive Folder Backup Without Asking Permission

An anonymous reader shares a report: Microsoft has made OneDrive slightly more annoying for Windows 11 users. Quietly and without any announcement, the company changed Windows 11’s initial setup so that it could turn on the automatic folder backup without asking for it. Now, those setting up a new Windows computer the way Microsoft wants them to (in other words, connected to the internet and signed into a Microsoft account) will get to their desktops with OneDrive already syncing stuff from folders like Desktop Pictures, Documents, Music, and Videos.

Depending on how much is stored there, you might end up with a desktop and other folders filled to the brim with shortcuts to various stuff right after finishing a clean Windows installation. Automatic folder backup in OneDrive is a very useful feature when used properly and when the user deliberately enables it. However, Microsoft decided that sending a few notification prompts to enable folder backup was not enough, so it just turned the feature on without asking anybody or even letting users know about it, resulting in a flood of Reddit posts about users complaining about what the hell are those green checkmarks next to files and shortcuts on their desktops.

Read more of this story at Slashdot.