Code-Generating AI Can Introduce Security Vulnerabilities, Study Finds

An anonymous reader quotes a report from TechCrunch: A recent study finds that software engineers who use code-generating AI systems are more likely to cause security vulnerabilities in the apps they develop. The paper, co-authored by a team of researchers affiliated with Stanford, highlights the potential pitfalls of code-generating systems as vendors like GitHub start marketing them in earnest. The Stanford study looked specifically at Codex, the AI code-generating system developed by San Francisco-based research lab OpenAI. (Codex powers Copilot.) The researchers recruited 47 developers — ranging from undergraduate students to industry professionals with decades of programming experience — to use Codex to complete security-related problems across programming languages including Python, JavaScript and C.

Codex was trained on billions of lines of public code to suggest additional lines of code and functions given the context of existing code. The system surfaces a programming approach or solution in response to a description of what a developer wants to accomplish (e.g. “Say hello world”), drawing on both its knowledge base and the current context. According to the researchers, the study participants who had access to Codex were more likely to write incorrect and “insecure” (in the cybersecurity sense) solutions to programming problems compared to a control group. Even more concerningly, they were more likely to say that their insecure answers were secure compared to the people in the control.

Megha Srivastava, a postgraduate student at Stanford and the second co-author on the study, stressed that the findings aren’t a complete condemnation of Codex and other code-generating systems. The study participants didn’t have security expertise that might’ve enabled them to better spot code vulnerabilities, for one. That aside, Srivastava believes that code-generating systems are reliably helpful for tasks that aren’t high risk, like exploratory research code, and could with fine-tuning improve in their coding suggestions. “Companies that develop their own [systems], perhaps further trained on their in-house source code, may be better off as the model may be encouraged to generate outputs more in-line with their coding and security practices,” Srivastava said. The co-authors suggest vendors use a mechanism to “refine” users’ prompts to be more secure — “akin to a supervisor looking over and revising rough drafts of code,” reports TechCrunch. “They also suggest that developers of cryptography libraries ensure their default settings are secure, as code-generating systems tend to stick to default values that aren’t always free of exploits.”

Read more of this story at Slashdot.

Linux 6.1 Released With Initial Support for Rust-Based Kernel Development

“Linus has released the 6.1 kernel,” reports LWN.net — and it’s the one with initial support for kernel development in Rust.
Elsewhere LWN explains the specifics of this milestone:
No system with a production 6.1 kernel will be running any Rust code, but this change does give kernel developers a chance to play with the language in the kernel context and get a sense for how Rust development feels….

There are other initiatives underway, including the writing of an Apple graphics driver in the Rust language. For the initial merge into the mainline kernel, though, Linus Torvalds made it clear that as little functionality as possible should be included. So those drivers and their support code were trimmed out and must wait for a future kernel release. What is there is the support needed to build a module that can be loaded into the kernel, along with a small sample module…. Torvalds asked for something that could do “hello world” and that is what we got. It is something that can be played with, but it cannot be used for any sort of real kernel programming at this point.
That situation will, hopefully, change in the near future.

Meanwhile, Linux 6.1 also includes “support for destructive BPF programs, some significant io_uring performance improvements, better user-space control over transparent huge-page creation, improved memory-tiering support.”

The Register adds:
Other interesting additions include more support for the made-in-China LoongArch CPU architecture, introductory work to support Wi-Fi 7 and security fixes for some flaky Wi-Fi routines in previous versions of the kernel. There’s also plenty of effort to improve the performance of Linux on laptops, and enhanced power efficiency for AMD’s PC-centric RYZEN silicon.

Read more of this story at Slashdot.

C++ Zooms Past Java in Programming Language Popularity Contest

“Java is no longer among the top three most popular programming languages in the TIOBE Index,” reports the Register, “one of several not particularly definitive yardsticks by which such things are measured.”

According to Paul Jansen, CEO of Netherlands-based TIOBE Software, the rising popularity of C++ has pushed Java down a notch. The index’s rankings are now:
– Python in first place
– C second
– C++ third, and
– Java fourth.

C++ stepped up to third, and Java fell to fourth. “C++ surpassed Java for the first time in the history of the TIOBE Index, which means that Java is at position 4 now,” said Jansen in the December update for the TIOBE Index. “This is the first time that Java is not part of the top 3 since the beginning of the TIOBE Index in 2001.”

The surge in C++, perhaps in part helped by the stable release of C++ 20 in December 2020, is particularly ironic in light of the language’s recent dismissal by Microsoft CTO Mark Russinovich, which coincides with industry evangelism for Rust and its capacity for memory safety.
The article points out that other rankings still show a slighty higher popularity for Java.

And ZDNet notes the other languages rising quickly in popularity over the last 12 months:
In a year-on-year comparison in Tiobe’s index, the languages now in the top 20 that made significant gains over the period are: Rust (up from 27 to 20), Objective-C (up from 29 to 19), science-specialized MATLAB (20 to 14), and Google’s Go language (up from 19 to 12).

Read more of this story at Slashdot.

Over 50 Programmers Generate 50,000-Word Novels For 9th Annual ‘Nanogenmo’ Event

Long-time Slashdot reader destinyland writes: Since 1999 fiction writers have tried starting and finishing the composition of 50,000-word novels in November for “National Novel Writing Month”. But for the last nine years, programmers have instead tried generating 50,000 word novels — and this year’s edition received more than 50 entries.

“The only rule is that you share at least one novel and also your source code at the end,” explains the event’s official page on GitHub.

From the repository’s README file:

The “novel” is defined however you want. It could be 50,000 repetitions of the word “meow” (and yes it’s been done!). It could literally grab a random novel from Project Gutenberg. It doesn’t matter, as long as it’s 50k+ words.

Please try to respect copyright. We’re not going to police it, as ultimately it’s on your head if you want to just copy/paste a Stephen King novel or whatever, but the most useful/interesting implementations are going to be ones that don’t engender lawsuits.

This year’s computer-generated novels include ” sunday in the sunday in the,” mapping the colors from each dot in the Pointillist painting Sunday Afternoon on the Island of La Grande Jatte onto words from the lyrics of a musical about that painting. (“Rush blind. Link adds shallot again….”)

Read more of this story at Slashdot.

Linux Kernel Gets More Infrastructure for Rust, Increasing Interest in the Language

Linux 6.1 (released last month) included what Linus Torvalds described as “initial Rust scaffolding,” remembers this update from SD Times But now, “work has already been done since the 6.1 release to add more infrastructure for Rust in the kernel, though still none of the code interacts with any C code.”

And there’s still no actual Rust code in Linux:

“You need to get all those things that can make sure that Rust can compile, and you can do the debugging and all these things,” explained Joel Marcey, director of advocacy and operations for the Rust Foundation, “and make sure that the memory safety is there and all that sort of stuff. And that has to happen first before you can actually write any real code in Rust for the Linux kernel itself.”

Marcey explained that Linux is going to be doing this inclusion very piecemeal, with lots of little integrations here and there over time so they can see how it is working. “I would imagine that over the next year, you’re going to see more small incremental changes to the kernel with Rust, but as people are seeing that it’s actually kind of working out, you’ll be able to maybe, for example, write Linux drivers or whatever with Rust,” said Marcey….

According to Bec Rumbul, executive director of the Rust Foundation, Rust being added to the kernel is an “enormous vote of confidence in the Rust programming language.” She explained that in the past other languages have been planned to make it into the kernel and ended up not getting put in. “I think having someone with the kind of intellectual gravity of Linus Torvalds saying ‘No, it’s going in there,’ that kind of says an awful lot about how reliable Rust already is and how much potential there is for the future as well,” she said.

Rumbul believes that there will be an increased interest in the language, which is still relatively new (It first made its debut in 2010) compared to some of the other languages out there to choose from. “I suspect that because Rust is now in the kernel, and it’s just being talked about much … more widely, that it will seem like an attractive prospect to a lot of people that are looking to develop their skills and their knowledge,” she said. Rumbul hopes people will also be inspired to participate in the language as contributors and maintainers, because those are some of the less popular roles within open source, but are extremely critical to the health of a language, she explained.
The Rust Foundation also launched a new security team in September to ensure best practices (including a dedicated security engineer). Their first initiative will be a security audit and threat modeling exercises.

“We want to basically shore up,” Rust operations director Marcey tells SD Times, “to ensure that Rust itself is actually as secure as we always say it is.”

In this year’s Stack Overflow Developer Survey, 86.73% of developers said they love Rust.

Read more of this story at Slashdot.

Survey of 26K Developers Finds Java, Python, Kotlin, and Rust Growing Rapidly

While the popularity of jQuery is decreasing, React.JS “is currently the most widely used client-side framework,” reports ZDNet, citing SlashData’s 23rd State of the Developer Nation report (compiled from more than 26,000 developers last summer from 163 countries).

ZDNet believe it shows developers “experimenting less and sticking with what they know and what works.”

JavaScript remains the largest programming language community, SlashData found. According to its research, there are an estimated 19.6 million developers worldwide using JavaScript every day in everything from web development and mobile apps to backend coding, cloud and game design. Java, meanwhile, is growing rapidly. In the last two years, the size of the Java community has more than doubled from 8.3 million to 16.5 million, SlashData found. For perspective, the global developer population grew about half as fast over the same period….

Python also continued to grow strongly, adding about eight million new developers over the last two years, according to SlashData. It accredited the rise of data science and machine learning as “a clear factor in Python’s growing popularity”. Approximately 63% of machine-learning developers and data scientists report using Python, whereas less than 15% use R, another programming language often associated with data science.

Both the Kotlin and Rust communities doubled in size in the past two years, the article points out. But according to the survey, only 9% of developers were involved in blockchain technologies.

Yet 27% of respondents reported they were learning about (if not currently working on) cryptocurrency-based projects. ZDNet summarizes the findings:

Of the three blockchain technologies covered in the report, non-fungible tokens (NFTs) were found to be of least interest to developers: 58% showed “no interest” in NFTs, which SlashData said was “likely due to its perception as a novelty”.

The report found that one-quarter (25%) of developers currently work on, or are learning about, blockchain applications other than cryptocurrencies.

Read more of this story at Slashdot.

NSA Urges Organizations To Shift To Memory Safe Programming Languages

In an press release published earlier today, the National Security Agency (NSA) says it will be making a strategic shift to memory safe programming languages. The agency is advising organizations explore such changes themselves by utilizing languages such as C#, Go, Java, Ruby, or Swift. From the report: The “Software Memory Safety” Cybersecurity Information Sheet (PDF) highlights how malicious cyber actors can exploit poor memory management issues to access sensitive information, promulgate unauthorized code execution, and cause other negative impacts. “Memory management issues have been exploited for decades and are still entirely too common today,” said Neal Ziring, Cybersecurity Technical Director. “We have to consistently use memory safe languages and other protections when developing software to eliminate these weaknesses from malicious cyber actors.”

Microsoft and Google have each stated that software memory safety issues are behind around 70 percent of their vulnerabilities. Poor memory management can lead to technical issues as well, such as incorrect program results, degradation of the program’s performance over time, and program crashes. NSA recommends that organizations use memory safe languages when possible and bolster protection through code-hardening defenses such as compiler options, tool options, and operating system configurations. The full report is available here (PDF).

Read more of this story at Slashdot.

Wired Hails Rust as ‘the Viral Secure Programming Language That’s Taking Over Tech’

A new article from Wired calls Rust “the ‘viral’ secure programming language that’s taking over tech.”
“Rust makes it impossible to introduce some of the most common security vulnerabilities. And its adoption can’t come soon enough….”

[A] growing movement to write software in a language called Rust is gaining momentum because the code is goof-proof in an important way. By design, developers can’t accidentally create the most common types of exploitable security vulnerabilities when they’re coding in Rust, a distinction that could make a huge difference in the daily patch parade and ultimately the world’s baseline cybersecurity….

[B]ecause Rust produces more secure code [than C] and, crucially, doesn’t worsen performance to do it, the language has been steadily gaining adherents and now is at a turning point. Microsoft, Google, and Amazon Web Services have all been utilizing Rust since 2019, and the three companies formed the nonprofit Rust Foundation with Mozilla and Huawei in 2020 to sustain and grow the language. And after a couple of years of intensive work, the Linux kernel took its first steps last month to implement Rust support. “It’s going viral as a language,” says Dave Kleidermacher, vice president of engineering for Android security and privacy. “We’ve been investing in Rust on Android and across Google, and so many engineers are like, ‘How do I start doing this? This is great’….”

By writing new software in Rust instead, even amateur programmers can be confident that they haven’t introduced any memory-safety bugs into their code…. These types of vulnerabilities aren’t just esoteric software bugs. Research and auditing have repeatedly found that they make up the majority of all software vulnerabilities. So while you can still make mistakes and create security flaws while programming in Rust, the opportunity to eliminate memory-safety vulnerabilities is significant….

“Yes, it’s a lot of work, it will be a lot of work, but the tech industry has how many trillions of dollars, plus how many talented programmers? We have the resources,” says Josh Aas, executive director of the Internet Security Research Group, which runs the memory-safety initiative Prossimo as well as the free certificate authority Let’s Encrypt. “Problems that are merely a lot of work are great.”

Here’s how Dan Lorenc, CEO of the software supply-chain security company Chainguard, explains it to Wired. “Over the decades that people have been writing code in memory-unsafe languages, we’ve tried to improve and build better tooling and teach people how to not make these mistakes, but there are just limits to how much telling people to try harder can actually work.

“So you need a new technology that just makes that entire class of vulnerabilities impossible, and that’s what Rust is finally bringing to the table.”

Read more of this story at Slashdot.

Stack Overflow CEO Shares Plans for Certification Programs, Opinions on No-Code Programming

“We serve about 100 million monthly visitors worldwide,” says the CEO of Stack Overflow, “making us one of the most popular websites in the world. I think we are in the top 50 of all websites in the world by traffic.”

In a new interview, he says the site’s been accessed about 50 billion times over the past 14 years — and then shares his thoughts on the notion that programmers could be replaced by no-code, low-code, or AI-driven pair programming:

A: Over the years, there have many, many tools, trying to democratize software development. That’s a very positive thing. I actually love the fact that programming is becoming easier to do with these onramps. I was speaking at Salesforce recently, and they’ve got people in sales organizations writing workflows, and that’s low code. You’ve got all these folks who are not software engineers that are creating their own automations and applications.

However, there is this trade-off. If you’re making software easier to build, you’re sacrificing things like customizability and a deeper understanding of how this code actually works. Back in the day, you might remember Microsoft FrontPage [an early HTML web page editor] as an example of that. You were limited to certain basic things, but you could get web work done. So similarly, these tools will work for general use cases. But, if they do that, without learning the fundamental principles of code, they will inevitably have some sort of a limit. For example, having to fix something that broke, I think they’re going to be really dumbfounded.

Still, I think it’s important, and I’m a believer. It’s a great way to get people engaged, excited, and started. But you got to know what you’re building. Access to sites like Stack Overflow help, but with more people learning as they’re building, it’s essential to make learning resources accessible at every stage of their journey….

Q: Is Stack Overflow considering any kind of certification? Particularly, as you just mentioned, since it’s so easy now for people to step in and start programming. But then there’s that big step from “Yes, I got it to work,” but now “I have to maintain it for users using it in ways I never dreamed of.”

A: “It’s very much part of our vision for our company. We see Stack Overflow going from collective knowledge to collective learning. Having all the information is fine and dandy, but are you learning? Now, that we’re part of Prosus’s edtech division, we’re very much looking forward to offering educational opportunities. Just as today, we can get knowledge to developers at the right place and time, we think we can deliver learning at just the right place and time. We believe we can make a huge impact with education and by potentially getting into the certification game.

Q: Some of the open-source nonprofits are moving into education as well. The Linux Foundation, in particular, has been moving here with the LF Training and Certification programs. Are you exploring that?

A: This is very much part of our vision….

Stack Overflow’s CEO adds that the site’s hot topics now include blockchain, machine learning, but especially technical cloud questions, “rising probably about 50% year over year over the past 10 years…. Related to this is an increase in interest in containerization and cloud-native services.”

Read more of this story at Slashdot.