This week on his podcast Bill Gates asked Sam Altman how his team is doing after his (temporary) ouster, Altman replies “a lot of people have remarked on the fact that the team has never felt more productive or more optimistic or better. So, I guess that’s like a silver lining of all of this. In some sense, this was like a real moment of growing up for us, we are very motivated to become better, and sort of to become a company ready for the challenges in front of us.”
The rest of their conversation was pre-ouster — but gave fascinating glimpses at the possible future of AI — including the prospect of very speedy improvements. Altman suggests it will be easier to understand how a creative work gets “encoded” in an AI than it would be in a human brain. “There has been some very good work on interpretability, and I think there will be more over time… The little bits we do understand have, as you’d expect, been very helpful in improving these things. We’re all motivated to really understand them, scientific curiosity aside, but the scale of these is so vast….”
BILL GATES: I’m pretty sure, within the next five years, we’ll understand it. In terms of both training efficiency and accuracy, that understanding would let us do far better than we’re able to do today.
SAM ALTMAN: A hundred percent. You see this in a lot of the history of technology where someone makes an empirical discovery. They have no idea what’s going on, but it clearly works. Then, as the scientific understanding deepens, they can make it so much better.
BILL GATES: Yes, in physics, biology, it’s sometimes just messing around, and it’s like, whoa — how does this actually come together…? When you look at the next two years, what do you think some of the key milestones will be?
SAM ALTMAN: Multimodality will definitely be important.
BILL GATES: Which means speech in, speech out?
SAM ALTMAN: Speech in, speech out. Images. Eventually video. Clearly, people really want that…. [B]ut maybe the most important areas of progress will be around reasoning ability. Right now, GPT-4 can reason in only extremely limited ways. Also reliability. If you ask GPT-4 most questions 10,000 times, one of those 10,000 is probably pretty good, but it doesn’t always know which one, and you’d like to get the best response of 10,000 each time, and so that increase in reliability will be important.
Customizability and personalization will also be very important. People want very different things out of GPT-4: different styles, different sets of assumptions. We’ll make all that possible, and then also the ability to have it use your own data. The ability to know about you, your email, your calendar, how you like appointments booked, connected to other outside data sources, all of that. Those will be some of the most important areas of improvement.
Areas where Altman sees potential are healthcare, education, and especially computer programming. “If you make a programmer three times more effective, it’s not just that they can do three times more stuff, it’s that they can — at that higher level of abstraction, using more of their brainpower — they can now think of totally different things. It’s like, going from punch cards to higher level languages didn’t just let us program a little faster — it let us do these qualitatively new things. And we’re really seeing that…
“I think it’s worth always putting it in context of this technology that, at least for the next five or ten years, will be on a very steep improvement curve. These are the stupidest the models will ever be.”
He predicts the fastest technology revolution “by far,” worrying about “the speed with which society is going to have to adapt, and that the labor market will change.” But soon he adds that “We started investing a little bit in robotics companies. On the physical hardware side, there’s finally, for the first time that I’ve ever seen, really exciting new platforms being built there.”
And at some point Altman tells Gates he’s optimistic that AI could contribute to helping humans get along with each other.
Read more of this story at Slashdot.
