Introduction to Generative AI
Google | May 2023
- - -
Podium is a venture fund across sports and entertainment, based in Silicon Valley.
- - -
Podium is a venture fund across sports and entertainment, based in Silicon Valley.
Showing posts with label llm. Show all posts
Showing posts with label llm. Show all posts
Wednesday, May 17, 2023
Sunday, April 9, 2023
Potential of AI and ChatGPT with Sam Altman and Reid Hoffman
Potential of AI and ChatGPT
with Sam Altman and Reid Hoffman
April 2023
EPISODE TRANSCRIPT
Reid Hoffman:
I am here with delight with my friend, Sam Altman, who has assembled an amazing team at OpenAI and who has obviously made a lot of amazing and wonderful progress demonstrating a lot of the good things that are capable with AI.
One of the things I love about Sam (and I think is accurate) is he says, “Look, what I do is I help amazing people do their work and I bring a little bit of my own high ambition to help add some spice into the mix than we do.” And that’s the thing I loved about Sam and the whole team.
So Sam, welcome to the podcast. Thank you.
Sam Altman:
Thanks for having me.
RH:
You were, I think, the second person I talked to about writing a book with GPT-4, and the discussion came out of this [concept of], “Oh my god, it’s a human amplifier.”
Well, let me not just talk the talk, let me walk the walk – or write the write as the case may be – I could do a book that could be interesting on this. And you and the whole team’s perspectives on various things on AI kind of led me to do this.
And so I coincided it with the release of GPT-4 (because obviously, we didn’t want to pre-show the game), and it’s a little bit of the AI is amplification intelligence, or augmentation intelligence versus artificial intelligence, or an “AHA!” moment amplifying human ability.
Say a little bit about your view on this kind of amplification and augmentation.
SA:
I’m really curious, and I’ve been waiting for this to ask you. I realize it’s not a perfect example because you’re writing about GPT-4 itself in so many ways. But how much of an amplifier did it feel like? Did it make it 50% easier to write the book? Five times easier to write the book?
RH:
At least two times, and it might be more.
Let me work through an answer. One part of it was the huge issue – that is across all professions – which is solving the blank page problem.
SA:
Yep.
RH:
You’re writing a product resource document, marketing copy, legal brief, term sheet, medical analysis, commencement speech, and you go into a prompt, type something, and it generates something! And then all of a sudden you have something, a foil. It’s like that’s the collaborator, the co-pilot to work on. And that was huge because, for example, what I would do is say, “Okay, well how could AI really help education?”, and, “What would be the critics of what AI would do in education?” And you get both [answers], and then you can go, “Okay, let me think about them, let me put them again.” And that’s all done in two minutes! So maybe it’s 2X easier and 10X the speed, something in those dimensions.
You also have this thing where something occurs to you like, “Huh – Liechtenstein has this theory of language. Pure language is following a rule. Well, maybe GPT-4 would say something interesting about it, especially when mixed with something else.” Say you just read something about Chomsky linguistics or something else, and it gives you something interesting to say about it.
All of that I think was really helpful. And so it limbered it up and it helped a whole bunch in a way. Now, it still wasn’t just like “Press button, get book,” right?
SA:
Yeah, one of the things that has been most gratifying to see is the success people have had using this as a creative tool to get past that blank page problem, to get unstuck on something, and to generate a bunch of new ideas. Clearly, it is not a replacement for creative work in any way, but as a new arrow in the quiver. I think people have had surprising success in many different ways and it has been very fun to watch the breadth of human creativity in finding out what to do with this.
"One of the things that has been most gratifying to see is the success people have had using this as a creative tool to get past that blank page problem."
RH:
Have there been any particular pieces of creativity that have particularly surprised or delighted?
SA:
One of the things that I hear a lot is parents of small kids talking about how every night their kids just want to make up stories with GPT-4, and there seems to be an unending appetite for this from some kids. And at least hearing parents report back some children are able to far outweigh what some adults are able to do with coming up with really creative stories and just being fluent quickly with a new way to use this technology. So that has been interesting.
RH:
And to amplify it because I’ve heard the same thing. It’s like, we tell a new story every night and we kind of say, “What do you want a story on?” “I would like to have a dragon and a teddy bear and an astronaut on a ballad. And you’re like, “Okay.” And I know even with GPT-3, let alone GPT-4, people have written children’s stories (illustrating sometimes through DALL-E) and have published those.
So while mine – maybe by navigation of timing, may be the first co-authorship with GPT-4, there were already books being published with GPT-3, with DALL-E, and so forth.
So, to get to kids: We obviously haven’t had a storm of education commentary, most of which I find to be from having a real profound lack of imagination. What do you think?
SA:
I totally get why this technology is causing a lot of change and some problems for extremely overworked teachers. That’s all very natural and the adaptation is going to take some time. But on the upside, I think what’s happening is remarkable and exciting.
There are a bunch of boring use cases of GPT-4 that I really love– translation, summarization, information access. I’ve come to rely on it for many categories – but I think the deepest thing that I have found so far with it is the ability to learn new things better than Wikipedia, which was my current leading way of learning something new fast. And I and many other people love the style of interactive chat with a sort of “AI tutor lite” to learn something new. And I am hopeful that this will be a great and new education tool.
RH:
I mean, it’s good for you to have that balanced perspective given that you’re driving the creation.
I am certain that it’s not just [a matter of] different students learning at different paces. Having an AI tutor who might be able to tie it to your interest, if it’s polar bears or your interest, if it’s climate change or your interest if it’s kind of Greek goddesses or whichever. But also kind of going at your pacing, being able to do stuff.
But there are other things too. For example, teachers go, “Well, wait a minute, it writes the essay.” [That’s just] a cognitive lack. And there is one thing is if you say, “Look, it’s really easy. I want to do an essay on Jane Austen and colonialism.” I use GPT-4, I generate eight essays. I hand those essays out this high school probably, maybe college, and I say, “This is a D plus, do better.” But they can use that as a basis to get better.
And this is the real pitch to the teachers. You can now use GPT-4 to help you grade. So as opposed to spending 60 hours on grading, you can spend three hours on grading and the other 57 hours with the students helping them. And that kind of thing is just like it’s there now.
SA:
Yeah. I mean, I think there are probably some concerns with using these systems for grading, but I also think we will find ways to make it okay and work in many ways. And that idea that you can free up whatever that comes out to 95% of your time for something, I think that’s a concept we hear again and again with the use of GPT-4.
RH:
In the speed of writing the book, we got the first chapter’s education to try to understand how we can help elevate all of us because we are all ideally ongoing through education, but of course, especially children and all the rest. And also re-skilling when you get to work. But also medicine. Say a little bit about the kind of line of sight to an AI tutor and an AI doctor on every cell phone or every smartphone.
SA:
Certainly, an exciting world to get to.
Through this lens of human amplification, the idea that we can have doctors that can provide a higher quality of care, and care to many more people using these tools to automate parts of it – but keep humans in the loop. There are a lot of companies now working in that direction. I think it’s rightfully going to be a long time before most people want a full-on AI doctor. But this hybrid approach where this is a tool to make our doctors much better and we can have super high quality, affordable medical care available to a lot more people seems tremendous.
On the education front, it’s obviously going to get so much better, but we’re sort of already there. People are now used to their phones being able to tutor them on things that they need, which is amazing.
RH:
And part of the thing that I think is kind of great about the API framework and learning is, are you seeing people doing this medical and education stuff already? What are you seeing that is already in development?
SA:
We are, yeah. One of the most fun parts of the job is getting to meet with developers and hear about what they’re already doing and what they’re planning to do. And certainly, medical advice and tutoring are frequent and extremely exciting areas.
RH:
Did you have any reflections on the Impromptu book, things that I should have worked more on, or themes I should have developed more on or things that were worth people especially paying attention to?
SA:
No, I had a very interesting response to it and I’m not sure if it’ll be the same way that other people respond to it. But I thought it was great. I think it raises a lot of issues.It was this weird sort of study of someone else studying something that I had been so close to for so long and I was like, “Oh man.” It’s like looking through fresh eyes at something which I have seen all of the parts that don’t work, all of the problems, and all of the kind of limitations. And you did a great job of still highlighting those (and to some embarrassment in some cases). But it was a very strange experience I think for me and for others who work at OpenAI in a way that is maybe different for people who didn’t watch the process of creation.
RH:
Yeah, no, and look, I saw the process of creation from more of a remote kind of helping. I will seek your feedback on how to continue to improve to help because I think what the team is doing is amazing.
Let’s go to the work section. I know that one of the things you guys are super directed on is how do you have a very positive impact on the world of work? What are you currently thinking about what this amplification means for the kind of next 10 years-ish, five years-ish – because all the impacts on work will take time – on what’s going on? And how should people think about amplification as a path for themselves, and a path for their companies?
SA:
I always try to be honest and say in the very long term, I don’t know what’s going to happen here and no one does. And I would like to at least acknowledge that. In the short term, it certainly seems like there was a huge overhang of the amount of output the world wanted. And if people are way more effective, they’re just doing way more. We’ve seen this first with coding and people that got early access to [Microsoft] Copilot reported this, and now that the tools are much better, people report it even more.
But we’re now in this sort of GPT-4 era, and seeing in all sorts of other jobs as well where you give people better tools and they just do more stuff, better stuff, and amplifying your ability.
I’m trying to remember how I heard this from (I’m pretty sure it was from Paul Graham), there is a huge cost premium on work that has to be split across two people. There’s the communication overhead. There’s the miscommunication. There’s everything else. And if you can make one person twice as productive, you don’t do as much as two people could do. Maybe you do as much as three and a half or four people could do and for many kinds of tasks. And I think one of the things that we are seeing with this GPT-4 application is that.
RH:
Yep. And also, I mean, that’s like the earlier book thing. You got a blank page problem. You have a lot more rote work amplification. So it’s like, how do you slot in to be more effective? And when people say, “Well, but wait, that really changes.” It’s like, well, mathematical intelligence used to be how much calculation can you do, and a calculator changed that. But that’s fine, we
don’t do the calculator or memory. Intelligence used to be memory and now you can use search on the internet in order to do it.
So I have confidence that our intelligences can tune to where the creatives add versus anything else, even for us – speaking more for myself than you – “older folk.”
SA:
I’m old enough now. You notice that. You notice it when it comes to… You get in these crazy work periods and the late nights get a little bit harder, or at least not as easy.
RH:
Yeah, indeed. And it doesn’t get easier, speaking from a generation ahead of you.
One of the things that’s on the work topic, people kind of worry a lot about.This is probably the most tangible thing that most people go to. Part of the reason for writing the book was to say, “Look, there are so many amazing things we can get to. We can get to an AI tutor for every person, every child. An AI doctor, for people who can’t afford medicine or people who don’t have access to or companionship with a doctor, et cetera, et cetera.”
But everyone worries about it because they worry about what happens with my job. And part of what I wanted to show was, “Look, here’s an instance of using it, where a smart, knowledgeable person is using it for actual work, and the work is writing this book, right?
SA:
Yeah, I love that you did it. I think it’s so tangible in it. It made me want to write one too.
RH:
I look forward to it. I will be one of your dialogue participants.
SA:
Great.
RH:
What do you think about the use of GPT-4 for the information problem? And I actually don’t think we’ve specifically talked about this. I mean, obviously there’s a lot of criticism saying, look, it’s a generative AI. It’s very good at generating, the generation is not necessarily truthful. It can be targeted in any direction which includes misinformation. Hence, we have an amplification of social media and other kinds of things that are a problem and therefore, “Oh my God, this is an assault on journalism.” I don’t agree with this point of view. Although, I do think one has to work to make it good.
What are your working thoughts about how to navigate this in a way that’s collective learning- positive and collective truth-positive we facilitate?
SA:
One of the areas that we most need to figure out is how to get these models to be super reliable while not losing the creativity that people like, or giving settings for when you want one or the other. We don’t have the answer here yet. We have a bunch of ideas and we’re able to make week-over-week improvement in the problem. But telling you when we’re going to be able to
really get to a system that is accurate to what you as the user want – I carefully use that instead of saying a system that is true, because that’s really hard. I think we don’t have an answer. We’re making progress. We need some new ideas and it’ll get better.
"One of the areas that we most need to figure out is how to get these models to be super reliable while not losing the creativity that people like."
RH:
I think one of the things that is contained in your answer I think is really important is the dynamics of learning. So if you said, “Well, we have an agreement about what kind of social model of truth or facts is that we update and so forth.” Let’s make it easy and say, “Well, it has to do with science and facts and other kinds of things,” versus even grounding in some kind of truth process, which is usually collaborative human study. Well, easy enough to take to train an agent like that, put it in the browser, and then, “Hey, I’m like this thing.” You hear the earth is actually flat or the moon is made out of blue cheese and it says, no, no, no, you should look over here for good information.
SA:
Look, those things, I think we can get right.
The following example that I’m going to give, I don’t even know if it’s true or not, which illustrates how hard this problem is. Maybe I just heard it repeated by people so often that I now believe it’s true. But I think that there was a time where some of the social media platforms were banning people for saying that COVID may have leaked from a lab and that you can’t say that, it’s xenophobic, it’s racist, it’s hateful, it’s inaccurate, it’s fake news, whatever. I think people were just having posts about that deleted, and then all of a sudden the kind of elite opinion on that changed. I remember there was one article that came out that seemed to really change it and you see this big backtrack there, but you literally weren’t able to discuss it in the channels that are the de facto important channels now.
And that’s the kind of thing where if you let someone dictate the truth, I think you do something dangerous. But there are lots of easy examples like the earth is not flat. The Holocaust did happen. I think we can make a system where we’re willing to make editorial calls on those two. But there is a lot of other stuff where I think all of the trickiness comes in.
RH:
Well, I chose easy examples. I noticed you didn’t dispute the moon is made out of blue cheese. Maybe you haven’t –
SA:
I’ll agree. I’ll agree that the moon is not made out of blue cheese too. Yes. I haven’t personally been there, but it seems like a safe bet.
RH:
Yes. I actually think how you can train these models is you say, “Well, it doesn’t have to yield an opinion on everything.” It doesn’t have to yield an opinion on controversial topics or places where humans. For example, one training answer is, “Oh, on this topic there’s a lot of human dispute.”
SA:
And we give those answers a lot. We are now experimenting with more steerability of the models where a user can say a lot more about how they want the model to behave in different ways.
And so if you want a high degree of careful calibration, the model can do that. And if you want something else, the model can do that too. If you want the model in a very creative mode where it tells you a story about astronauts going to the moon and finding that it was made of blue cheese, there could be good reasons for that. That could be a fun sci-fi story, right?
RH:
Yes.
SA:
And the whole one-size-fits-all, I think people are finding more and more, it doesn’t quite work. Things they think they want a model to do, or not do, or never do doesn’t always apply and it’s uncomfortable.
But giving the user a lot more control is something we think is important. And if you want that story about the astronaut going to the moon and being sure it was made out of blue cheese, in some context, that’s totally fine. But it’s telling you something that is clearly not a fact in some other sense.
Can I tell a quick story?
RH:
Yes, of course.
SA:
It’s totally unrelated to anything, but I just think it’s one of the most beautiful stories. I don’t know why I just thought of this right now. I guess it is because we are talking about the moon. There was this astronaut who talked to a reporter decades after going to the moon. And they kind of asked him what it was like and he said, “You know, I kind of forget about it. And then sometimes I’m out in my backyard at night looking at how beautiful the moon is. And I just remembered like, ‘Oh man, all these years ago I went there and walked around on it and now I forget.'” And I think that is just an amazing story.
But I also think when we kind of look back at this age of creating the first AI systems, we’re all going to have our own version, all of us that have been involved and are going to have our own version of that.
RH:
Yeah, no, I completely agree.
Basically, the last chapter in the Impromptu book was kind of my very strongly held belief – but also argument –is that we evolve our humanity through our technologies. We evolve our humanity through the clothes we wear.
SA:
I love this part.
RH:
Yeah, yeah. Chairs, we sit in. The podcast mics, we talk to each other across video conferencing, all of this stuff. There are a number of major scale problems facing the world: climate change, economic transformation, and elevation of humanity, people adapting to the new world, whether it’s education skills, et cetera, social justice of various forms. I know one of the ones you and I talk a lot about is criminal justice and having fairness across this society. Are there any that you think that AI is not a particularly good fit for? And then after that, what’s a surprisingly good fit where people, it had probably hadn’t occurred to them?
SA:
So I think technology is the fundamental ingredient in making the world better in all of these ways. But technology on its own doesn’t always or doesn’t usually do it. You need society. You need the participation of people. You need the kind of contributions of all of our institutions, and what has made this – from the enlightenment-until-now era – so impressive coming together with technology, and also to enable the discovery of and creation and distribution of such technology to make this work.
If AI can start churning out new scientific discoveries, that will be wonderful. And that’s personally one of the things I’m most excited about. But even to get those into the hands of people and to do it in a rapid and just way, you need all of the rest of society to play its role too.
And so I don’t view this as AI is just going to come along and wave a magic wand and fix everything. It’s like AI is this new meta-tool that humanity has to enable us along with all of our people, institutions, whatever, to go to greater and greater heights.
"I think technology is the fundamental ingredient in making the world better in all of these ways. But technology on its own doesn't always or doesn't usually do it. You need society."
RH:
And what’s like… For example, is it climate change? Is it the opposite of poverty?
SA:
If we can build AI systems that help us figure out the cure to many diseases, that’s a pretty big triumph. And this is not my field of expertise by far, but the people that are in it seem so excited about what this is about to unlock.
RH:
Yep. I completely agree. Because by the way, diseases are broadly pretty indiscriminate across rich, poor, gender, race, et cetera.
What do you see in terms of AI as a platform? You just recently did Chat GPT as an API with some announced partners and other kinds of things. How should people think about the platform as they are navigating their lives?
SA:
I think the greatest platforms, people pretty quickly forget are platforms.
RH:
Um hmm (affirmative)
SA:
So there was, for a while after the first smartphones and app stores launched, all these companies were saying, “I’m a mobile company.” And that was a big deal because they were on the mobile platform. And today it would be ridiculous to say you’re a mobile company because every company has a mobile part of its strategy.
And we’re going through a moment right now where everyone is talking about AI and AI is the platform of the future (probably true), all these AI companies, whatever. My hope is that 10 years from now, intelligence is expected in every product and service and it’s so ubiquitous, we forget it’s a platform. It’s just part of everything we do.
RH:
And what do you think about the probability that, in addition to the kind of large language models – scaling, and refining them – there is another amazing major component that will be discovered and/or deployed in the next three to five years?
SA:
High, high. Well, “discovered” – high. It may take longer. But my belief is we need at least one more really big idea.
RH:
We have journalism as a chapter because we want to kind of show the actual positive impact on crafting perspective and news for people. We as human beings share things with each other as part of the reason we value journalists, even though some people like their flavor of journalists better than other people’s. What are the things that you like, how would you tell a journalist to
use GPT-4 today? This is how it helps you.
SA:
I have talked to a few journalists since the GPT-4 launch and I would ask this question, because I’m always curious to know more. [GPT-4] is not useful at all, at least in the ways people have found so far, for the reporting process. But it’s very useful for the drafting, writing, and synthesizing part of the process and the ability to put a bunch of notes in there, ask questions about it later, help kind of find themes between things. People have found a lot of use there.
RH:
That’s interesting. And you referenced Wikipedia before. But it’s like a mini research assistant that gives you a mini brief right away. Not always a hundred percent accurate, especially when you get to very specific details on very specific questions. When I asked it, “Did Reid Hoffman make a knockoff of Settlers of Catan? And if so, what was it?” It said, “Yes, he did.” Which is true. I made a game called Startups Silicon Valley, which I only give to my friends who’ve already bought Settlers. You have a copy.
SA:
I do.
RH:
At least one. But then it said the game [I invented] was Secret Hitler, which is actually in fact a board game made by the Cards Against Humanity people. I’ve heard it’s quite funny, but not a game that I’ve even cracked the box on yet, let alone co-designed. And so I would say, “Look, crosscheck this stuff,” but the quick research assistant briefing that is done immediately in 30
seconds is I think actually part of it. It is not a, “Tell me what I should say about Sam Altman or what question I should ask.
SA:
That makes sense. That makes sense, yeah.
RH:
And that’s part of the use of some imagination. Again, part of the reason why the Impromptu was kind of this travel log across these domains of human existence.
SA:
Oh, that’s a great phrase for it. That is what it felt like to read.
RA:
Yes, exactly, it’s just like when you start encountering search. When you play with Chat GPT or GPT-4, try something and literally think to yourself, “How do I try something that may be different than I’ve ever thought about trying before?”
The one that may seem pedestrian that surprised me was the, “Well, I got these seven ingredients in my refrigerator, what should I make for dinner?” And it would just never occur to me to ask that question. You certainly wouldn’t put it in the search engine. You wouldn’t put it into Bing or Google or whatever.
And here it’s like, “Well you could do this, you could do this, you could do this. Or are you in the mood for… With those seven agreements, are you in the mood for Thai or are you in the mood for Mexican?” You’re like, “Oh, all right.”
One of the things that I think is (too often) not said about the transformation of work that AI and making intelligent tools is going to be, is by moving it out of more of the rote stuff. You can also not just make it into more creative stuff, which some people are fearful of, but also the more joyful parts of work. And I think that that transformation of joy is again, amplifying human beings.
"My belief is we need at least one more really big idea."
SA:
I deeply believe this. It’s a little too early to declare victory on it, but it certainly seems like people stay in the flow state much more and stay in the parts of the job they enjoy much more. AI is good at doing the repetitive stuff that most of us just find a little bit monotonous.
Anecdotally, we hear reports of this a lot.
RH:
Yeah, well, and actually I did that because for example, part of usually in writing books, having done it before, yu, get to periods where you’re just slogging through and getting it done. And I’d say, “Look, I have a break.” I kind of do a prompt to discover something kind of fun and interesting. I’d think, “Hey, let me take this section of the essay that I’m doing and let me ask you before to make it into a sonnet, or make a ballad about this.” And I look at it and sometimes I go, “Okay, that was cool,” and put it aside. Sometimes I keep a copy. Sometimes I go, “Oh, that was kind of interesting,” and that’s keeping me limber and fresh versus the, “Ah, I just have to figure out the next sentence.”
Everybody has been sweating blood in order to bring it to make it available for people. Is there anything that you use GPT-4 to your delight?
SA:
I mean, I mentioned this before, but it is still so delightful to learn something new and to feel like I just got a cheat code is very delightful.
Another one is we just released this very early preview of plugins and I think the fact that GPT-4 can now, with a lot of limits, write and execute code is so cool that just as a citizen of humanity, I feel delight every time, even if it’s just doing something very simple.
RH:
One thing I think it’s probably important that we haven’t gotten to that I don’t think can be said often enough is what you guys are learning about how to align these systems with human interest and human values as they scale.
SA:
It’s nice of you to say. I think we have made real progress relative to what people thought this was going to look like.
But we do not know and probably aren’t even close to knowing how to align a super intelligence. And RLHF is very cool for what we use it for today, but thinking that the alignment problem is now solved would be a very grave mistake indeed. I am hopeful that we’re going to make better and better tools that are going to help us come up with better and better alignment ideas.
RH:
I think that answer is exactly right because staying deeply focused and deeply concerned as the dynamic thing proceeds is the way to maximize safety.
SA:
Well, I think it’s worth being very precise here. What we do is we know how to make GPT-4 safe; safe in the ways we know about so far. And again, big surprise, big success, but very… A lot more work in front of us. And thankfully, we now have a new tool to help us.
RH:
Yeah, I think people should take heart and positive energy out of this… That’s the right way you should be responding. You shouldn’t be, “Oh no, no, it’s no problem. We’re just cruising down the road. It’s no problem.” This is [a case of] “No, we’re paying a lot of attention. The problem will require more work to improve the solutions we have, the tools we have can continue to improve it. And by the way, so far as we scale, we are learning better ways to use the tool to help us align better. Even though there are a lot of open questions and a lot of journey we are going ahead.”
And this is part of the thing is when people look at a dynamic and future with change, they always tend to go, “Oh, what about the worst case?” And you’re like, “Okay, well what about the best case?”
SA:
Yeah. One thing that I think is great about people using GPT-4 and Chat GPT is they see, they feel the upside in a way that if you only hear about it, you discount and you really feel the downside. Now, the downside is still really there, but using it I think adds some healthy caution to people too. But it really does let people taste the upside and figure out how to participate in this new world we are all going to create together. And I think that’s great.
RH:
Yeah. The parallel that I see is that when you ask someone who is part of the creative industry,
RH:
Yeah. The parallel that I see is that when you ask someone who is part of the creative industry, writing, painting, photography, whatever, and they haven’t had access, they haven’t tried these tools, they’re always like, “Oh my God, is it going to take my job away?” Is it now going to be the journalist, the author, the novelist, the painter, et cetera? Then when they play with it, they realize, “Oh my gosh, this could be a tool that really amplifies what I’m doing,” And they begin to see the upsides. And that’s precisely why like I created Impromptu (also with the pun in the name, I am prompting you) as part of –
SA:
Incredible name. Ten out of 10.
RH:
Thanks. It’s to say “Try it,” right?
SA:
Yes.
RH:
It’s a little bit like if someone described a car to you if you weren’t already familiar and was like, “Well, it’s a two-ton death machine that can run over children, it causes 40,000 deaths per year in the continental United States. Wouldn’t you like to get into one?” Right? They’re like, “What?” And then you go for a ride and you’re like, “Whoa, this is cool, it transforms space and I can do all these things!”
And it’s that kind of thing – try it and realize what all the options are. And by the way, of course, you still have to be attentive to safety. You’ve got to go, “All right, how do we make these things safer? How do we put bumpers and automatic crash detection and other kinds of things in it? Lane changes and so forth in order to make them a lot safer, decade after decade in turn.”
And that’s what I think the AI stuff is too.
SA:
For sure.
RH:
You have obviously been over the last couple of weeks kind of helping introduce the world to Chat GPT, GPT-4. Impromptu is an effort to try to show why AI can be amazing for humanity and why I just have belief and confidence that we really can get there just by intentionally (metaphor) driving it there. What would you say that you’ve learned in the last couple of weeks
that you would want to share, as our final thought, with listeners to a podcast about what we’re trying to get to with AI in the next few years and the next decade?
SA:
You touched on it at the very beginning. So it’s a great thing to close with. This idea that this is a tool of amplification and this is a tool that makes you do better at whatever you need to do better at. We can say that. It’s nothing like just trying it. So just go try it out. That’s my advice. We really hope you love it.
The feedback that we get from people is very helpful as we think about future versions. People really have found new incredible things, new places where it breaks. But yeah, the ability to,… In whatever way you would like to be amplified, use this tool to amplify yourself. I think that’s the message that I would love to end on.
RH:
Absolutely. And it’s the reason why the book description from Impromptu is it’s not a read, it’s the start of a conversation.
And so with that, Sam, thank you for this conversation and as always, look forward to talking to you soon.
SA:
Me too. Thanks, Reid.
Video and transcript courtesy GreylockVC
* bonus: visit this link to get Reid Hoffman's "Impromptu" book (pdf) for FREE
- - -
Podium is a venture fund across sports and entertainment, based in Silicon Valley.
Saturday, April 1, 2023
Google CEO Sundar Pichai on AI, Bard and ChatGPT
Google CEO Sundar Pichai
on AI, Bard and ChatGPT
March 31, 2023
For years, Google was seen as one of the most cutting-edge developers of A.I.
But, with OpenAI’s release of ChatGPT, and other chatbots beating Google to market, is that distinction still the case?
Google’s chief executive Sundar Pichai is in an unenviable position: Scramble to catch up or, in the face of potentially harmful technology, move slowly.
Casey Newton and Kevin Roose discuss AI and are joined by Sundar Pichai at 9:34 into the podcast.
- - - transcript - - -
So as of last week, Bard, Google’s effort at building consumer-grade AI, is out in the world. And I think it’s fair to say the early reviews were not amazing. And I sort of imagined that we would discuss that at a really high level this week. But then last week, I got a phone call.
And someone I know at Google said, would you maybe want to talk this week to Sundar Pichai? And I said yes.
That didn’t take you a lot of deliberation on that one.
[MUSIC PLAYING]
I’m Kevin Roose. I’m a tech columnist at “The New York Times.”
I’m Casey Newton from “Platformer.”
And you’re listening to “Hard Fork.” This week, we hit the road and take a trip to the Googleplex to talk to Google’s CEO, Sundar Pichai.
[MUSIC PLAYING]
So last week, we talked about Google’s new chat bot called Bard, which is supposed to be their answer to ChatGPT and some of these other generative AI chat bots. And I think it’s safe to say that the reaction among the public to Bard so far has been pretty lukewarm. My Twitter timeline is not full of screenshots of Bard conversations like it was of ChatGPT conversations late last year when that came out. It doesn’t seem to have landed with nearly as big a splash.
It was a little muted. You know, I think by this point, a lot of people have tried chat bots. And they feel like ChatGPT in particular gives really good results. And I think when people put these things through their paces, a lot of people felt like, I’m not sure if Bard is as good.
Right. And that kind of fits with this narrative that has been developing in the AI community over the past year or two, which is that Google is somehow behind in this race for generative AI. They’ve been working on this stuff for a long time. Google certainly had a dominant position in AI research for many years. They came out with this thing, the Transformer, that revolutionized the field of AI and created the foundations for ChatGPT and all these other programs.
But then the perception, at least, is that they kind of fell behind. A lot of their researchers left and did their own startups or went to competitors. They didn’t really turn their research into products at a pace that people could actually use and appreciate. And they got sort of hamstrung by a lot of — to hear people inside Google tell it — big company politics and bureaucracy. And I think it’s safe to say that they got sort of upstaged by OpenAI.
With the release of ChatGPT last year, it seemed to catch Google off guard. And so in December, just a month after ChatGPT came out, my colleagues at “The New York Times” reported that Google’s management had declared a code red.
Yeah. And look, if you’re a business and you’re developing a lot of amazing technology, and no one else out there has released similar technology, that gives you a good reason to stay quiet and to not release it, right? We know that there are real safety concerns. There’s responsibility issues, ethics issues, regulatory issues.
Google actually did have a lot of good reasons to kind of sit on its hands. But then OpenAI forced its hand, and in a way that makes me wish I had used hands in a different way in the previous sentence because now I’ve just said hands too many times. Anyways, I love how dramatic a code red sounds. Makes you think like, what, are employees chained to their desks 24/7?
My understanding is that what it meant for a lot of people there was all of a sudden, the goals that you had to get your next promotion, get your bonus, they were tied to whether you hit some goal related to AI. And the question is, is that going to get them where they want to go, or is it going to be a moment where they act a little panicked and they make a lot of mistakes?
Yeah. And I think that Sundar Pichai is in a really fascinating and tough position here as the CEO of Google. And I think it’s fair to say that they are more threatened than they have been in a very long time.
That’s right. And Google has been a relatively conflict-averse company for the past half decade-plus. They don’t like picking fights. If they can just keep their heads down, quietly do their work, and print money with a monopolistic search advertising business, they’re happy to do it.
Totally. And they also have this other problem, which is kind of a classic problem in business, which is the innovator’s dilemma problem. This is a term that was coined by Clayton Christensen a long time ago. It’s used to talk about the dynamics between new startups that enter a market and the incumbents in that market.
And basically, Google is in the position of an incumbent. It has this huge, profitable search business that it doesn’t want to sort of diminish or leave behind in any way. At the same time, it’s got OpenAI and now Microsoft, which is partnering with OpenAI, who are potentially eating into their search business using these generative AI tools. And so they have to somehow figure out, how do we capitalize on generative AI without destroying our own search business?
Sometimes as a business journalist, you look at a situation, and you say, well, I know exactly what I would do there. But when you present me with the problem that Google has right now, which is, how do you introduce generative AI and not blow up our whole search advertising business, that seems like a very tricky problem to me.
But I am not as certain as you are that there is a real existential risk here, although there might be. I do think that there is a real generational opportunity, though — that if they figure this out, there is a chance that they become an even more enormous company than they are today. Google plays a huge role in my life. That’s where my email is. That’s how I get around town. It’s how I waste hours of my life on YouTube.
And when they introduce these generative tools across their entire suite of products in ways that we haven’t even imagined yet, there’s going to be enormous opportunity for them both financially, but also to kind of set the pace again. And so I think one of the big questions heading into this interview is, is Google truly slow because of the nature of huge companies being unable to be nimble, or have they truly been trying to be safer and more responsible from some of their peers at a time when a lot of really smart people are starting to ring alarm bells and saying, this stuff is moving awfully fast, and we’re not sure that you’ve done all the safety work that you need to?
Right. Was your homework late because you were taking extra time to make sure it was good, or did you just decide to go to the club and not do your homework? That’s a horrible analogy.
Well, I decided to go to the club.
So we have a lot of questions. And I talked with Google last week. They said that Sundar would sit down with us and talk about these questions and more. So you and I are going to take a road trip to ask the man himself.
We are. Are you driving?
You’re driving, I’m hoping.
I’m driving.
Are you driving?
Yeah, I’ll pick you up.
Oh, that’s fantastic.
I’ve got to clean my car.
Yeah. I’ll send you the link on Google Maps.
[MUSIC PLAYING]
Either way, that’s my water or someone else’s water. Oh, there’s my water. Thank you.
All right. Sundar Pichai, welcome to “Hard Fork.”
Great to be here. Thanks for having me.
Yeah. So Sundar, I have spent a lot of time talking with AI chat bots recently, including Bard. And I have learned —
Welcome to the club.
Yes.
I have learned that one way to get really good responses out of these AI chat bots is to prime them first. And one way to prime them is to use flattery. So instead of just saying, write me an email, you say, you are an award-winning writer. Your prose is sparkling. Now write me this email. So I’ve always thought like, I wonder if that strategy works on humans, too. So I thought we should start today by saying, Sundar, you are a brilliant technician — technical thinker, a genius answerer of podcast questions, and you’re going to respond to all of our questions with brilliant insight and wit today and not pre-rehearsed talking points.
How did I do?
Oh, it kind of worked, I think.
OK, good, good, good, good. OK, good. So speaking of AI chat bots, Bard came out a little more than a week ago, was released to the public. And Casey and I have been playing around with it. I think it’s fair to say that the reaction among the public to Bard has been somewhat muted. Some people are saying this is not as good or it’s not giving me the same kinds of answers as ChatGPT or other products on the market.
And I guess I’m curious how you’re feeling about it at launch a week-plus later. And what have you made of the response to Bard so far?
We knew when we were putting Bard out we wanted to be careful. It’s the beginning of a journey for us. There are a few things you have to get right when you put these models out. Getting that user feedback cycle and being able to improve your models, build a trust and safety layer turns out to be an important thing to do. Since this was the first time we were putting out, we wanted to see what type of queries we would get. We obviously positioned it carefully.
It was an experiment. We tried to prime users to its creative collaborative queries, but people do a variety of things. I think it was slightly maybe lost. We did say we are using a lightweight and efficient version of LaMDA. So in some ways, we put out one of our smaller models out there, what’s powering Bard. And we were careful.
So it’s not surprising to me that’s the reaction. But in some ways, I feel like we took a souped-up Civic, kind of put it in a race with more powerful cars. And what surprised me is how well it does on many, many, many classes of queries.
But we are going to be training fast. We clearly have more capable models. Pretty soon, maybe as this goes live, we will be upgrading Bard to some of our more capable PaLM models, so which will bring more capabilities, be it in reasoning, coding. It can answer math questions better. So you will see progress over the course of next week.
And to me, it was important to not put a more capable model before we can fully make sure we can handle it well. We are all in very, very early stages. We will have even more capable models to plug in over time. But I don’t want it to be just who’s there first, but getting it right is very important to us.
Yeah. And look, we have plenty of questions about the AI safety stuff. But I also want to talk about the opportunity. The thing that is different about Bard compared to some of these other chat bots is that it’s connected to Google. And so much of my life is in Google.
If you let me, I would plug Bard into my Gmail right now, just to see what it could do. Would you do this?
I might, yeah.
Yeah. Like, I would love it to just kind of start drafting my emails. But how do you hope this stuff transforms some of these products? And how long do you think it’s going to take to get to somewhere like that?
You can go crazy thinking about all the possibilities, because these are very, very powerful technologies. I think, in fact, as we are speaking now, I think today some of those features in Gmail is actually rolling out now externally to trusted testers — a limited number of trusted testers.
Do you trust us? Because we would love to try.
Oh. Maybe. We can talk maybe after this, yeah.
OK, good. Good, good, good.
So now it’s basic. You can kind of give it a few bullets, and it can compose an email. You can say — you can choose the style of the email, et cetera.
But you’re absolutely right. We want to figure out in a safe, privacy-preserving way, to fine tune this on your data. The enterprise use case is obvious. You can fine tune it on an enterprise’s data so it makes it much more powerful, again with all the right privacy and security protections in place.
But I think, wow. Yeah, can it be a very, very powerful assistant for you? I think yes. Anybody at work who works with a personal assistant, you know how life changing it is. But now imagine bringing that power in the context of every person’s day-to-day lives. That is a real potential we are talking about here. And so I think it’s very profound.
And so we’re all working on that. And again, we have to get it right. But those are the possibilities. Getting everyone their own personalized model, something that really excites me, in some ways, this is what we envisioned when we were building Google Assistant. But we have the technology to actually do those things now.
How are you using generative AI tools like Bard, like LaMDA, like PaLM in your own life?
It’s interesting. My journey — maybe it was two years ago when we started playing around with LaMDA. We were getting ready to put it out in I/O and talk about it. The way we primed it was imagine you were Pluto as a planet.
And I remember playing around with my son at home talking to LaMDA back and forth. And there were a couple of conversations you really got deeply into it being Pluto. Because Pluto is far out in space, it became really lonely. And you kind of can anthropomorphize some of this experience, like probably what you went through.
And so it was fascinating to see, kind of unsettling a bit, so —
Did Pluto try to break up your marriage?
Not quite. But you know, I felt sad at that point talking to it. But I think the area where it shines the most is asking questions. Like, my dad is about to turn 80. And I was like, hey, what do I do with my dad on an 80th birthday?
It’s not that it’s profound, but it says things and kind of sparks the imagination. In my case, it said, you should make a scrapbook. And I was like, great. It’s not that — you know, but it’s fine. It kind of oriented me a particular way.
So asking questions in which — I think there are two categories where it works well, where it’s fun, creative, imaginative. You’re just kind of looking to spark some stuff. Hey, what movies can I watch on a Friday night? It says things different from what I find in other places, so sometimes movies I haven’t heard of. And I can iterate that way.
Sometimes, it’s good, if you understand the area well, where you can tell the difference between what’s real versus not. You can kind of play around back and forth with it because you are able to parse —
Right. You can fact check the chat bot.
Yeah, with your context. In those cases — but it also goes in certain directions, which can again inspire you. So that’s what I find fun, yeah.
I want to ask about that example because I’ve been using these technologies in the same way. It strikes me that, “what should I do for dad’s birthday?” is a question that you also could have put into Google. And when you rolled out Bard, the company was careful to say, this is not a replacement for search. And in fact, we’ll show you a Google It button underneath the box. But in practice, I find that they’re really good for a lot of queries that I might have previously used a search engine for. So as somebody who runs the biggest search engine, how do you feel about that? And also, are these things just going to kind of merge over time?
It’s exciting in the sense that from a user standpoint, it expands the possibilities of what you can do. So you can do more. I think these models will get more capable. So we’ll follow the user journey here. And I think people will evolve over time.
I do think people originally come in and try a lot of these queries, et cetera. But over time, I think they kind of adjust their behavior a bit to what the models can do. So I think time will tell. But for me, it’s exciting, because in search, we have had to adapt when videos came in.
And today, you can make the same case. People go to YouTube and look for all kinds of things. Like, how do we think about it? Like, great, people are looking for information. So to me, it looks, so far, from a zero-sum game, because it’s such early stages of a new technology.
And I think the best way we can approach it is really embrace it. We’ve been working on this for a long time. I view this as an iterative experience with users. We’ll put stuff out. They will tell us what they want.
So for example, in Bard already, we can see people look for a lot of coding examples, if you’re developers. I’m excited. We’ll have coding capabilities in Bard very soon, right? And so you just kind of play with all this, and go back and forth, I think. Yeah.
I want to talk to you about the race that is shaping up in AI right now. So in September of last year, you were asked by an interview who Google’s competitors were. And you listed Amazon, Microsoft, Facebook, sort of, all the big companies — TikTok.
One company you did not mention in September was OpenAI. And then, two months after that interview, ChatGPT comes out and turns the whole tech industry on its head and sets off all this competition among other tech companies to sort of match their progress. Did OpenAI and ChatGPT catch you by surprise?
Well, first of all, I’ve always assumed it’s a certainty, that with all the innovation around, there will be things which emerge out of nowhere. It’s always been true, and so on. I actually don’t think, with OpenAI, we had a lot of context. There are some incredibly good people, some of whom have been at Google before.
And so we knew the caliber of the team. So I think OpenAI’s progress and surprises — I think ChatGPT — you know, credit to them for finding something with a product market fit. The reception from users, I think, was a pleasant surprise for — maybe even for them, and for a lot of us.
Because one of these things with these models is, we are like, maybe from a Google vantage standpoint, we looked at in all the areas where it goes wrong, maybe, a bit more. But users are kind of seeing the potential in these models a lot as well. So I would say that part — maybe more of a surprise. But we had been following GPT 2, GPT 3. We knew the caliber of the folks there, so that part wasn’t a surprise at all.
Do you think that they were reckless to release it when they did?
No, I don’t think so. You know, I’ve heard Sam and Greg, Ilya, et cetera, speak about it. I think you could have many different reasonable points of view around how you approach this technology. And I think there will be a lot of debate around it.
I think one of the things I’ve heard them talk about it is, one of the reasons to put this out sooner is you give society a chance to understand, adapt, et cetera, which I think is a reasonable point to take. I do know folks that who are very thoughtful. And so yeah, I didn’t feel that way.
I’m curious if one reason why Bard didn’t come out last year was that safety was on your mind. How much of it was a safety thing and how much of it was a product thing?
It’s tough to say. Because the reason we built LaMDA — to be a conversational dialogue thing — LaMDA was trained to be a conversational dialogue agent, right? So because we were working on Google Assistant, and we realized the limitations of, like, approaching the assistant with the underlying technology approach we had, so it wasn’t an accident that what we worked on LaMDA to be a conversational dialogue.
So we understood the power of — because people are talking to the Google Assistant back and forth. But I think it’s, again, a set of factors which come together in the culmination of a product. Having built products, I always admire when it happens to me. It’s an exciting moment, regardless of whether we had done it.
Obviously, you always wish you had done it, you know. But I admire the fact that I would not underestimate the product engineering, all the work that goes into making that kind of a fit come together. So that’s how I think about it.
When Microsoft relaunched the new Bing with this OpenAI, what we now was GPT 4 running under the hood, Satya Nadella, CEO of Microsoft, was very sort of jubilant and proud, especially because he thought that it had given Microsoft a new way to compete with Google in search.
And he said at the time that Google was the 800-pound gorilla of search, and that Microsoft, by releasing this new Bing, would make Google want to come out and dance — basically, claiming that Microsoft had kind of been able to shake Google out of a stupor and force you all to innovate. So is he right? Are you dancing now?
Well, part of the reason I think he said it that way is so that you would ask me this question.
He’s very savvy that way, yeah.
So first of all, tremendous respect for Microsoft and teams, Satya and team. I do think it’s a bit ironic that Microsoft can call someone else an 800-pound gorilla, given the scale and size of their company. Maybe I would say we’ve been incorporating AI in search for a long, long time.
When we built transformers here, one of the first use cases of Transformer was birthed, and later, MUM. So we literally took transformer models to help improve language understanding and search deeply. And it’s been one of our biggest quality events for many, many years.
And so I think we’ve been incorporating AI in search for a long time. With LLMs, there is an opportunity to more natively bring them into search in a deeper way, which we will. But search is where people come because they trust it to get information right.
And so to me, the craftsmanship that goes into delivering that high-quality trusted experience is important to us. So we’re going to work hard to get that right, and so that’s the way I think about it.
I do think sometimes I get concerned when people use the word, race, and being first, thought about AI for a long time, and we are definitely working with technology, which is going to be incredibly beneficial, but clearly has the potential to cause harm in a deep way. And so I think it’s very important that we are all responsible in how we approach it.
Yeah. Well, let’s talk about that approach. It’s been reported that in December, you declared a code red inside Google. Can you tell us what is a code red? How does life change around here after you’ve said that?
I’m laughing, because first of all, I did not issue a code red. You know, I’ll tell you what happened. For me, seeing that, look, we are at that point of inflection, it’s one of the most exciting moments. So across our products, we see so much opportunity.
So collectively harnessing the resources in the company to move forward, to rise to the moment is what I’m interested in. So I’m definitely communicating that. I am definitely asking teams to move with urgency.
We are definitely working across. There are many areas. I’m asking, in a deep way, engaging with the teams to understand how we are going to use LLMs or generative AI to translate into deep, meaningful experiences.
And so we are moving. I think we have a responsibility at this moment to deliver, given all the investment we have put into it. And to be very clear, there are people who have probably sent emails saying there is a code red. So I’m not quibbling with — all I’m saying is, did I issue a code red? No, and every time I say that, I’m worried Casey is going to look at me and say, did you or did you not issue a code red!
And so people, to get stuff done, can paraphrase and say, well, there’s a code red, et cetera, but I did not issue code red. It’s genuinely an exciting moment for us. And I think as a company, we’ve long worked towards a moment like this. In 2015, I wanted the company to think in AI-first way, so to me, I’m just excited at the possibilities here.
And it’s also been reported by my colleagues at “The Times” that Larry Page and Sergey Brin, the founders of Google, are being very hands-on about this new generative AI push, that they are back in a sort of literal or metaphorical sense, and that they are getting their hands into these projects. What’s that been like?
So to be very clear, both Larry and Sergey are very active as board members. To me, what was exciting about this moment, part of the reason I called and spoke to them — look, we have been speaking about AI for, pretty much, as long as they can remember. Right?
Part of the reason — I remember being with them — this was maybe in 2012, in a lab not far from here, with Jeff Dean and Geoff Hinton and team, where we saw the early signs of neural network can recognize images, images of a cat, et cetera.
We later brought DeepMind in. So this has been a long journey for us. So it’s an exciting moment. You know, I had a few meetings with them. Sergey has been hanging out with our engineers for a while now.
And he’s a deep mathematician and a computer scientist. So to him, the underlying technology — I think if I were to use his words, he would say it’s the most exciting thing he has seen in his lifetime. So it’s all that excitement, and I’m glad. They’ve always said, call us whenever you need to, and I call them. So that’s what it is.
Yeah. Well, so “The Times” also reported that as part of an effort to get these products to market maybe a little bit faster, you set up what’s called a green lane to maybe accelerate the review and approval of some of these new products. You know, I think sometimes we hear something like that and say, well, are safety checks still being applied?
So what can you tell — and I think it’s also just sort of an interesting question about how you’re changing the company to meet this moment, right? And try to get more products out the door. So how are you kind of balancing that innovation and safety calculus?
I mean, super important. We’ve been very deliberate in how we are moving through this moment. Some of these products, we could have put the market earlier. We are taking our time to do that, and we’ll continue to be very, very responsible.
So I think all we are doing is, we are a big company. So when many parts of the company are moving, you can create bottlenecks, and you can slow down. There’s a difference between being efficient as a company, making sure you’re not bureaucratic as a large company. I think those are the things we are talking about here.
But the work we do around privacy, safety, responsible AI, I think, if anything, is more important. And so our commitment there is going to be unwavering, to get all of this right.
One more question about these language models, maybe before we move on to some other stuff. Last year, one of your engineers came forward to say that he believed LaMDA, this precursor to Bard, was sentient. I never believed that was true, but it did worry me that one of your employees did.
Do you worry about this kind of belief spreading? And is there anything Google can do about it as more people start using these technologies?
I think it’s one of the things we have to figure out over time, as these models become more capable. So my short answer is yes, I think you will see more like this. You’ve just seen the conversations even over the last couple of weeks.
You know, I said this before. AI is the most profound technology humanity will ever work on. I’ve always felt that for a while. I think it will get to the essence of what humanity is. And so this is the tip of the iceberg, if anything, on any of these kinds of issues, I think.
We’ll be right back.
Sundar, let’s talk about some of the big-picture stakes here, with AI and how to get this balance between innovation and safety right. So recently, more than 1,000 technology leaders and researchers, including people like Elon Musk, along with some employees of Google and DeepMind, signed a letter calling for a pause, of at least six months, on the training of large language models more powerful than GPT 4.
And they said that they’re calling for this sort of pause, because they believe that more advanced AI poses, quote, “profound risks to society.” What did you think of that letter, and what do you think of this idea of slowing down the development of big models for six months?
Look, in this area, I think it’s important to hear concerns. I mean, there are many thoughtful people, people who have thought about AI for a long time. I remember talking to Elon eight years ago, and he was deeply concerned about AI safety then. And I think he has been consistently concerned.
And I think there is merit to be concerned about it. So I think while I may not agree with everything that’s there in the details of how you would go about it, I think the spirit of it is worth being out there. I think you’re going to hear more concerns like that.
This is going to need a lot of debate. No one knows all the answers. No one company can get it right. We have been very clear about responsible AI — one of the first companies to put out AI principles. We issue progress reports.
AI is too important an area not to regulate. It’s also too important an area not to regulate well. So I’m glad these conversations are underway. If you look at an area like genetics in the ‘70s, when the power of DNA and recombinant DNA came into being, there were things like the Asilomar Conference.
Paul Berg from Stanford organized it. And a bunch of the leading experts in the field got together and started thinking about voluntary frameworks as well. So I think all those are good ways to think about this.
I’m curious if there is a regulation that you would tell lawmakers might be good to pass in the next six months. Like, for example, I have a friend who thinks a lot about AI issues, and he thinks that beyond a certain size, one of these language models probably shouldn’t be able to run on your laptop. Right?
Or if you found that a model could send phishing emails that had a 1 percent chance of success, you wouldn’t want that to be able to run on any laptop. Pick any example you like. Is there stuff out there where you’re like, well, I hope I don’t see any of the other companies out there doing this?
I would start a little bit more in a basic way. So for example, I would make sure we get privacy regulation right. So because if we have a foundational approach to privacy, that should apply to a technologies, too.
Yeah.
I think there are many areas people underestimate where there are strong regulations already in place. Like, health care is a very regulated industry, right? And so when AI is going to come in, it has to conform with all regulations.
So you also want to build on existing regulation where you can. I think that would allow innovation to proceed as well. Once you start getting into specifics like that, I think what I would be worried about is, this is such fast-evolving technology. Being very opinionated early on, I think, is difficult.
But I think notions of transparency, where people are aware of what other people are doing, has some element of reasonableness to it, how easy it is to do at a global scale, I think those are hard. The thing that gives me hope is I’ve never seen a technology in its earliest days with as much concern as AI.
And just one more thing on this letter calling for this six-month pause. Are you willing to entertain that idea? I know you haven’t committed to it, but is that something you think Google would do?
So I think in the actual specifics of it, it’s not fully clear to me. How would you do something like that, right, today?
Well, you could send an email to your engineers and say, OK, we’re going to take a six-month break.
No, no, no, but How would you do — but if others aren’t doing that. So what does that mean? I’m talking about the how would you effectively —
It’s sort of a collective action problem.
To me at least there is no way to do this effectively without getting governments involved.
Yeah.
So I think there’s a lot more thought that needs to go into it. I think the people behind it intended it, probably, as a conversation starter. And so I think the spirit of it is, I think, good, but I think we need to take our time thinking through these things.
Yeah. There’s sort of two categories of AI risk that people are worried about. There’s sort of the short-term worries — the chat bots that get things wrong, or maybe they’re biased or they’re giving people bad answers. Then, there’s the kind of long-term or longer-term worries about, frankly, AI destroying human civilization.
You know, Sam Altman, CEO of OpenAI, has talked about the possibility of AGI, this artificial general intelligence that could become super human and effect dramatic and bad change in the world. Do you believe that we’re headed toward AGI? And, do you want to build that?
It is so clear to me that these systems are going to be very, very capable. And so it almost doesn’t matter whether you’ve reached AGI or not. You’re going to have systems which are capable of delivering benefits at a scale we have never seen before and potentially causing real harm.
So can we have an AI system which can cost disinformation at scale? Yes. Is it AGI? It really doesn’t matter. Why do we need to worry about AI safety? Because you have to anticipate this and evolve to meet that moment. And so today, we do a lot of things with AI people have taken it for granted.
Yeah.
Right? Think about how big a moment Deep Blue was, or when we did AlphaGo. But you can’t take it all for granted. And so I think this will play out differently than thinking through a moment like AGI.
Right, there is that thing where people just refer to anything you can’t do yet as something AI will handle in the future. I remember the first time I searched Google Photos for dogs, and it just showed me all the dogs on my Camera Roll. I mean, that is AI, at least by some definitions. But I think you’re right — people do take it for granted.
And I remember when we launched Photos, we had to explain at Google IO what neural networks were, what deep learning was, and we’re trying to explain that this is different technology. This is — yeah, it’s interesting.
Yeah, so if you had to give a question on the AGI or the more long-term concerns, what would you say is the chance that a more advanced AI could lead to the destruction of humanity?
There is a spectrum of possibilities. And what you’re saying is in one of that possibility ranges, right? And so if you look at even the current debate about where AI is today or where LLMs are, you see people who are strongly opinionated on either side.
There are a set of who believe these LLMs, they’re just not that powerful. They are statistical models which are —
They’re just fancy autocomplete.
Yes, that’s one way of putting it, right. And there are people who are looking at this and saying, these are really powerful technologies. You can see emergent capabilities — and so on.
We could hit a wall two iterations down. I don’t think so, but that’s a possibility. They could really progress in a two-year time frame. And so we have to really make sure we are vigilant and working with it.
One of the things that gives me hope about AI, like climate change, is it affects everyone. And so these are both issues that have similar characteristics in the sense that you can’t unilaterally get safety in AI. By definition, it affects everyone. So that tells me the collective will come over time to tackle all of this responsibly.
So I’m optimistic about it because I think people will care and people will respond. But the right way to do that is by being concerned about it. So I would never — at least for me, I would never dismiss any of the concerns, and I’m glad people are taking it seriously. We will.
Yeah, it just strikes me that you are in such a tricky position because you have this one group of people that’s saying, like, move faster. Release the stuff faster. Go compete with all these other people. You built all this technology. Don’t let that lead go to waste.
And then you have other people saying what Kevin just said, which is like there’s a non-zero risk that this stuff does something really, really bad. What is that like for you, waking up every day and just having both of those things in your ear?
There is a sense of some whiplash, ? Right it’s like asking, hey, why aren’t you moving fast and breaking things again?
Yeah, yeah.
Which, for all of us, over the past few years. I think we realize we are going to be bold and responsible. We are working with urgency. We are excited at this moment. There’s so much we can do. So you will see us be bold and ship things, but we are going to be very responsible in how we do it.
So there will be times when we will hold back things. I think what we are doing in Bard, for us, is an example of it. We haven’t hooked up Bard to our most capable models yet, and we plan to do it deliberately. And so through this moment, I think we are going to stay balanced, but we are going to innovate. And there is a genuine excitement at this moment, so we’ll do that.
I hear you saying that what gives you hope for the future when it comes to AI is that other people are concerned about it — that they’re looking at the risks and the challenges. So on one hand, you’re saying that people should be concerned about AI. On the other hand, you’re saying the fact that they are concerned about AI makes you less concerned. So which is —
Sorry, I’m saying the fact that the way you get things wrong is by not worrying about it. So if you don’t worry about something, you’re just going to completely get surprised. So to me, it gives me hope that there is a lot of people — important people — who are very concerned, and rightfully so.
Am I concerned? Yes. Am I optimistic and excited about all the potential of this technology? Incredibly. I mean, we’ve been working on this for a long time. But I think the fact that so many people are concerned gives me hope that we will rise over time and tackle what we need to do.
So we should continue to write columns where we’re very nervous about where all this is going?
As well as columnns where you’re excited about the possible benefits of all of this.
Yeah, I hear you on the whiplash. I feel whiplash every day just reading the news about AI. I can only imagine what you’re feeling.
I do too.
Another question that people have about AI in the sort of medium and long term is about its effects on jobs. And there have been all these predictions about LLMs and what kinds of work they could replace or will replace. I actually — I got a text from a software engineer a friend of mine the other day who was asking me if he should go into construction or welding because all of the software jobs are going to be taken by these large language models.
And he was sort of joking, but sort of not. You have a lot of software engineers here at Google that work for you. How should they feel about that question?
With any technology, you have adaptation. I think this one, there’ll be a lot of societal adaptation. And as part of that, we all may need to course-correct in certain areas.
To your specific question, I think for software engineers, there are two things that will also be true. One is some of the grunt work you’re doing as part of programming is going to get better. So maybe it’ll be more fun to program over time — no different from the Google Docs make it easier to write. And so if you’re a programmer, over time, having this collaborative IDs with the assistance built in, I think, is going to make it easier.
The other thing that excites me is programming is going to become more accessible to more people. And so it’s such an important role in the world. You’re creating things. And today, the bar is very high.
So we are going to evolve to a more natural language way of programming over time. So to me, that means things no different from to do a podcast — to do something like this 40 years ago, just imagine what access you would need to have to be able to do an interview like this.
We need a radio tower — [LAUGHS]
Yeah.
[LAUGHS]:
But you know, but we will think, this has enabled more people.
Yeah.
I think the same thing will be true for software engineering as well. So I think those are all important, exciting use cases to think about.
Well I want to ask a more near-term question, near and dear to my heart — kind of about media and publishing, but also search on the web. Today, lots of digital publishers rely on the traffic they get from Google. They get ad impressions. That pays their bills.
When Bard is at its best, it answers my questions without me having to visit another website. I know you’re cognizant of this. But man, if Bard gets as good as you want it to be, how does the web survive?
I think through our work across, I think we’ll be committed to getting it right with the publisher ecosystem. In search today, while these things are contentious, in search, we take pride, it’s one of the largest sources of traffic. If I look at it year-on-year, the traffic we send outside has only grown. That’s what we’ve accomplished as a company.
Part of the reason we are also being careful with things like Bard, amongst many reasons, we do want to engage with the publisher ecosystem, not presume how things should be done. And so you will see us thoughtfully evolve there as well.
Yeah, I mean, I know we can’t really predict what the final form of all of this stuff will be, but I have to believe that, I don’t know, in five years, what used to be the Google Search Bar is just essentially a command line that I can write in to get anything I want — whether I want to change something on my phone, write myself a little app, access the subtotal of human knowledge, have it draft my emails. Does that feel like a potential final destination to you, or do I have it all wrong?
You know, I mean, there’s a part which is consistent with our mission to do that. But I think I want to be careful where Google has always been about helping you the way that makes sense to you. We have never thought of ourselves as the be-all and end-all of how we want people to interact.
So while I think the possibility space is large, for me, it’s important to do it in a way in which users use a lot of things, and we want to help them do things in a way that makes sense to them. And out of that North Star is whatever the answer it leads us to. But I don’t want to get it — so that’s the way I think about it, at least in my head.
Sundar, thank you for joining us.
Thank you, Sundar.
Thanks, Kevin, Casey. Pleasure, yeah.
Casey, we are back from the Googleplex. I really enjoyed our little field trip today. Thank you for that enlightening and enriching road trip, and also for allowing us to stop at In and Out for lunch on the way back.
Yeah, it turns out if you order your fries well done, which is not on the menu, they arrive much crispier and more delicious.
Yeah, that’s a pro tip for you.
You know, that wasn’t my only takeaway from today, Kevin.
Yeah, what’d you think?
Well, you know, a lot of times when companies tell us about the new technologies they’re introducing, they do so in a really grandiose way. And I was struck today by the humility that Sundar uses when he talks about where the company is now. He is not here to tell you that Bard is the best language model out there. He said that, in fact, it is quite limited.
Yeah, he said — he compared it to a souped-up Civic.
[LAUGHS]: Yeah, which I wasn’t expecting. But he broke a little news with us. He told us that Bard is going to be upgraded. And man, I’m really curious to see if Bard feels any different in a few days.
Yeah, and I really was struck by what he called “whiplash,” where he’s got people telling him, you know, you’ve got to move faster, and compete with GPT, and release everything you’ve got — and then, also, this very real sense of like, didn’t we get in trouble for doing this the last time with, you know, all the products that we released in the last decade? Shouldn’t we be slow and deliberate? So I would want to switch jobs with them.
Yeah.
It sounds very hard.
Also, I thought we would mostly be solving mysteries this week, but I feel like we are leaving with one, which is, who did order the Code Red at that company?
- - -
Podium is a venture fund across sports and entertainment, based in Silicon Valley.
Subscribe to:
Posts (Atom)