This lecture discusses generative AI, explaining its core technology (language modeling using neural networks, particularly Transformers) and its evolution from simple tools like Google Translate to sophisticated models like ChatGPT. The speaker highlights the crucial role of scaling (increasing model size and training data) in improving performance, but also emphasizes the challenges of aligning AI behavior with human values (helpfulness, honesty, harmlessness) and the risks associated with misuse, including the creation of deepfakes and the potential displacement of jobs. The overall message is that generative AI is a powerful tool with significant potential benefits but also considerable risks requiring careful management and regulation. The speaker defines generative AI, differentiating it from general AI by its ability to create new content (audio, code, images, text, video). The lecture's structure—past, present, and future of AI—is introduced, emphasizing that generative AI is not a new concept but has recently gained significant attention. Examples like Google Translate and Siri are given to illustrate this point. the fuss? What happened? So in 2023 open AI, which is a company in California, in fact, in San Francisco, if you go to San Francisco, you can even see the lights at night of their building. Um, it announced gp4 and it claimed that it can be 90% of humans on the SAT. For those of you who don't know, SAT is a standardized tex test that American, um school children have to take to enter university. it's an admissions test and it's multiple choice and it's considered not so easy. So gp4 can do it. They also claimed that it can get top marks in law medical exams, um, and other exams, they have a whole suite of things that they claim, uh, well, not the claim they show that gp4 can do it. Okay. Aside from that it can pass exams, we can ask it to do other things. So you can ask it to, um, write text for you. For example, you can have a prompt, this little thing that you see up there. it's a prompt. It's what the human wants the tool to do for them. And a potential prompt could be I'm writing an essay about the use of mobile phones during driving. Can you give me three arguments in favor. This is quite sophisticated, if you ask me, I'm not sure I can come up with three arguments you can also do. And these are real prompts that actually the tool can do. Um, you tell CH, GPT or GPT in general act as a JavaScript developer, write a program that checks the information on a form, name and email are required, but address and age are not. So i'm just writing this. And the tool will spit out a program. And this is the best one. One, create an about me page for a website. I like rock climbing, outdoor sports. And I like to program, I started my career as a quality engineer in the industry. Blah, blah, blah. So I give this version of what I want the website to be and it will create it for me. So you see, we've gone from Google Translate and Siri and the auto completion to something, which is a lot more sophisticated, I can do a lot more The speaker transitions to explaining the core technology behind ChatGPT, focusing on language modeling. This section introduces the concept of predicting the next word in a sequence based on context, highlighting the shift from simple counting methods to sophisticated neural networks. The speaker sets the stage for a deeper dive into the technical aspects in the following segments.The speaker delves into the mechanics of language modeling, explaining how it works by predicting the most likely word continuation given a context. The concept of "prompt" is clarified, and the speaker emphasizes the predictive nature of these models, acknowledging their potential for errors when predicting less likely but still valid continuations. The foundation for building a language model is laid out, focusing on data collection and the use of neural networks for prediction. done. But what people discovered is if you actually take GPT and you put it out there, it actually doesn't behave like people want it to behave because this is a language model trained to predict and complete sentences And humans want to use GPT for other things because they want they have their own tasks that the developers hadn't thought of. So then the notion of fine-tuning comes in, it never left us. So now what we're going to do is we're going to collect a lot of instructions. So instructions are examples of what people want Chad GPT to do for them, such as answer the following question, or answer the question step by step. And so we're going to give these demonstrations to the most model. And in fact, one, almost 2,000 of 2,000 of such examples. And we're going to fine tune. So we're going to tell this language model, look, these are the tasks that people want, try to learn them. And then an interesting thing happens is that we can actually then generalize to unseen tasks, unseen instructions, because you and I may have different usage purposes for these language models. Okay, but here's the problem. We have an alignment problem. And this is actually very important and something that, uh, will not leave us uh, for the future. And the question is, how do we create an agent that behaves in accordance with what a human wants? And I know this, there's many words in questions here, but the real question is, if we have AI systems with skills that, that we find important or useful, how do we adapt those systems to reliably use those skills to do the things we want? And there is a framework that, um, is called the HHH framing of the problem. So we want GPT to be helpful, honest, and harmless. And this is the bare minimum. So what does it mean helpful? It can follow. It should follow instructions and perform the tasks we wanted to perform and provide answers for them and ask relevant questions according to the user intent and clarify. So if you've been following in the beginning, GPDD did nothing, none of this, but slowly it became better. And it now actually asks for these clarification questions, it should be accurate something that is not 100% there. Even to this, there is, you know, inaccurate information and avoid toxic biased or offensive responses. And now here's a question I have for you. How will we get the model to do all of these things? You know, the answer fine tuning, except that we're going to do a different fine-tuning. We're going to ask the humans to do some preferences for us. So in terms of helpful, we're going to ask an example is what causes the seasons to change. And then we'll give two options to them. to. the human changes occur all the time and it's an important aspect of life. Bad The seasons are caused primarily by the tilt of the air axis. Good. So we'll get this preference course and then we'll train the model again and then it will know. So fine-tuning is very important and now it was expensive as it was. Now we make it even more expensive because we add a human into the mix right? Because we have to pay these humans that give us the preferences we have to think of the tasks The same. For honesty, is it possible to prove that P equals NP U. No it's impossible. It's not great as an answer. That is considered a very difficult and unsolved problem in computer science. it's better And we have similar for harmless okay so I think it's time let's see if we'll do a demo yeah that's bad if you remove all the files um okay hold on okay so now we have GPT here I'll do some questions and then we'll take some questions from the audience okay so let's ask one question. is the UK a monarchy can you see it up there I'm not sure and it's not generating oh perfect okay so what do you observe first thing too long I always have this beef with this it's too long you see what it says as of my last knowledge update in September 2021 the United Kingdom is a constitutional monarchy. it could be that it wasn't anymore right something happened this means that while there is a monarch the reigning monarch as to that time was Queen Elizabeth Iii so it tells you you know, I don't know what happened at that time. would you build your own language model? This is a recipe. This is how everybody does this. So step one, we need a lot of data. We need to collect a ginormous corpus. So these are words. And where will we find such a ginormous corpus? I mean, we go to the web, right, and we download the whole of Wikipedia stack Overflow pages, qu, a social media Github, Reddit, whatever you can find out there, I mean, work out the permissions it has to be legal. You download all this corpus and then what do you do then you have this language model? I haven't told you what exactly this language model is. There is an example and I haven't told you what the neural network that does the prediction is. But assume you have it. So you have this machinery that will do the learning for you. And the task now is to predict the next word. But how do we do it? And this is the genius part. We have the sentences in the corpus. we can remove some of them. And we can have the language model predict the sentences we have removed. This is dead cheap. I just remove things. I pretend they're not there. And I get the language model to predict them. So I will randomly truncate. truncate means remove the last part of the input sentence, I will calculate with this neural network, the probability of the missing word, If I get it right, I'm good. If I'm not right, I have to go back and reestimate some things because obviously I made a mistake and I keep going, I will adjust and feed back to the model. And then I will compare what the model predicted to the ground truth, because I've removed the words in the first place. So I actually know what the real truth is. And we keep going for some months or maybe years, no months, let's say. So it will take some time to do this process. because as you can appreciate, I have a very large corpus, and I have many sentences and I have, do the, do the prediction, and then go back and correct my mistake, and so on. But in the end, the thing will converge. and I will get my answer. So the tool in the middle that I've shown, uh, this tool here, this language model, a very simple language model looks a bit like this. So, and maybe the audience has seen this, this is a very naive graph. Um, but it helps to illustrate the point of what it does. So this neural network language model will have some input, which is these, um, nodes in the, as we look at it, well, my right, and your right, okay. So the notes here on the right are the input, and the nodes at the very left are the output. So we will present, present this neural network with, uh, five inputs, the five circles, and we have three outputs, the three circles, and there is stuff in the middle that I didn't say anything about These are layers, these are more nodes that are supposed to be abstractions of my input. So they generalized, the idea is, if I put more layers on top of layers, the middle layer layers will generalize the input and will be able to see patterns that are not there. So you have these nodes, and the input to the nodes are not exactly words, they're vectors. So, series of numbers. But forget that for now. So we have some input, we have some layers in the middle, we have some output. and this now has these connections, these edges, which are the weights, this is what the network will learn. And these weights are basically numbers. And here, it's all fully connected. So, I have very many connections. why am I going through this process of actually telling you all of that? you will see in a minute. So you can work out how big or how small this neural network is, depending on the numbers of connections it has. So for this toy neural network we have here, I have worked out the number of weights we call them. also parameters that this neural network has, and that the model needs to learn. So the parameters are the number of units as input. In this case, it's five times the units in the next layer, Eight plus eight, this plus eight is a bias. it's, um, a cheating thing that this neural networks have, uh, again, you need to learn it and it sort of corrects a little bit the neural network, if it's off, it's actually genius if the prediction is not right, it tries to correct it a little bit. So for the purposes of this talk, I'm not going to go into the details. All I want you to see This segment discusses the inherent risks and challenges associated with large language models, including the difficulty of regulating their content, the persistence of historical biases, and the potential for significant financial consequences from inaccuracies. The speaker uses the example of Google's Bard model and its inaccurate response about the James Webb Space Telescope to illustrate the potential for substantial financial losses due to these errors.