Veritasium: What Everyone Gets Wrong About AI and Learning – Derek Muller Explains

This talk examines how AI impacts education. The speaker argues against the overused "revolutionize" claim, citing past technological advancements (motion pictures, radio, TV, computers, MOOCs) that failed to fundamentally change education. He emphasizes the importance of cognitive processes (System 1 & 2), highlighting the limitations of working memory and the need for deliberate practice to build long-term memory and expertise. AI's role should be to provide timely feedback and scaffolding, not replace effortful learning. The speaker concludes that in-person learning's success stems from social interaction and accountability, aspects AI currently cannot replicate. Here's a breakdown of System 1 and System 2 cognitive processes: System 1: This system operates automatically and quickly, often without conscious effort. It's more intuitive and handles immediate reactions or familiar tasks. System 2: This is the conscious, reasoning self – the voice inside your head that you identify as "you". It's slow, effortful, and requires attention and concentration. It handles complex computations (like 13 x 17), logical reasoning, and focused tasks. , System 2 has a limited capacity, often cited as being able to handle about seven (plus or minus two) pieces of new information in working memory at one time. How, how many pieces of information can I just work with at one time? So, the way this test goes is I show you some numbers that you can look at briefly, and then I take them away.16:16And what you try to do is, say, each number on the beat, there should be like a beat going. I don't know how we're going to do that, but uh, you say each number, but you add one to it.16:29So increment each digit by one after the numbers have disappeared. I think I'll do it for you. So you guys don't feel weird about this, but I just wanted to demonstrate how we can show that this span is is quite limited.16:41Okay, so hopefully I I get this right because I know I know what the numbers are. Okay. So uh, here we go. There's some numbers 6. 5, 0. 2. right? Okay, that wasn't too hard. You can make the task harder by instead of adding one, you add three and instead of having four numbers, you can go to five or to six to really push yourself and see what happens.17:05What's amazing about this task. What's interesting about it is that while you're doing it while you're really activating system 2 there's physiological responses that take place Your heart beats a bit faster, your skin gets a little sweatier and interestingly your uh pupils dilate.17:23So when you're really thinking hard when you're using that system to thinking your pupils are actually expanding just a little bit So I got a super macro lens and some friends and I showed them uh, numbers and then got them to do the the add one task and so we can try to see if you can spot the growing in the pupil of this this eye while he's doing the task here we go 4.17:513. 97 two Okay, there it goes a little bit 5. 4. 0. 8. 3. I feel like in the middle when he's like in the middle of the numbers and he's still got to remember the remaining numbers and it's like you know like it's just and then at the end it sort of starts collapsing back He's like okay now I can relax a little bit again So it's interesting to see just how effortful Uh this is uh this uh measure is known as cognitive load or another way of thinking of how much mental effort you are investing in something and we can break down how the cognitive load is being expended into three categories.18:31So there's intrinsic cognitive load which is just the amount of cognitive load you need for a particular task. So in that case you know if I'm giving you six numbers it's it's going to require a large intrinsic cognitive load or sometimes when we're doing something in physics even something simple like f equals ma Which to you know physicists and stuff kind of is not that hard but for a new student Each of those things is new and sophisticated.18:59it's a complex concept and so in fact even that just teaching that has a large intrinsic cognitive load there's also things that are totally extraneous. These could be things like, uh, I don't know maybe there's someone chewing next to you or your seat's uncomfortable.19:15there's just distractions and things around you that are are taking away from your ability to focus so um, extraneous cognitive load is not great Uh, and then finally we have Germaine Cognitive load which is can you actually use your attention use system 2 to do some maybe thinking about thinking like observing kind of what you what you're going through as you're thinking through a problem or noticing things that might be helpful patterns for you to use later.19:43So um, germaine load is is great. Um, if you can get it. Now, when it comes to this is another really classic study. Um, because one question you might have is, if we have such a limited system too, how do we accomplish anything? And if you go back several decades, people were wondering, what makes a chess master? a chess master? Do they have really high IQ? Do they have great spatial reasoning? Do they have a really large, uh, working memory span, which is the kind of thing that we've been looking at, You know, what makes them unique? And these researchers got together and they did a study where they showed some different level chess players from beginner intermediate and, and grandmaster.20:30I don't know if they were grandmaster, maybe just a master, but someone who's very experienced in chess. They showed these different people, uh, a chess board like this, that was basically stopped in the middle of a game. Okay, they showed them this chess board for 5 seconds.20:44They had it in front of them, chessboard's there. And then they covered up the chessboard and they had another blank chessboard beside it And they said to these chess players, ",okay, you've seen it for 5 seconds now, put the pieces on the board exactly as they were in that chess board that you just observed.?" when you ask noviceses to do this, they've only seen it for five seconds.21:08When you ask the novice to do this, they will get about four pieces correct after looking at the board. Just once in in further iterations of the study, they were allowed to look at it for another five seconds and then put some more down.21:23And so they were sort of counting up how many it took. But just on the first glance, a beginner could do four pieces. The grandmaster could do 16 pieces after one glance, after just five seconds of looking at it.21:39Why were the grandmasters so good at remembering where all those pieces were on the board? It's a lot. it's because of a phenomenon called chunking, which is where you can see things that look like, disperate bits of information, separate pieces of information, but you can actually see them as one thing.22:00So in the case of the example I gave you earlier, if you reverse the order of these numbers, you get 1945, which is no longer four numbers, it's one thing the year that the Second World War ended, right? And so that makes it much easier to remember.22:18And this is true for basically everything we come into contact with, we're not normally dealing with it in its most, uh, basic chunk. So words, you know, you don't have to parse out every letter, even phrases, you know, they have meaning.22:32And you can think of them as one thing or equations like this one for a physicist, those become one thing, even the Schroinger equation. Yes, for for a lot of people in the room, I imagine that's just a single entity in working memory and you don't have to, you know, you could write that down, you close your eyes and write, write it down again.22:50So the point is that the more we get experience with things, the more we practice, the more we interact, the more we use System 2 to, uh, work through problems, the more we develop this long-term memory and that long-term memory allows us to chunk the things in our world.23:08And that allows us to deal with much more complex situations. So that's exactly what's happening with a chess master. I have heard a lot of people say this, that class really taught me how to think or that teacher taught me how to think.23:29And I think that's more of a question than, than it might seem at first. I mean, at first, it might seem like, yeah, that that feels reasonable. Like, I think a lot of us have this experience where we feel like, oh yeah, after that, I, I really learned how to think.23:44I learned how to reason. But if you go back to this, uh, chessboard study, the next thing they did as part of the study was they rearranged the pieces, same number of pieces as before, but now they positioned them as though they would never occur in a regular chess match.24:00They just put the, put the pieces all over the place at random. you would never ever see this in, uh, a regular chess match, and they tested again, the noviceses and the masters and now the masters did no better than the novices.24:15The thing that made them so good at remembering where the pieces should go was that they had seen a lot of chess boards, they'd seen a lot of configurations just like that. And if the pattern wasn't there, then they were no better off than the noviceses.24:32The exact same situation, My argument here is that there is no general thinking skill, there is no general problem solving skill. What there is, are these complex webs of long--term memory. This is what we build up over our lifetimes.24:54So a physicist is not necessarily going to be a good chess player. And a chess player is not going to be a good physicist. This applies to all domains, I would say, I mean, you can make arguments for specifics where you could say, yes, such and such skill will transfer.25:09But I think in general, the idea is an expert in one field is not an expert in another. And it's just because of this, it's because that long-term memory that your system one is working with is really specialized to, to reflect the experiences you've had and to have recognized all the patterns in those experiences, to allow you to, to chunk, uh, new situations when they come to you, it's so powerful when your system.25:36One is this really complex web that you don't even feel like you're thinking in the same way that those people said 10 cents to me, if you ask Magnus Carlson, what is he doing when he plays chess? Most of the time I know what to do.25:53I don't have to figure it out, right? He's saying, I don't need system two. System one is so welldeveloped that it can solve any problem. at a glance for him, chess is a game of recognition. He recognizes the board in the same way that we recognize faces or physicists recognize a physics problem, right? that's the way we get to be really good at things.26:24So really what we want to be doing in education is using system 2's resources very carefully and repeatedly, such that we store information in long-term memory to allow our system one to do things that effectively feel automatic. So what are the implications of this for education? The first one is that we should eliminate extraneous cognitive load, which is fairly obvious.26:56So you should have a comfortable seat. you should be able to see the board Everything should be legible. The sound from my microphone should be pristine and pure, I shouldn't, I shouldn't have an accent if I have an accent.27:08It makes it harder for you. That's extraneous. Just trying to think, uh, what was that word stuff like that having subtitles sometimes helps My wife loves to watch all TV shows with subtitles. So now that's what I do too.27:22And uh, yeah. Um, so how can we limit intrinsic cognitive load? This is point number two. We don't want to overfill what you can handle. And my guess for you is that the reason why a lot of physics lectures fail sometimes if they do is because the professor doesn't limit intrinsic cognitive load enough.27:47They go through too much novel material in the one lesson and it's just too much for anyone to handle in terms of their working memory. Once you're overloaded, there's basically nothing you can do. Your System 2 doesn't, you know, know how to deal with that. So, I think this is one of those things where we limit intrinsic cognitive load by getting students to play songs. They already know another way to limit intrinsic cognitive load is to slow things down. So anyone who's practicing music, for example, like this harpist, who's a friend of mine.29:19uh, she describes her practice like this. You can practice everything exactly as it is and exactly as it's written. Um, but at just such a speed that you have to think about and, and know exactly where you are and what your fingers are doing and what it feels like.29:35So she's really talking about using system two there very slowly deliberately effort thinking through every little thing that she's doing in those moments. And then once she does that enough, times enough repetitions of this, you get something that looks like [music], this.29:56Yeah, so good, right Every time I see, you know, superhuman performance. The thing that I'm thinking about is how they have used System 2 very slowly and effort to build up that structure in long-term memory that allows them to do things that look so amazing and effort effortless. Here's a breakdown of constructivism and its potential pitfalls related to discovery learning: Constructivism: This is an educational paradigm based on the idea that students are active constructors of their own knowledge. The core principle aligns with the idea that engaging System 2 (active, effortful thinking) is crucial for learning. Potential Harm/Misinterpretation: The speaker doesn't disagree with the fundamental idea of constructivism. The harm arises from a flawed implementation where some educators believed that if students need to be active, then direct instruction ("telling") is ineffective. This misinterpretation led to removing necessary support (scaffolding) too early. This resulted in a form of "discovery learning" where students were given problems to solve without being taught the methods or foundational knowledge needed, essentially being told to "figure it out" on their own. , The speaker views this specific approach to discovery learning, born from a misunderstanding of constructivism, as potentially dangerous or ineffective. For example if you go to a new city and you don't use a GPS, it's really hard to figure out where you have to go.32:13but if you do it and you use your system two to figure out and follow every street and look at all the signs around and the you read these things and you yeah you look at really carefully at everything.32:24At the end of that you'll probably be able to retrace your steps If you use the GPS, it'll be much easier for you. you're offloading processing from system 2, you're offloading it to the GPS. Can you find your way back? No.32:37you're going to need the GPS to find your way back. I think there has to be some balance in education between the kind of GPS guidance and the internal guidance. Ideally, I would love to see like a gradual phasing out of the support to allow, you know, the person to figure out the directions on their own.32:57Uh, this has been, I think, well, backed up in research of things like the worked example effect. And, and this is even a further idea of sort of fading out the assistance as you go. But the idea of like first giving someone, hey, here's a problem.33:10And here's how you solve it. Here's a a problem that is not quite at the end. Here's a problem partially done. And here's one for you to do from start to finish, right? But it is this idea of like, I'm going to reduce your intrinsic cognitive load.33:22I'm not just going to give you this problem at the end and say figure it out. I'm going to give you this scaffolding because I know that what your working memory is is very limited, that system two has very limited resources and I'm not going to tax them with you trying to think about all sorts of things at the same time.33:39I'm going to give you this assistance. So to me, this is the scaffolding that helps system one move through and learn new problems. I think again, this is kind of a problem we have, um, in complex domains like physics, where to the physics professor? everything's perfectly clear because their system one is so fully developed but to a student it's not so the the, this is the expert novice divide.34:06The professor can't see with the student eyes what that problem looks like, right? Only the student can see it that way. Okay, so what else do we need to do in education? We need to repeat that effortful practice until we achieve mastery.34:23Why is mastery so important? Because when you show a skill with mastery, that means it's now a system one domain. There are some some different industries and some different professions where people seem like experts, but they're actually not stock pickers is one example because you may get some feedback that like, maybe this was a good decision or a bad decision.39:37But honestly, the stock market is at least in the short term, incredibly random. And so that feedback is never very informative for what you should do the next time. This has also been shown with uh, political pundits and some economists, anyone who's looking to predict these future trends, which are uh, much less reliable in terms of the validity of the environment.40:01So one thing I see AI doing is providing this timely feedback, which will really help as we saw in the, in the first case, um, with Salon and his son. But the thing that I'm really worried about is how AI has this opportunity to reduce effortful practice.40:18I have four kids who are 8, six, four and zero, and uh, and I worry about them that, you know, if they're going to be, will they write an essay, will they write a hundred essays? If there is a generative AI that can write for them, what forces them to practice crafting those sentences.40:41and if they don't craft those sentences, what happens to their brains? The argument here is that you get good at your command of the English language, you get good at being able to speak in front of people, at being able to express your thoughts in writing, by doing it again and again and again and again.41:00And you should suck at the beginning and you shouldn't let that stop you. And you should keep going and going and making slight tweaks and improving and getting feedback and and getting going. If they never do that, I really worry what gets into system one, you know, what is that do? They have an, an amazing network of connected, uh, knowledge that they can draw on.41:23Do they have things that are automated? I fear that they won't, how do we force people to have to do that painful, effortful work when there's a magic machine that'll do it for you. that's a big concern. What about drawing? You know, if you can just ask it to make a picture of whatever you like the bat in the ball was AI by the way, Um, I can't draw So but again, like what will happen to people's artistic abilities? So this is I think my biggest concern is if it prevents us from going through this painful effortful process which is the core process of learning using your limited system 2 resources to engage with things and practice again and again and again even when it's hard, even when it doesn't feel good, even when you're not great at it.42:11That is my big concern. I want to come back to these two big questions. Why aren't people learning and why haven't the education revolutions material materialized to the first question. At the beginning of this talk our brains are designed to help us be effective in this world, which means finding food and shelter, finding a mate, integrating socially so that we're not ostracized you know being able just to hang out and have fun like all of those things are what we should be doing and maybe it shouldn't be such a surprise that people don't know what elements it takes to make water or even that CO2 could pose an existential threat.42:59I know everyone in this room will will agree like that is that's that's important we should all know that and and we should figure out how to work with that. I guess what I'm saying is I think it's understandable that a lot of people don't focus on that don't know that don't think about it it's not part of the world that they exist in just because you know they're so busy with social media and Instagram and whatever because that's about connecting with other people.43:23So I want to be a little bit generous there And when it comes to this question of why haven't education revolutions materialized with film and TV and radio and computers and mukes. And now AI part of me wants to say I think we might have already found the best thing being in a room with other people, other learners, a teacher and some time to talk.43:58Education is it's a social activity. You know, people care about other people. I think that the tech hype comes from a place of believing that the problem of education is not being able to get the information to the student.44:19That's not the problem. It's not the problem now. And it wasn't the problem. Aundred years ago, when you have books, I mean the information is all there, assuming people have access, the students had access to those books and yet they're probably not going to learn very much unless they have a great teacher unless they have a group of like--minded peers to go through that with them unless they have a reason to do it An analogy for you you know the world is full of heavy objects and yet most people are not ripped.44:56Do you do you see where I'm going with this? The world is full of fields, not many people running on them. there's plenty of ways to get exercise, but obesity is a huge problem. I hope the analogy I'll get there I think about teachers because I was one I kind of am one um, I think about teachers a little bit like personal trainers like the gyms are there but unless there's someone who you're going and you're meeting there and you're held accountable to and someone to say "another one and another give me another rep, keep going it's burning." yeah keep going you know someone to to tell you the homework and someone to hold you accountable and someone to really energize you and maybe a group of other people who are doing it at the same time and we're all like going to this together that's when you see results this doesn't just happen in a vacuum you know so i guess that's my big thought that you know teachers are are some of the greatest people in the world doing doing an incredible job of connecting with students and creating communities of learners and that's what it's about it's about that social experience it's about getting excited and holding people accountable and and forcing them to put the reps in so for me that's why none of these technologies are ever going to revolutionize education thank you Yeah. Uh Hi, Derek, Um earlier you touched on the way that people learn like an effective way that people are in the classroom where it's like you have a framework and you slowly take away um, like line by line of the equation.56:53Uh, kind of within this context. How do you think or where do you think, uh, the place of AI would be like within this, uh, this way of learning if at all, Yeah, it's a great, a great question. I think, you know, AI can offer lots of scaffolding.57:10You can like say, ",hey, can I get a hint?" you know, so there's, there's a remarkable capability for it to sort of fill all kinds of gaps in learning or like you could say to the AI ",hey, I need to learn about, you know, this period of Canadian history.57:24Ask me 50 questions about this." like what a great tool to be able to have if you can use it effectively. Right. Um, so all of those ways that you can use it as an educational aid, I think will be great.57:37I think the place where it won't be great is where it allows you to do the work without doing the work and it's that work which is essential for learning so like that's my big concern yeah thank you Um, so if you find that in--person education succeeds because people are forced to engage with the content They're forced to put a conscious effort in. Why do you find that online edutainment succeeds when people can check out so easily? Yeah, so I kind of think maybe they're doing different things like if I was your teacher or lecturer then there would be stuff to slog through as well as stuff to enjoy.01:04:27I think being an online educator, I get to just pick some highlights and make some exciting stuff that hopefully energizes people shows them what's possible. I feel like my job is sort of a combination of educator but also uh, magician, uh also you know comedian or something not that i'm particularly funny, but you know, uh, i yeah, i think it's all those things Um, so I i think there's a place for that and then there's a separate place for like now you have to sit down and and practice and that can be you know more tedious.01:05:00Um, so yeah, hopefully one leads to the other at least in some cases yeah. so do you find that people tend to like self--select for edutainment 100%? Yeah, I mean, one of the reasons we could do What we do is because we aggregate from across the world if I was ever on a just a broadcast channel somewhere like you know the CBC here or ABC in Australia like I could never go into the depth as as soon as I would put down an a derivative on screen or something they'd be like, no, no, no no, no, you cannot no don't do that.01:05:30Let's do an analogy to like skiing or something you know, you just lose all the rigor. So it's the thing I'm so grateful for is the internet came along at a time when I wanted to do that And there was people across the world who were hungry for it.01:05:43Um, so I just feel so incredibly lucky to be able to to do this and go into depth and yeah, reach a lot of people with it. Thank you. Thank you Hi. Um, my question is if you like start talking to AI or like if AI was in a human body like talk likes a human acts like a human and you like become friends with it? would it count as social interacting? I think that is an amazing question, amazing question.01:06:15[music] Let me say this that one of the best ways to learn that's been shown has been if you have like a one-on-one tutor, there's what's called the two sigma effect that the performance of those students is two standard deviations higher than everyone else.01:06:34So it's a huge effect and if we could replicate that effect by having something that is so natural and so human seeming uh that it would just feel like you're there with another human that might be really really really powerful I wonder I feel like this is such a big question Like how much do we need to know that it's a real human there? or how much could a fake human you know actually make us feel in the exact same way? And I think that's what it comes down to that's a such an amazing question.01:07:08Thank you. thanks we go there we go um if we didn't completely remove ai from education what tools do you think we would add and what tools do you think we would remove if we didn't completely remove if we did or didn't didn't if we don't completely remove ai from education What kind of tools are we going to add and what are we going to take away? I don't know if I fully grasp it Maybe what maybe what parts of AI would you keep in the classroom? Yeah, Yeah, I mean for me the the big role of AI is that kind of like um, drill and practice type stuff Really getting people to do the reps giving immediate feedback with whatever you know someone's working on I think there can be some powerful use cases.01:09:48Yeah, the real risky use case is the one where people use it to do their work for them, Which is I think the one where it's being used most unfortunately so that's that's the one that's concerning Yeah, that's the one we got to get rid of.