Lex Friedman Podcast - Guido Van Rossum

00:00:00,000 --> 00:00:07,600 Can you imagine possible features that Python 4.0 might have that would
00:00:08,960 --> 00:00:16,320 necessitate the creation of the new 4.0, given the amount of pain and joy,
00:00:17,200 --> 00:00:22,800 suffering and triumph that was involved in the move between version 2 and version 3?
00:00:25,200 --> 00:00:29,920 The following is a conversation with Guido van Rossum, his second time on this podcast.
00:00:29,920 --> 00:00:36,480 He is the creator of the Python Programming Language and is Python's Emeritus BDFL,
00:00:36,480 --> 00:00:41,840 Benevolent Dictator for Life. This is the Lex Friedman Podcast. To support it,
00:00:41,840 --> 00:00:47,520 please check out our sponsors in the description. And now, dear friends, here's Guido van Rossum.
00:00:48,560 --> 00:00:57,680 Python 3.11 is coming out very soon. In it, CPython claimed to be 10 to 60% faster.
00:00:57,680 --> 00:01:03,440 How did you pull that off? And what's CPython? CPython is the last Python implementation
00:01:03,440 --> 00:01:08,240 standing, also the first one that was ever created. The original Python implementation
00:01:08,240 --> 00:01:12,000 that I started over 30 years ago. So what does it mean that Python,
00:01:12,000 --> 00:01:16,880 the programming language, is implemented in another programming language called C?
00:01:16,880 --> 00:01:21,280 What kind of audience do you have in mind here? People who know programming?
00:01:21,280 --> 00:01:25,760 No, there's somebody on a boat that's into fishing and have never heard about programming,
00:01:25,760 --> 00:01:29,360 but also some world-class programmers. You're going to have to speak to both.
00:01:29,360 --> 00:01:33,600 Imagine a boat with two people. One of them has not heard about programming and is really
00:01:33,600 --> 00:01:39,840 into fishing. And the other one is like an incredible Silicon Valley programmer that's
00:01:39,840 --> 00:01:46,240 programmed in everything C, C++, Python, Rust, Java. He knows the entire history of
00:01:46,240 --> 00:01:48,400 programming languages. So you're going to have to speak to both.
00:01:49,040 --> 00:01:54,480 I imagine that boat in the middle of the ocean. I'm going to please the guy who knows how to fish
00:01:54,480 --> 00:01:58,640 first. Yes, please. He seems like the most useful
00:01:58,640 --> 00:02:01,520 in the middle of the ocean. You got to make him happy.
00:02:01,520 --> 00:02:07,520 I'm sure he has a cell phone. So he's probably very suspicious about what goes on in that cell
00:02:07,520 --> 00:02:13,360 phone, but he must have heard that inside his cell phone is a tiny computer. And a programming
00:02:13,360 --> 00:02:17,280 language is computer code that tells the computer what to do.
00:02:17,280 --> 00:02:24,080 It's a very low level language. It's zeros and ones, and then there's assembly, and then...
00:02:24,080 --> 00:02:30,400 Oh, yeah. We don't talk about these really low levels because those just confuse people. I mean,
00:02:30,960 --> 00:02:35,920 when we're talking about human language, we're not usually talking about vocal tracts and
00:02:35,920 --> 00:02:40,880 how you position your tongue. I was talking yesterday about how when you have a Chinese
00:02:40,880 --> 00:02:47,760 person and they speak English, this is a bit of a stereotype they often don't know.
00:02:47,760 --> 00:02:54,080 They can't seem to make the difference well between an L and an R. And I have a theory
00:02:54,080 --> 00:03:00,960 about that, and I've never checked this with linguists, that it probably has to do with
00:03:00,960 --> 00:03:05,840 the fact that in Chinese there is not really a difference. And it could be that there are
00:03:05,840 --> 00:03:14,240 regional variations in how native Chinese speakers pronounce that one sound that sounds
00:03:14,240 --> 00:03:19,200 to L, like L to some of them, like R to others.
00:03:19,200 --> 00:03:24,560 So it's both the sounds you produce with your mouth throughout the history of your life
00:03:25,120 --> 00:03:29,120 and what you're used to listening to. I mean, every language has that. Russian has...
00:03:29,120 --> 00:03:29,520 Exactly.
00:03:29,520 --> 00:03:36,960 The Slavic languages have sounds like zh, the letter zh. Americans or English speakers
00:03:36,960 --> 00:03:42,960 don't seem to know the sound zh. They seem uncomparable with that sound.
00:03:42,960 --> 00:03:43,920 Yeah.
00:03:43,920 --> 00:03:50,160 Oh, yes. Okay. So we're not going to the shapes of tongues and the sounds that the
00:03:50,160 --> 00:03:51,680 mouth can make. Fine. Words.
00:03:51,680 --> 00:03:56,960 Similarly, we're not going into the ones and zeros or machine language. I would say
00:03:56,960 --> 00:04:04,720 a programming language is a list of instructions, like a cookbook recipe that sort of tells
00:04:04,720 --> 00:04:10,640 you how to do a certain thing, like make a sandwich. Well, acquire a loaf of bread,
00:04:10,640 --> 00:04:19,600 cut it in slices, take two slices, put mustard on one, put jelly on the other or something,
00:04:20,160 --> 00:04:25,760 then add the meat, then add the cheese. I've heard that science teachers can actually
00:04:26,640 --> 00:04:32,400 do great stuff with recipes like that and trying to interpret their students' instructions
00:04:32,400 --> 00:04:36,400 incorrectly until the students are completely unambiguous about it.
00:04:36,400 --> 00:04:41,600 With language, see, that's the difference between natural languages and programming
00:04:41,600 --> 00:04:49,200 languages. I think ambiguity is a feature, not a bug in human spoken languages. Like
00:04:50,640 --> 00:04:53,120 that's the dance of communication between humans.
00:04:54,240 --> 00:05:01,520 Well, for lawyers, ambiguity certainly is a feature. For plenty of other cases,
00:05:01,520 --> 00:05:07,040 the ambiguity is not much of a feature, but we work around it, of course.
00:05:07,680 --> 00:05:09,760 What's more important is context.
00:05:10,560 --> 00:05:16,880 So with context, the precision of the statement becomes more and more concrete, right? But
00:05:16,880 --> 00:05:22,800 when you say, I love you to a person that matters a lot to you, the person doesn't
00:05:22,800 --> 00:05:28,320 try to compile that statement and return an error saying, please define love, right?
00:05:28,320 --> 00:05:34,160 No, but I imagine that my wife and my son interpret it very differently.
00:05:35,440 --> 00:05:35,920 Yes.
00:05:35,920 --> 00:05:37,360 Even though it's the same three words.
00:05:37,360 --> 00:05:38,560 But imprecisely still.
00:05:39,920 --> 00:05:40,480 Oh, for sure.
00:05:41,920 --> 00:05:44,480 Lawyers have a lot of follow-up questions for you.
00:05:44,480 --> 00:05:48,160 Nevertheless, the context is already different in that case.
00:05:48,160 --> 00:05:56,880 Yes, fair enough. So that's a programming language, is ability to unambiguously state a recipe.
00:05:56,880 --> 00:06:03,120 Actually, let's go back. Let's go to PEP-8. You go through in PEP-8, the style guide for
00:06:03,120 --> 00:06:10,960 Python code, some ideas of what this language should look like, feel like, read like. And
00:06:10,960 --> 00:06:15,680 the big idea there is that code readability counts. What does that mean to you? And how
00:06:15,680 --> 00:06:18,320 do we achieve it? So this recipe should be readable.
00:06:18,320 --> 00:06:25,520 That's a thing between programmers. Because on the one hand, we always explain the concept
00:06:25,520 --> 00:06:32,320 of programming language as computers need instructions, and computers are very dumb
00:06:32,320 --> 00:06:37,520 and they need very precise instructions because they don't have much context. In fact, they
00:06:37,520 --> 00:06:44,800 have lots of context, but their context is very different. But what we've seen emerge
00:06:44,800 --> 00:06:53,120 during the development of software, starting probably in the late 40s, is that software
00:06:53,120 --> 00:07:00,080 is a very social activity. A software developer is not a mad scientist who sits alone in his
00:07:00,080 --> 00:07:09,360 lab writing brilliant code. Software is developed by teams of people. Even the mad scientist
00:07:09,360 --> 00:07:14,560 sitting alone in his lab can't type fast enough to produce enough code, so that by
00:07:14,560 --> 00:07:20,480 the time he's done with his coding, he still remembers what the first few lines he wrote
00:07:20,480 --> 00:07:29,440 mean. So even the mad scientist coding alone in his lab would be sort of wise to adopt
00:07:29,440 --> 00:07:37,760 conventions on how to format the instructions that he gives to the computer. So that the
00:07:37,760 --> 00:07:43,840 thing is, there is a difference between a cookbook recipe and a computer program. The
00:07:43,840 --> 00:07:52,160 cookbook recipe, the author of the cookbook writes it once, and then it's printed in 100,000
00:07:52,160 --> 00:08:00,480 copies, and then lots of people in their kitchens try to recreate that recipe, that particular
00:08:00,480 --> 00:08:10,480 pie or dish from the recipe. And so there, the goal of the cookbook author is to make
00:08:10,480 --> 00:08:20,400 it clear to the human reader of the recipe, the human amateur chef in most cases. When
00:08:20,400 --> 00:08:29,680 you're writing a computer program, you have two audiences at once. It needs to tell the
00:08:29,680 --> 00:08:40,400 computer what to do, but it also is useful if that program is readable by other programmers,
00:08:40,400 --> 00:08:49,600 because computer software, unlike the typical recipe for a cherry pie, is so complex that
00:08:49,600 --> 00:08:56,320 you don't get all of it right at once. You end up with the activity of debugging, and
00:08:56,320 --> 00:09:04,320 you end up with the activity of... So debugging is trying to figure out why your code doesn't
00:09:04,320 --> 00:09:06,760 run the way you thought it should run.
00:09:06,760 --> 00:09:11,600 That means, broadly, it could be stupid little errors, or it could be big, logical errors.
00:09:11,600 --> 00:09:13,120 It could be anything.
00:09:13,120 --> 00:09:14,120 Spiritual.
00:09:14,120 --> 00:09:21,920 Yeah, it could be anything from a typo to a wrong choice of algorithm to building something
00:09:21,920 --> 00:09:26,080 that does what you tell it to do, but that's not useful.
00:09:26,080 --> 00:09:33,160 Yeah, it seems to work really well 99% of the time, but does weird things 1% of the
00:09:33,160 --> 00:09:35,440 time on some edge cases.
00:09:35,440 --> 00:09:37,760 That's pretty much all software nowadays.
00:09:37,760 --> 00:09:39,160 All good software, right?
00:09:39,160 --> 00:09:42,320 Well, yeah, for bad software, the...
00:09:42,320 --> 00:09:47,160 That 99 goes down a lot. But it's not just about the complexity of the program. It's,
00:09:47,160 --> 00:09:53,720 like you said, it is a social endeavor in that you're constantly improving that recipe
00:09:53,720 --> 00:09:55,600 for the cherry pie.
00:09:55,600 --> 00:10:03,500 But you're in a group of people improving that recipe, or the mad scientist is improving
00:10:03,500 --> 00:10:11,080 the recipe that he created a year ago and making it better, or adding something. He
00:10:11,080 --> 00:10:18,040 decides that he wants, I don't know, he wants some decoration on his pie, or icing, or...
00:10:18,040 --> 00:10:23,080 So there's broad philosophical things, and there's specific advice on style. So first
00:10:23,080 --> 00:10:30,280 of all, the thing that people first experience when they look at Python, it is very readable,
00:10:30,280 --> 00:10:36,240 but there's also like a spatial structure to it. Can you explain the indentation style
00:10:36,240 --> 00:10:40,040 of Python and what is the magic to it?
00:10:40,040 --> 00:10:47,680 Spaces are important for readability of any kind of text. If you take a cookbook recipe
00:10:47,680 --> 00:10:58,160 and you remove all the bullets and other markup, and you just crunch all the text together,
00:10:58,160 --> 00:11:03,600 maybe you leave the spaces between the words, but that's all you leave. When you're in the
00:11:03,600 --> 00:11:09,840 kitchen trying to figure out, oh, what are the ingredients and what are the steps? And
00:11:09,840 --> 00:11:14,200 where does this step end and the next step begin? You're going to have a hard time if
00:11:14,200 --> 00:11:17,760 it's just one solid block of text.
00:11:17,760 --> 00:11:24,160 On the other hand, what a typical cookbook does, if the paper is not too expensive, each
00:11:24,160 --> 00:11:30,000 recipe starts on its own page. Maybe there's a picture next to it. The list of ingredients
00:11:30,000 --> 00:11:38,760 comes first. There's a standard notation. There's shortcuts so that you don't have to
00:11:38,760 --> 00:11:44,320 sort of write two sentences on how you have to cut the onion, because there are only three
00:11:44,320 --> 00:11:49,960 ways that people ever cut onions in a kitchen, small, medium, and in slices, or something
00:11:49,960 --> 00:11:50,960 like that.
00:11:50,960 --> 00:11:51,960 Right.
00:11:51,960 --> 00:11:56,680 None of my examples make any sense to real cooks, of course, but...
00:11:56,680 --> 00:12:02,080 Yeah. We're talking to programmers with a metaphor of cooking. I love it. But there
00:12:02,080 --> 00:12:09,360 is a strictness to the spacing that Python defines. So there's some looser things, some
00:12:09,360 --> 00:12:17,080 stricter things, but the four spaces for the indentation is really interesting. It really
00:12:17,080 --> 00:12:20,200 defines what the language looks and feels like.
00:12:20,200 --> 00:12:25,760 Because indentation sort of taking a block of text and then having inside that block
00:12:25,760 --> 00:12:34,520 of text a smaller block of text that is indented further as sort of a group, it's like you
00:12:34,520 --> 00:12:42,120 have a bulleted list in a complex business document, and inside some of the bullets are
00:12:42,120 --> 00:12:49,080 other bulleted lists. You will indent those too. If each bulleted list is indented several
00:12:49,080 --> 00:12:55,400 inches, then at two levels deep, there's no space left on the page to put any of the words
00:12:55,400 --> 00:13:00,160 of the text. So you can't indent too far. On the other hand, if you don't indent at
00:13:00,160 --> 00:13:05,760 all, you can't tell whether something is a top-level bullet or a second-level bullet
00:13:05,760 --> 00:13:15,440 or a third-level bullet. So you have to have some compromise. And based on ancient conventions
00:13:15,440 --> 00:13:26,760 and the typical width of a computer screen in the 80s and all sorts of things, we came
00:13:26,760 --> 00:13:34,760 up with four spaces as a compromise. I mean, there are large groups of people who code
00:13:34,760 --> 00:13:43,160 with two spaces per indent level. For example, the Google Style Guide, all the Google Python
00:13:43,160 --> 00:13:47,520 code, and I think also all the Google C++ code is indented with only two spaces per
00:13:47,520 --> 00:13:56,040 block. If you're not used to that, it's harder to, at a glance, understand the code,
00:13:56,040 --> 00:14:01,320 because the high-level structure is determined by the indentation.
00:14:01,320 --> 00:14:07,200 On the other hand, there are other programming languages where the indentation is eight spaces
00:14:07,200 --> 00:14:13,600 or a whole tab stop in sort of classic Unix. And to me, that looks weird, because you sort
00:14:13,600 --> 00:14:17,320 of, after three indent levels, you've got no room left.
00:14:17,320 --> 00:14:23,840 Well, there are some languages where the indentation is a recommendation. It's a stylistic one.
00:14:23,840 --> 00:14:29,720 The code compiles even without any indentation. And then Python, really, indentation is a
00:14:29,720 --> 00:14:33,080 fundamental part of the language, right?
00:14:33,080 --> 00:14:40,840 It doesn't have to be four spaces. You can code Python with two spaces per block or six
00:14:40,840 --> 00:14:50,240 spaces or 12 if you really want to go wild. But everything that belongs to the same block
00:14:50,240 --> 00:14:53,120 needs to be indented the same way.
00:14:53,120 --> 00:14:59,440 In practice, in most other languages, people recommend doing that anyway. If you look at
00:14:59,440 --> 00:15:09,160 C or Rust or C++, all those languages, Java, don't have a requirement of indentation. But
00:15:09,160 --> 00:15:17,160 except in extreme cases, they're just as anal about having their code properly indented.
00:15:17,160 --> 00:15:24,880 So any IDE that does syntax highlighting that works with Java or C++, they will yell at
00:15:24,880 --> 00:15:28,840 you aggressively if you don't do proper indentation.
00:15:28,840 --> 00:15:37,240 It suggests the proper indentation for you. In C, you type a few words and then you type
00:15:37,240 --> 00:15:46,480 a curly brace, which is their notion of begin an indented block. Then you hit return and
00:15:46,480 --> 00:15:51,960 then it automatically indents four or eight spaces depending on your style preferences
00:15:51,960 --> 00:15:54,040 or how your editor is configured.
00:15:54,040 --> 00:15:58,920 Was there a possible universe in which you considered having braces in Python?
00:15:58,920 --> 00:16:00,920 Absolutely, yeah.
00:16:00,920 --> 00:16:07,360 Was it 60-40, 70-30 in your head? What was the trade-off?
00:16:07,360 --> 00:16:17,400 For a long time, I was actually convinced that the indentation was just better. Without
00:16:17,400 --> 00:16:27,680 context, I would still claim that indentation is better. It reduces clutter. However, as
00:16:27,680 --> 00:16:35,880 I started to say earlier, context is almost everything. In the context of coding, most
00:16:35,880 --> 00:16:43,520 programmers are familiar with multiple languages, even if they're only good at one or two. Apart
00:16:43,520 --> 00:16:49,160 from Python and maybe Fortran, I don't know how that's written these days anymore, but
00:16:49,160 --> 00:16:56,480 all the other languages, Java, Rust, C, C++, JavaScript, TypeScript, Perl, are all using
00:16:56,480 --> 00:17:04,320 curly braces to sort of indicate blocks. So Python is the odd one out.
00:17:04,320 --> 00:17:11,380 So it's a radical idea. As a radical renegade revolutionary, do you still stand behind this
00:17:11,380 --> 00:17:20,680 idea of indentation versus braces? Can you dig into it a little bit more, why you still
00:17:20,680 --> 00:17:23,040 stand behind indentation?
00:17:23,040 --> 00:17:30,680 Because context is not the whole story. History, in a sense, provides more context. So for
00:17:30,680 --> 00:17:39,480 Python, there's no chance that we can switch. Python is using curly braces for something
00:17:39,480 --> 00:17:45,200 else, dictionaries mostly. We would get in trouble if we wanted to switch. Just like
00:17:45,200 --> 00:17:55,840 you couldn't redefine C to use indentation, even if you agree that indentation in a greenfield
00:17:55,840 --> 00:18:03,400 environment would be better, you can't change that kind of thing in a language. It's hard
00:18:03,400 --> 00:18:10,640 enough to reach agreement over much more minor details. I mean, in the past, in Python, we
00:18:10,640 --> 00:18:17,640 did have a big debate about tabs versus spaces and four spaces versus fewer or more. And
00:18:17,640 --> 00:18:24,640 we sort of came up with a recommended standard and sort of options for people who want to
00:18:24,640 --> 00:18:27,360 be different.
00:18:27,360 --> 00:18:32,480 But yes, I guess the thought experiment I'd like you to consider is if you could travel
00:18:32,480 --> 00:18:39,400 back through time, when the compatibility is not an issue, and you started Python all
00:18:39,400 --> 00:18:45,600 over again, can you make the case for indentation still?
00:18:45,600 --> 00:18:53,480 Well, it frees up a pair of matched brackets, of which there are never enough in the world
00:18:53,480 --> 00:19:03,880 for other purposes. It really makes the language slightly easier to grasp for people who don't
00:19:03,880 --> 00:19:13,160 already know another programming language. Because one of the things, and I mostly got
00:19:13,160 --> 00:19:21,040 this from my mentors who taught me programming language design in the earlier 80s, when you're
00:19:21,040 --> 00:19:31,120 teaching programming for the total newbie who has not coded before, not in any other
00:19:31,120 --> 00:19:43,520 language, a whole bunch of concepts in programming are very alien or sort of new and maybe very
00:19:43,520 --> 00:19:48,440 interesting but also distracting and confusing. And there are many different things you have
00:19:48,440 --> 00:20:00,680 to learn. In a typical 13-week programming course, if it's like really learning to program
00:20:00,680 --> 00:20:05,440 from scratch, you have to cover algorithms, you have to cover data structures, you have
00:20:05,440 --> 00:20:15,320 to cover syntax, you have to cover variables, loops, functions, recursion, classes, expressions,
00:20:15,320 --> 00:20:25,640 operators. There are so many concepts if you can spend a little less time having to worry
00:20:25,640 --> 00:20:35,680 about the syntax. The classic example was often, oh, the compiler complains every time
00:20:35,680 --> 00:20:42,800 I put a semicolon in the wrong place or I forget to put a semicolon. Python doesn't
00:20:42,800 --> 00:20:50,400 have semicolons in that sense. So you can't forget them. And you are also not sort of
00:20:50,400 --> 00:20:55,080 misled into putting them where they don't belong because you don't learn about them
00:20:55,080 --> 00:20:57,040 in the first place.
00:20:57,040 --> 00:21:03,360 The flip side of that is forcing the strictness onto the beginning programmer to teach them
00:21:03,360 --> 00:21:09,960 that programming values attention to details. You don't get to just write the way you write
00:21:09,960 --> 00:21:10,960 an English paper.
00:21:10,960 --> 00:21:16,320 There are details that they have to pay attention to, so I think they'll still get the message
00:21:16,320 --> 00:21:19,800 about paying attention to details.
00:21:19,800 --> 00:21:24,840 The interesting design choice, I still program quite a bit in PHP, and I'm sure there's other
00:21:24,840 --> 00:21:32,080 languages like this, but the dollar sign before a variable, that was always an annoying thing
00:21:32,080 --> 00:21:38,480 for me. It didn't quite fit into my understanding of why this is good for a programming language.
00:21:38,480 --> 00:21:42,000 I'm not sure if you ever thought about that one.
00:21:42,000 --> 00:21:50,760 That is a historical thing. There is a whole lineage of programming languages. PHP is one.
00:21:50,760 --> 00:22:00,200 Perl was one. The Unix shell is one of the oldest, or all the different shells. The dollar
00:22:00,200 --> 00:22:07,760 was invented for that purpose because the very earliest shells had a notion of scripting,
00:22:07,760 --> 00:22:13,200 but they did not have a notion of parameterizing the scripting.
00:22:13,200 --> 00:22:21,240 A script is just a few lines of text where each line of text is a command that is read
00:22:21,240 --> 00:22:27,240 by a very primitive command processor that then takes the first word on the line as the
00:22:27,240 --> 00:22:35,240 name of a program and passes all the rest of the line as text into the program for the
00:22:35,240 --> 00:22:40,600 program to figure out what to do with as arguments.
00:22:40,600 --> 00:22:47,840 By the time scripting was slightly more mature than the very first script, there was a convention
00:22:47,840 --> 00:22:55,720 that just like the first word on the line is the name of the program, the following
00:22:55,720 --> 00:23:06,320 words could be names of files. Input.txt, output.html, things like that.
00:23:06,320 --> 00:23:12,960 The next thing that happens is, oh, it would actually be really nice if we could have variables
00:23:12,960 --> 00:23:20,160 and especially parameters for scripts. Parameters are usually what starts this process.
00:23:20,160 --> 00:23:27,760 But now you have a problem because you can't just say the parameters are x, y, and z and
00:23:27,760 --> 00:23:34,240 so now we call, say, let's say x is the input file and y is the output file and let's forget
00:23:34,240 --> 00:23:41,080 about z for now. I have my program and I write program x, y, well that already has a meaning
00:23:41,080 --> 00:23:49,600 because that presumably means x itself is the file. It's a file name, it's not a variable
00:23:49,600 --> 00:23:52,680 name.
00:23:52,680 --> 00:24:00,200 And so the inventors of things like the Unix shell and I'm sure job command language at
00:24:00,200 --> 00:24:11,360 IBM before that had to use something that made it clear to the script processor, here
00:24:11,360 --> 00:24:18,440 is an x that is not actually the name of a file which you just pass through to the program
00:24:18,440 --> 00:24:26,080 you're running. Here is an x that is the name of a variable. And when you're writing a script
00:24:26,080 --> 00:24:33,760 processor, you try to keep it as simple as possible because certainly in the 50s and
00:24:33,760 --> 00:24:41,360 60s, the thing that interprets the script had to be a very small program because it
00:24:41,360 --> 00:24:48,720 had to fit in a very small part of memory. And so saying, oh, just look at each character
00:24:48,720 --> 00:24:53,960 and if you see a dollar sign, you jump to another section of the code and then you gobble
00:24:53,960 --> 00:24:59,120 up characters or say until the next space or something and you say that's the variable
00:24:59,120 --> 00:25:00,720 name.
00:25:00,720 --> 00:25:11,160 And so it was sort of invented as a clever way to make parsing of things that contain
00:25:11,160 --> 00:25:19,200 both variable and fixed parts very easy in a very simple script processor. It also helps
00:25:19,200 --> 00:25:28,520 even then, it also helps the human author and the human reader of the script to quickly
00:25:28,520 --> 00:25:34,800 see, oh, 20 lines down in the script, I see a reference to x, y, z. Oh, it has a dollar
00:25:34,800 --> 00:25:39,480 in front of it. So now we know that x, y, z must be one of the parameters of the script.
00:25:39,480 --> 00:25:45,320 Well, this is fascinating. Several things to say, which is the leftovers from the simple
00:25:45,320 --> 00:25:51,920 script processor languages are now in code bases like behind Facebook or behind most
00:25:51,920 --> 00:25:55,920 of the backend. I think PHP probably still runs most of the backend of the internet.
00:25:55,920 --> 00:26:00,640 Oh, yeah. Yeah, I think there's a lot of it in Wikipedia too, for example.
00:26:00,640 --> 00:26:05,400 It's funny that those decisions, or not funny, it's fascinating that those decisions permeate
00:26:05,400 --> 00:26:07,380 through time.
00:26:07,380 --> 00:26:15,080 Kind of like biological systems, right? I mean, the sort of, the inner workings of DNA
00:26:15,080 --> 00:26:20,360 have been stable for, well, I don't know how long it was, like 300 million years, half
00:26:20,360 --> 00:26:27,080 a billion years. And there are all sorts of weird quirks there that don't make a lot of
00:26:27,080 --> 00:26:32,960 sense if you were to design a system like self-replicating molecules from scratch.
00:26:32,960 --> 00:26:41,000 Well, that system has a lot of interesting resilience. It has redundancy that results,
00:26:41,000 --> 00:26:45,640 like it messes up in interesting ways that still is resilient when you look at the system
00:26:45,640 --> 00:26:52,960 level of the organism. Code doesn't necessarily have that, a computer programming code.
00:26:52,960 --> 00:27:01,000 You'd be surprised how much resilience modern code has. I mean, if you look at the number
00:27:01,000 --> 00:27:11,840 of bugs per line of code, even in very well-tested code that in practice works just fine, there
00:27:11,840 --> 00:27:19,840 are actually lots of things that don't work fine. And there are error-correcting or self-correcting
00:27:19,840 --> 00:27:23,000 mechanisms at many levels.
00:27:23,000 --> 00:27:25,000 Including probably the user of the code.
00:27:25,000 --> 00:27:32,080 Well, in the end, the user who sort of is told, well, you got to reboot your PC is part
00:27:32,080 --> 00:27:40,840 of that system. And a slightly less drastic thing is reload the page, which we all know
00:27:40,840 --> 00:27:47,320 how to do without thinking about it when something weird happens. You try to reload a few times
00:27:47,320 --> 00:27:50,400 before you say, oh, there's something really weird.
00:27:50,400 --> 00:27:54,280 Or try to click the button again if the first time didn't work.
00:27:54,280 --> 00:27:59,480 Well, yeah, we should all have learned not to do that because that's probably just going
00:27:59,480 --> 00:28:01,080 to turn the light back off.
00:28:01,080 --> 00:28:08,400 Yeah, true. So do it three times. That's the right lesson. And I wonder how many people
00:28:08,400 --> 00:28:16,800 actually like the dollar sign. Like you said, it is documentation. So to me, it's whatever
00:28:16,800 --> 00:28:23,080 the opposite of syntactic sugar is, syntactic poison. To me, it is such a pain in the ass
00:28:23,080 --> 00:28:29,160 that I have to type in a dollar sign. Also super error prone. So it's not self-documenting.
00:28:29,160 --> 00:28:34,280 It's like a bug generating thing. It is a kind of documentation that's the pro and the
00:28:34,280 --> 00:28:37,440 con is it's a source of a lot of bugs.
00:28:37,440 --> 00:28:42,280 But actually I have to ask you, this is a really interesting idea of bugs per line of
00:28:42,280 --> 00:28:48,360 code. If you look at all the computer systems out there, from the code that runs nuclear
00:28:48,360 --> 00:28:54,320 weapons to the code that runs all the amazing companies that you've been involved with and
00:28:54,320 --> 00:29:00,640 not, the code that runs Twitter and Facebook and Dropbox and Google and Microsoft Windows
00:29:00,640 --> 00:29:08,720 and so on, and we like laid out, wouldn't that be a cool like table, bugs per line of
00:29:08,720 --> 00:29:15,360 code? And let's put like actual companies aside. Do you think we'd be surprised by the
00:29:15,360 --> 00:29:18,640 number we see there for all these companies?
00:29:18,640 --> 00:29:26,720 That depends on whether you've ever read about research that's been done in this area before.
00:29:26,720 --> 00:29:34,080 And I didn't know that the last time I saw some research like that, there was probably
00:29:34,080 --> 00:29:40,720 in the nineties and the research might've been done in the eighties. But the conclusion
00:29:40,720 --> 00:29:50,960 was across a wide range of different software, different languages, different companies,
00:29:50,960 --> 00:29:58,040 different development styles. The number of bugs is always, I think it's in the order
00:29:58,040 --> 00:30:07,480 of about one bug per thousand lines in sort of mature software that is considered as good
00:30:07,480 --> 00:30:08,480 as it gets.
00:30:08,480 --> 00:30:13,560 Can I give you some facts here? There's a lot of good papers. So you said mature software,
00:30:13,560 --> 00:30:23,200 right? So here's a report from a programming analytics company. Now this is from a developer
00:30:23,200 --> 00:30:28,880 perspective. Let me just say what it says, because this is very weird and surprising.
00:30:28,880 --> 00:30:36,320 On average, a developer creates 70 bugs per 1000 lines of code. 15 bugs per 1000 lines
00:30:36,320 --> 00:30:41,760 of code find their way to the customers. This is in the software they've analyzed.
00:30:41,760 --> 00:30:45,360 I was wrong by an order of magnitude there.
00:30:45,360 --> 00:30:50,480 Fixing a bug takes 30 times longer than writing a line of code. That I can believe.
00:30:50,480 --> 00:30:51,480 Yeah, totally.
00:30:51,480 --> 00:30:57,920 75% of a developer's time is spent on debugging. That's for an average developer. They analyze
00:30:57,920 --> 00:31:08,320 this 1500 hours a year. In the US alone, $113 billion is spent annually on identifying and
00:31:08,320 --> 00:31:10,800 fixing bugs.
00:31:10,800 --> 00:31:15,800 And I imagine this is marketing literature for someone who claims to have a golden bullet
00:31:15,800 --> 00:31:23,280 or a silver bullet that makes all that investment in fixing bugs go away. But that is usually
00:31:23,280 --> 00:31:27,200 not going to. That's not going to happen.
00:31:27,200 --> 00:31:30,920 But they're, I mean, they're referencing a lot of stuff, of course, but it is a page
00:31:30,920 --> 00:31:38,600 that is, you know, there's a contact us button at the bottom. Presumably, if you just spend
00:31:38,600 --> 00:31:44,640 a little bit less than $100 billion, we're willing to solve the problem for you. Right.
00:31:44,640 --> 00:31:48,640 And there's also a report on Stack Exchange and Stack Overflow on the exact same topic.
00:31:48,640 --> 00:31:53,200 But when I open it up at the moment, the page says Stack Overflow is currently offline for
00:31:53,200 --> 00:31:54,200 maintenance.
00:31:54,200 --> 00:31:55,200 Oh, that is ironic.
00:31:55,200 --> 00:32:02,120 Yes. By the way, their error page is awesome. Anyway. I mean, can you believe that number
00:32:02,120 --> 00:32:03,120 of bugs?
00:32:03,120 --> 00:32:04,120 Oh, absolutely.
00:32:04,120 --> 00:32:09,520 Isn't that scary? That's 70 bugs per 1000 lines of code. So even 10 bugs per 1000 lines
00:32:09,520 --> 00:32:10,520 of code.
00:32:10,520 --> 00:32:14,480 Well, that's about one bug after every 15 lines. And that's when you're first typing
00:32:14,480 --> 00:32:15,480 it in.
00:32:15,480 --> 00:32:22,080 Yeah, from a developer, but like, how many bugs are going to be found? If you're typing
00:32:22,080 --> 00:32:23,080 it in?
00:32:23,080 --> 00:32:29,760 The development process is extremely iterative. Typically, you don't make a plan for what
00:32:29,760 --> 00:32:36,120 software you're going to release a year from now. And work out all the details, because
00:32:36,120 --> 00:32:46,280 actually all the details themselves consist, they sort of compose a program. And that being
00:32:46,280 --> 00:32:53,720 a program, all your plans will have bugs in them too, and inaccuracies. But what you actually
00:32:53,720 --> 00:33:04,000 do is, you do a bunch of typing. And I'm actually really, I'm a really bad typist. That's just,
00:33:04,000 --> 00:33:06,280 I've never learned to type with 10 fingers.
00:33:06,280 --> 00:33:08,560 How many do you use?
00:33:08,560 --> 00:33:15,600 Well, I use all 10 of them, but not very well. But I never took a typing class and I never
00:33:15,600 --> 00:33:22,160 sort of corrected that. So the first time I seriously learned, I had to learn the layout
00:33:22,160 --> 00:33:29,160 of a QWERTY keyboard was actually in college, in my first programming classes, where we
00:33:29,160 --> 00:33:39,200 used punch cards. And so with my two fingers, I sort of pecked out my code.
00:33:39,200 --> 00:33:46,800 Watch anyone give you a little coding demonstration. They'll have to produce like four lines of
00:33:46,800 --> 00:33:53,760 code. And now see how many times they use the backspace key. Yeah, because they made
00:33:53,760 --> 00:34:03,680 a mistake. And some people, especially when someone else is looking, will backspace over
00:34:03,680 --> 00:34:12,480 20, 30, 40 characters to fix a typo earlier in a line. If you're slightly more experienced,
00:34:12,480 --> 00:34:17,040 of course you use your arrow buttons to go or your mouse to, but the mouse is usually
00:34:17,040 --> 00:34:25,120 slower than the arrows. But a lot of people, when they type a 20 character word, which
00:34:25,120 --> 00:34:31,400 is not unusual, and they realize they made a mistake at the start of the word, they backspace
00:34:31,400 --> 00:34:36,680 over the whole thing and then retype it. And sometimes it takes three, four times to get
00:34:36,680 --> 00:34:44,440 it right. So I don't know what your definition of bug is. Arguably mistyping a word and then
00:34:44,440 --> 00:34:51,920 correcting it immediately is not a bug. On the other hand, you already do sort of lose
00:34:51,920 --> 00:35:00,120 time and every once in a while, there's sort of a typo that you don't get in that process.
00:35:00,120 --> 00:35:05,680 And now you've typed like 10 lines of code and somewhere in the middle of it, you don't
00:35:05,680 --> 00:35:12,820 know where yet, is a typo or maybe a thinko where you forgot that you had to initialize
00:35:12,820 --> 00:35:14,760 a variable or something.
00:35:14,760 --> 00:35:17,800 But those are two different things. And I would say, yes, you have to actually run the
00:35:17,800 --> 00:35:23,200 code to discover that typo. But forgetting to initialize a variable is a fundamentally
00:35:23,200 --> 00:35:27,960 different thing because that thing can go undiscovered.
00:35:27,960 --> 00:35:32,560 That depends on the language. In Python, it will not. And sort of modern compilers are
00:35:32,560 --> 00:35:36,720 usually pretty good at catching that, even for C.
00:35:36,720 --> 00:35:45,200 So for that specific thing, but actually deeper, there might be another variable that is initialized,
00:35:45,200 --> 00:35:52,360 but logically speaking, the one you meant related. It's like name the same, but it's
00:35:52,360 --> 00:35:58,240 a different thing. And you forgot to initialize whatever, some counter or some basic variable
00:35:58,240 --> 00:35:59,240 they're using.
00:35:59,240 --> 00:36:02,000 I can tell that you've coded.
00:36:02,000 --> 00:36:06,480 By the way, I should mention that I use a Kinesis keyboard, which has the backspace
00:36:06,480 --> 00:36:13,960 under the thumb. And one of the biggest reasons I use that keyboard is because you realize
00:36:13,960 --> 00:36:22,320 in order to use the backspace on a usual keyboard, you have to stretch your pinky out. And like
00:36:22,320 --> 00:36:27,840 the, for most normal keyboards, the backspace is under the pinky. And so I don't know if
00:36:27,840 --> 00:36:34,320 people realize the pain they go through in their life because of the backspace key being
00:36:34,320 --> 00:36:38,200 so far away. So with the Kinesis, it's right under the thumb, so you don't have to actually
00:36:38,200 --> 00:36:40,680 move your hands, the backspace and the delete.
00:36:40,680 --> 00:36:47,200 What do you do if you're ever not with your own keyboard and you have to use someone else's
00:36:47,200 --> 00:36:50,560 PC keyboard that has a standard layout?
00:36:50,560 --> 00:36:54,880 So first of all, it turns out that you can actually go your whole life always having
00:36:54,880 --> 00:36:57,040 the keyboard with you.
00:36:57,040 --> 00:37:01,800 So this, well, except for that, that little tablet that you're using for note taking right
00:37:01,800 --> 00:37:02,800 now, right?
00:37:02,800 --> 00:37:07,720 Yeah. So it's very inefficient note taking, but I'm not, I'm just looking stuff up. But
00:37:07,720 --> 00:37:14,080 in most cases, I would be actually using the keyboard here right now. I just don't anticipate,
00:37:14,080 --> 00:37:18,480 you have to calculate how much typing do you anticipate? If I anticipate quite a bit, then
00:37:18,480 --> 00:37:21,640 I'll just, I have a keyboard with me.
00:37:21,640 --> 00:37:22,640 You pull it out.
00:37:22,640 --> 00:37:30,800 And same with, I mean, the embarrassing, I've accepted being the weirdo that I am, but when
00:37:30,800 --> 00:37:36,080 I go on an airplane and I anticipate to do programming or a lot of typing, I will have
00:37:36,080 --> 00:37:42,640 a laptop that will pull out a Kinesis keyboard in addition to the laptop. And it's just who
00:37:42,640 --> 00:37:49,440 I am. You have to accept who you are. But also it's, you know, for a lot of people,
00:37:49,440 --> 00:37:54,440 for me certainly, there's a comfort space where there's a certain kind of setups that
00:37:54,440 --> 00:38:01,400 are maximized productivity. And it's like some people have a warm blanket that they
00:38:01,400 --> 00:38:08,040 like when they watch a movie. I like the Kinesis keyboard. It takes me to a place of focus.
00:38:08,040 --> 00:38:15,040 And I still mostly, I'm trying to make sure I use the state-of-the-art IDs for everything,
00:38:15,040 --> 00:38:21,400 but my comfort place, just like the Kinesis keyboard is still Emacs.
00:38:21,400 --> 00:38:28,920 So I still use, I still, I mean, that's one of some of the debates I have with myself
00:38:28,920 --> 00:38:34,760 about everything from a technology perspective is how much to hold on to the tools you're
00:38:34,760 --> 00:38:41,240 comfortable with versus how much to invest in using modern tools. And the signal that
00:38:41,240 --> 00:38:46,320 the communities provide you with is the noisy one, because a lot of people year to year
00:38:46,320 --> 00:38:52,320 get excited about new tools and you have to make a prediction. Are these tools defining
00:38:52,320 --> 00:38:56,920 a new generation or something that will transform programming? Or is this just a fad that will
00:38:56,920 --> 00:39:04,320 pass? Certainly with JavaScript frameworks and front-end and back-end of the web, there's
00:39:04,320 --> 00:39:11,440 a lot of different styles that came and went. I remember learning, what was it called, ActionScript.
00:39:11,440 --> 00:39:19,040 I remember for Flash, learning how to program in Flash, learning how to design, do graphic
00:39:19,040 --> 00:39:23,600 animation, all that kind of stuff with Flash. Same with Java applets. I remember creating
00:39:23,600 --> 00:39:27,880 quite a lot of Java applets, thinking that this potentially defines the future of the
00:39:27,880 --> 00:39:29,600 web, and it did not.
00:39:29,600 --> 00:39:38,400 Well, you know, in most cases like that, the particular technology eventually gets replaced,
00:39:38,400 --> 00:39:49,280 but many of the concepts that the technology introduced or made accessible first are preserved,
00:39:49,280 --> 00:39:55,920 of course, because yeah, we're not using Java applets anymore, but the notion of reactive
00:39:55,920 --> 00:40:05,340 web pages that sort of contain little bits of code that respond directly to something
00:40:05,340 --> 00:40:13,040 you do, like pressing a button or a link or hovering even, has certainly not gone away.
00:40:13,040 --> 00:40:22,560 And that those animations that were made painfully complicated with Flash, I mean, Flash was
00:40:22,560 --> 00:40:31,720 an innovation when it first came up, and when it was replaced by JavaScript equivalent stuff,
00:40:31,720 --> 00:40:37,940 it was a somewhat better way to do animations, but those animations are still there. Not
00:40:37,940 --> 00:40:47,680 all of them, but sort of, again, there is an evolution and often, so often with technology,
00:40:47,680 --> 00:40:56,120 but the technology that was eventually thrown away or replaced was still essential to sort
00:40:56,120 --> 00:41:03,600 of get started. There wouldn't be jet planes without propeller planes, I bet you.
00:41:03,600 --> 00:41:11,280 But from a user perspective, yes, from the feature set, yes, but from a programmer perspective,
00:41:11,280 --> 00:41:19,120 it feels like all the time I've spent with ActionScript, all the time I've spent with
00:41:19,120 --> 00:41:24,480 Java on the applet side for the GUI development, well, no, Java I have to push back, that was
00:41:24,480 --> 00:41:29,680 useful, because it transfers, but the Flash doesn't transfer, so some things you learn
00:41:29,680 --> 00:41:30,680 and invest time in.
00:41:30,680 --> 00:41:41,000 Yeah, what you learned, the skill you picked up learning ActionScript was sort of, it was
00:41:41,000 --> 00:41:49,720 perhaps a super valuable skill at the time you picked it up, if you learned ActionScript
00:41:49,720 --> 00:41:57,360 early enough, but that skill is no longer in demand.
00:41:57,360 --> 00:42:00,760 Well, that's the calculation you have to make when you're learning new things, like today
00:42:00,760 --> 00:42:06,960 people started learning programming, today I'm trying to see what are the new languages
00:42:06,960 --> 00:42:14,360 to try, what are the new systems to try, what are the new ideas to try, to keep improving.
00:42:14,360 --> 00:42:21,840 That's why we start when we're young, right? But that seems very true to me, that when
00:42:21,840 --> 00:42:27,600 you're young, you have your whole life ahead of you and you're allowed to make mistakes.
00:42:27,600 --> 00:42:34,680 In fact, you should feel encouraged to do a bit of stupid stuff. Try not to get yourself
00:42:34,680 --> 00:42:42,720 killed or seriously maimed, but try stuff that deviates from what everybody else is
00:42:42,720 --> 00:42:50,560 doing, and like 9 out of 10 times you'll just learn why everybody else is not doing that,
00:42:50,560 --> 00:42:58,080 or why everybody else is doing it some other way, and 1 out of 10 times you sort of, you
00:42:58,080 --> 00:43:03,160 discover something that's better or that somehow works. I mean, there are all sorts of crazy
00:43:03,160 --> 00:43:11,480 things that were invented by accident, by people trying stuff together.
00:43:11,480 --> 00:43:15,000 That's great advice to try random stuff, make a lot of mistakes.
00:43:15,000 --> 00:43:20,280 Once you're married with kids, you're probably going to be a little more risk-averse because
00:43:20,280 --> 00:43:26,340 now there's more at stake and you've already hopefully had some time where you were experimenting
00:43:26,340 --> 00:43:27,340 with crazy shit.
00:43:27,340 --> 00:43:32,560 I like how marriage and kids solidifies your choice of programming language. How does that
00:43:32,560 --> 00:43:37,560 work? The Robert Frost poem with the road less taken, which I think is misinterpreted
00:43:37,560 --> 00:43:43,480 by most people. But anyway, I feel like the choices you make early on, especially if you
00:43:43,480 --> 00:43:49,680 go all in, they're going to define the rest of your life's trajectory in a way that, like
00:43:49,680 --> 00:43:56,960 you basically are picking a camp. So if you invest a lot in PHP, if you invest a lot in
00:43:56,960 --> 00:44:07,200 .NET, if you invest a lot in JavaScript, you're going to stick there. That's your life journey.
00:44:07,200 --> 00:44:08,200 It's very hard to tell.
00:44:08,200 --> 00:44:18,920 Well, only as far as that technology remains relevant. I mean, if at age 16 you learn coding
00:44:18,920 --> 00:44:30,400 in C and by the time you're 26, C is like a dead language, then there's still time to
00:44:30,400 --> 00:44:37,280 switch. There's probably some kind of survivor bias or whatever it's called in sort of your
00:44:37,280 --> 00:44:43,960 observation that you pick a camp because there are many different camps to pick. And if you
00:44:43,960 --> 00:44:52,120 pick.NET, then you can coast for the rest of your life because that technology is now
00:44:52,120 --> 00:44:58,080 so ubiquitous, of course, that even if it's bound to die, it's going to take a very long
00:44:58,080 --> 00:44:59,080 time.
00:44:59,080 --> 00:45:06,600 Well, for me personally, I had a very difficult and in my own head, brave leap that I had
00:45:06,600 --> 00:45:13,040 to take relevant to our discussion, which is most of my life I programmed in C and C++.
00:45:13,040 --> 00:45:19,880 And so having that hammer, everything looked like a nail. So I would literally even do
00:45:19,880 --> 00:45:27,040 scripting in C++. I would create programs that do script like things. And when I first
00:45:27,040 --> 00:45:32,720 came to Google and before then, it became already before TensorFlow, before all of that,
00:45:32,720 --> 00:45:38,880 there was a growing realization that C++ is not the right tool for machine learning. We
00:45:38,880 --> 00:45:43,840 could talk about why that is. It's unclear why that is. A lot of things has to do with
00:45:43,840 --> 00:45:49,080 community and culture and how it emerges and stuff like that. But for me to decide to take
00:45:49,080 --> 00:45:56,840 the leap to Python, like all out, basically switch completely from C++ except for highly
00:45:56,840 --> 00:46:04,600 performant robotics applications. There was still a culture of C++ in the space of robotics.
00:46:04,600 --> 00:46:11,700 That was a big leap. Like I had to, you know, like people have like existential crises or
00:46:11,700 --> 00:46:17,120 midlife crises or whatever. You have to realize almost like walking away from a person you
00:46:17,120 --> 00:46:23,860 love because I was sure that C++ would have to be a lifelong companion. For a lot of problems
00:46:23,860 --> 00:46:28,480 I would want to solve, C++ would be there. And it was a question to say, well, that might
00:46:28,480 --> 00:46:32,840 not be the case because C++ is still one of the most popular languages in the world, one
00:46:32,840 --> 00:46:39,000 of the most used, one of the most depended on. It's also still evolving quite a bit.
00:46:39,000 --> 00:46:48,440 I mean, that is not a sort of a fossilizing community. They are doing great innovative
00:46:48,440 --> 00:46:53,640 work actually. A lot. But yet that sort of their innovations are hard to follow if you're
00:46:53,640 --> 00:46:59,320 not already a hardcore C++ user. Well, this was the thing. It pulls you in. It's a rabbit
00:46:59,320 --> 00:47:04,880 hole. I was a hardcore. The old metaprogramming, template programming, like I would start using
00:47:04,880 --> 00:47:12,680 the modern C++ as it developed, right? Not just the shared pointer and the garbage collection
00:47:12,680 --> 00:47:16,880 that makes it easier for you to work with some of the flaws, but the detail, like the
00:47:16,880 --> 00:47:22,760 metaprogramming, the crazy stuff that's coming out there. But then you have to just empirically
00:47:22,760 --> 00:47:30,920 look and step back and say, what language am I more productive in? Sorry to say, what
00:47:30,920 --> 00:47:38,000 language do I enjoy my life with more? And readability and able to think through and
00:47:38,000 --> 00:47:43,960 all that kind of stuff. Those questions are harder to ask when you already have a loved
00:47:43,960 --> 00:47:51,840 one, which in my case was C++. And then there's Python, like that meme. The grass is greener
00:47:51,840 --> 00:47:58,000 on the other side. Am I just infatuated with a new, fad, new, cool thing? Or is this actually
00:47:58,000 --> 00:48:01,560 going to make my life better? And I think a lot of people face that kind of decision.
00:48:01,560 --> 00:48:07,960 It was a difficult decision for me when I made it. At this time, it's an obvious switch
00:48:07,960 --> 00:48:13,880 if you're into machine learning. But at that time, it wasn't quite yet so obvious. So it
00:48:13,880 --> 00:48:19,600 was a risk. And you have the same kind of stuff with... I still, because of my connection
00:48:19,600 --> 00:48:27,960 to WordPress, I still do a lot of backend programming in PHP. And the question is, you
00:48:27,960 --> 00:48:34,440 know, Node.js, Python, do you switch backend to any of those programmings? There's the
00:48:34,440 --> 00:48:42,760 case for Node.js for me. Well, more and more and more of the frontend, it runs in JavaScript.
00:48:42,760 --> 00:48:47,080 And fascinating cool stuff is done in JavaScript. Maybe use the same programming language for
00:48:47,080 --> 00:48:52,800 the backend as well. The case for Python for the backend is, well, you're doing so much
00:48:52,800 --> 00:48:59,400 programming outside of the web in Python. So maybe use Python for the backend. And then
00:48:59,400 --> 00:49:05,240 the case for PHP, well, most of the web still runs in PHP. You have a lot of experience
00:49:05,240 --> 00:49:12,200 with PHP. Why fix something that's not broken? Those are my own personal struggles, but I
00:49:12,200 --> 00:49:16,280 think they reflect the struggles of a lot of people with different programming languages,
00:49:16,280 --> 00:49:19,840 with different problems they're trying to solve. It's a weird one.
00:49:19,840 --> 00:49:25,640 And there's not a single answer, right? Because depending on how much time you have to learn
00:49:25,640 --> 00:49:31,960 new stuff, where you are in your life, what you're currently working on, who you want
00:49:31,960 --> 00:49:40,800 to work with, what communities you like, there's not one right choice. Maybe if you sort of,
00:49:40,800 --> 00:49:45,880 if you can look back 20 years, you can say, well, that whole detour through ActionScript
00:49:45,880 --> 00:49:56,920 was a waste of time, but nobody could know that. So you can't beat yourself up over that.
00:49:56,920 --> 00:50:03,200 You just need to accept that not every choice you make is going to be perfect. Maybe sort
00:50:03,200 --> 00:50:13,440 of keep a plan B in the back of your mind, but don't overthink it. Don't try to sort
00:50:13,440 --> 00:50:22,760 of create a spreadsheet where you're trying to estimate, well, if I learn this language,
00:50:22,760 --> 00:50:28,960 I expect to make X million dollars in a lifetime. And if I learn that language, I expect to
00:50:28,960 --> 00:50:35,960 make Y million dollars in a lifetime. And which is higher and which has more risk and
00:50:35,960 --> 00:50:47,860 where is the chance? It's like picking a stock. Kind of, kind of, but I think with stocks,
00:50:47,860 --> 00:50:55,240 you can do, diversifying your investment is good. With productivity in life, boy, that
00:50:55,240 --> 00:51:01,840 spreadsheet is possible to construct. Like if you actually carefully analyze what your
00:51:01,840 --> 00:51:07,160 interests in life are, where you think you can maximally impact the world, there really
00:51:07,160 --> 00:51:12,360 is better and worse choices for a programming language that are not just about the syntax,
00:51:12,360 --> 00:51:18,560 but about the community, about where you predict the community's headed, what large systems
00:51:18,560 --> 00:51:19,840 are programmed in that.
00:51:19,840 --> 00:51:24,340 But can you create that spreadsheet? Because that's sort of, you're mentioning a whole
00:51:24,340 --> 00:51:29,200 bunch of inputs that go into that spreadsheet where you have to estimate things that are
00:51:29,200 --> 00:51:36,400 very hard to measure and even harder. I mean, they're hard to measure retroactively and
00:51:36,400 --> 00:51:42,920 they're even harder to predict. Like, what is the better community? Well, better is one
00:51:42,920 --> 00:51:49,160 of those incredibly difficult words. What's better for you is not better for someone else.
00:51:49,160 --> 00:51:53,920 No, but we're not doing a public speech about what's better. We're doing a personal spiritual
00:51:53,920 --> 00:52:00,360 journey. I can determine a circle of friends, circle one and circle two, and I can have
00:52:00,360 --> 00:52:05,960 a bunch of parties with one and a bunch of parties with two, and then write down or take
00:52:05,960 --> 00:52:11,320 a mental note of what made me happier, right? And that, you know, you have, if you're a
00:52:11,320 --> 00:52:15,600 machine learning person, you want to say, okay, I want to build a large company that
00:52:15,600 --> 00:52:22,240 does, that is grounded in machine learning, but also has a sexy interface that has a large
00:52:22,240 --> 00:52:26,960 impact in the world. What languages do I use? You look at what Facebook is using, you look
00:52:26,960 --> 00:52:32,960 at what Twitter is using. Then you look at performant, more newer languages like Rust,
00:52:32,960 --> 00:52:37,760 or you look at languages that have taken, that most of the community uses in machine
00:52:37,760 --> 00:52:41,840 learning space, that's Python. And you can like think through, you can hang out and think
00:52:41,840 --> 00:52:47,560 through it. And it's always a invest, and the level of activity of the community is
00:52:47,560 --> 00:52:52,220 also really interesting. Like you said, C++ and Python are super active in terms of the
00:52:52,220 --> 00:52:54,680 development of the language itself.
00:52:54,680 --> 00:52:59,680 But do you think that you can make objective choices there?
00:52:59,680 --> 00:53:04,560 No, no, no. But there's a gut you build up. Like, don't you, don't you believe in that
00:53:04,560 --> 00:53:05,560 gut feeling?
00:53:05,560 --> 00:53:09,900 Oh, everything is very subjective. And yes, you most certainly can have a gut feeling
00:53:09,900 --> 00:53:14,560 and your gut can also be wrong. That's why there are billions of people because they're
00:53:14,560 --> 00:53:21,080 not all right. I mean, clearly there are more people living in the Bay Area who have plans
00:53:21,080 --> 00:53:26,760 to sort of create a Google sized company, then there's room in the world for Google
00:53:26,760 --> 00:53:33,280 sized companies. And they're going to have to duke it out in the market space.
00:53:33,280 --> 00:53:38,320 And there's many more choices than just the programming language. Speaking of which, let's
00:53:38,320 --> 00:53:44,840 go back to the boat with the fisherman who's tuned out long ago. Let's talk to the programmer.
00:53:44,840 --> 00:53:50,560 Let's jump around and go back to CPython that we tried to define as the reference implementation.
00:53:50,560 --> 00:53:54,520 And one of the big things that's coming out in 3.11, what's the right way?
00:53:54,520 --> 00:54:02,120 We tend to say 3.11, because it really was like, we went 3.8, 3.9, 3.10, 3.11, and we're
00:54:02,120 --> 00:54:04,120 planning to go up to 3.99.
00:54:04,120 --> 00:54:07,860 99? What happens after 99?
00:54:07,860 --> 00:54:11,000 Probably just 3.100, if I make it there.
00:54:11,000 --> 00:54:17,280 Okay. And go all the way to 420. I got it. Forever Python v3. We'll talk about 4, but
00:54:17,280 --> 00:54:20,360 more for fun.
00:54:20,360 --> 00:54:26,880 So 3.11 is coming out. One of the big sexy things in it is it'll be much faster. So how
00:54:26,880 --> 00:54:32,600 did you, beyond hiring a great team or working with a great team, make it faster? What are
00:54:32,600 --> 00:54:37,360 some ideas that makes it faster?
00:54:37,360 --> 00:54:45,640 It has to do with simplicity of software versus performance. And so even though C is known
00:54:45,640 --> 00:54:55,480 to be a low-level language, which is great for writing a high-performance language interpreter,
00:54:55,480 --> 00:55:06,280 when I originally started Python or CPython, I didn't expect there would be great success
00:55:06,280 --> 00:55:21,480 and fame in my future. So I tried to get something working and useful in about three months.
00:55:21,480 --> 00:55:30,120 And so I cut corners. I borrowed ideas left and right when it comes to language design
00:55:30,120 --> 00:55:42,800 as well as implementation. I also wrote much of the code as simple as it could be. And
00:55:42,800 --> 00:55:50,400 there are many things that you can code more efficiently by adding more code. It's a bit
00:55:50,400 --> 00:56:03,080 of a time-space trade-off where you can compute a certain thing from a small number of inputs.
00:56:03,080 --> 00:56:08,760 And every time you get presented with a new input, you do the whole computation from the
00:56:08,760 --> 00:56:15,360 top. That can be simple-looking code. It's easy to understand. It's easy to reason about
00:56:15,360 --> 00:56:25,200 that you can tell quickly that it's correct, at least in the mathematical sense of correct.
00:56:25,200 --> 00:56:36,280 Because it's implemented in C, maybe it performs relatively well. But over time, as the requirements
00:56:36,280 --> 00:56:44,520 for that code and the need for performance go up, you might be able to rewrite that same
00:56:44,520 --> 00:56:53,560 algorithm using more memory, maybe remember previous results so you don't have to recompute
00:56:53,560 --> 00:57:00,960 everything from scratch. Like the classic example is computing prime numbers. Like,
00:57:00,960 --> 00:57:07,760 is 10 a prime number? Well, you sort of, is it divisible by 2? Is it divisible by 3? Is
00:57:07,760 --> 00:57:14,840 it divisible by 4? And we go all the way to, is it divisible by 9? And it is not, well
00:57:14,840 --> 00:57:20,840 actually 10 is divisible by 2, so there we stop, but say 11. Is it divisible by 10? The
00:57:20,840 --> 00:57:27,000 answer is no, 10 times in a row. So now we know 11 is a prime number.
00:57:27,000 --> 00:57:32,680 On the other hand, if we already know that 2, 3, 5 and 7 are prime numbers, and you know
00:57:32,680 --> 00:57:38,880 a little bit about the mathematics of how prime numbers work, you know that if you have
00:57:38,880 --> 00:57:44,960 a rough estimate for the square root of 11, you don't actually have to check is it divisible
00:57:44,960 --> 00:57:50,800 by 4 or is it divisible by 5. All you have to check in the case of 11 is, is it divisible
00:57:50,800 --> 00:57:59,400 by 2? Is it divisible by 3? Because take 12. If it's divisible by 4, well 12 divided by
00:57:59,400 --> 00:58:06,160 4 is 3, so you should have come across the question, is it divisible by 3 first. So if
00:58:06,160 --> 00:58:12,400 you know basically nothing about prime numbers except the definition, maybe you go for x
00:58:12,400 --> 00:58:24,800 from 2 through n minus 1, is n divisible by x? And then at the end, if you got all nos
00:58:24,800 --> 00:58:29,920 for every single one of those questions, you know, oh, it must be a prime number. Well,
00:58:29,920 --> 00:58:35,760 the first thing is you can stop iterating when you find a yes answer. And the second
00:58:35,760 --> 00:58:43,360 is you can also stop iterating when you have reached the square root of n, because you
00:58:43,360 --> 00:58:48,840 know that if it has a divisor larger than the square root, it must also have a divisor
00:58:48,840 --> 00:58:55,160 smaller than the square root. Then you say, oh, except for 2, we don't need to bother
00:58:55,160 --> 00:59:00,040 with checking for even numbers because all even numbers are divisible by 2. So if it's
00:59:00,040 --> 00:59:06,080 divisible by 4, we would already have come across the question, is it divisible by 2?
00:59:06,080 --> 00:59:11,800 And so now you go special case check, is it divisible by 2? And then you just check 3,
00:59:11,800 --> 00:59:19,360 4, 5, 6, 7, 11. And so now you've sort of reduced your search space by 50% again by
00:59:19,360 --> 00:59:26,320 skipping all the even numbers except for 2. If you think a bit more about it, or you just
00:59:26,320 --> 00:59:33,080 read in your book about the history of math, one of the first algorithms ever written down,
00:59:33,080 --> 00:59:38,360 all you have to do is check, is it divisible by any of the previous prime numbers that
00:59:38,360 --> 00:59:47,280 are smaller than the square root? And before you get to a better algorithm than that, you
00:59:47,280 --> 00:59:54,400 have to have several PhDs in discrete math. So that's as much as I know.
00:59:54,400 --> 00:59:58,600 So of course that same story applies to a lot of other algorithms. String matching is
00:59:58,600 --> 01:00:05,340 a good example of how to come up with an efficient algorithm. And sometimes the more efficient
01:00:05,340 --> 01:00:10,200 algorithm is not so much more complex than the inefficient one. But that's an art and
01:00:10,200 --> 01:00:17,080 it's not always the case. In the general cases, the more performant the algorithm, the more
01:00:17,080 --> 01:00:20,800 complex it's going to be. There's a kind of trade-off.
01:00:20,800 --> 01:00:27,840 The simpler algorithms are also the ones that people invent first. Because when you're looking
01:00:27,840 --> 01:00:35,320 for a solution, you look at the simplest way to get there first. And so if there is a simple
01:00:35,320 --> 01:00:42,520 solution, even if it's not the best solution, not the fastest or the most memory efficient
01:00:42,520 --> 01:00:50,960 or whatever, a simple solution, and simple is fairly subjective, but mathematicians have
01:00:50,960 --> 01:00:58,400 also thought about what is a good definition for simple in the case of algorithms.
01:00:58,400 --> 01:01:07,400 But the simpler solutions tend to be easier to follow for other programmers who haven't
01:01:07,400 --> 01:01:13,680 made a study of a particular field. And when I started with Python, I was a good programmer
01:01:13,680 --> 01:01:19,760 in general. I knew sort of basic data structures. I knew the C language pretty well. But there
01:01:19,760 --> 01:01:30,360 were many areas where I was only somewhat familiar with the state of the art.
01:01:30,360 --> 01:01:37,780 And so I picked, in many cases, the simplest way I could solve a particular sub-problem.
01:01:37,780 --> 01:01:43,800 Because when you're designing and implementing a language, you have many hundreds of little
01:01:43,800 --> 01:01:51,960 problems to solve. And you have to have solutions for every one of them before you can sort
01:01:51,960 --> 01:01:56,720 of say, I've invented a programming language.
01:01:56,720 --> 01:02:03,080 First of all, so CPython, what kind of things does it do? It's an interpreter. It takes
01:02:03,080 --> 01:02:08,320 in this readable language that we talked about, that is Python. What is it supposed to do?
01:02:08,320 --> 01:02:18,420 The interpreter basically, it's sort of a recipe for understanding recipes. So instead
01:02:18,420 --> 01:02:29,120 of a recipe that says bake me a cake, we have a recipe for, well, given the text of a program,
01:02:29,120 --> 01:02:34,660 how do we run that program? And that is sort of the recipe for building a computer.
01:02:34,660 --> 01:02:42,480 The recipe for the baker and the chef. What are the algorithmically tricky things that
01:02:42,480 --> 01:02:46,840 happen to be low-hanging fruit that could be improved on? Maybe throw out the history
01:02:46,840 --> 01:02:53,560 of Python, but also now, how is it possible that 3.11 in year 2022 is possible to get
01:02:53,560 --> 01:02:57,240 such a big performance improvement?
01:02:57,240 --> 01:03:08,720 We focused on a few areas where we still felt there was low-hanging fruit. The biggest one
01:03:08,720 --> 01:03:16,220 is actually the interpreter itself. And this has to do with details of how Python is defined.
01:03:16,220 --> 01:03:20,840 So I didn't know if the fisherman is going to follow this story.
01:03:20,840 --> 01:03:27,080 He already jumped off the boat. He's bored, he's stupid.
01:03:27,080 --> 01:03:32,640 Python is actually, even though it's always called an interpreted language, there's also
01:03:32,640 --> 01:03:38,880 a compiler in there. It just doesn't compile to machine code. It compiles to bytecode,
01:03:38,880 --> 01:03:45,680 which is sort of code for an imaginary computer that is called the Python interpreter.
01:03:45,680 --> 01:03:50,940 So it's compiling code that is more easily digestible by the interpreter or is digestible
01:03:50,940 --> 01:03:51,940 at all?
01:03:51,940 --> 01:03:56,780 It is the code that is digested by the interpreter. That's the compiler. We tweaked very minor
01:03:56,780 --> 01:04:04,040 bits of the compiler. Almost all the work was done in the interpreter, because when
01:04:04,040 --> 01:04:10,600 you have a program, you compile it once and then you run the code a whole bunch of times.
01:04:10,600 --> 01:04:17,480 Or maybe there's one function in the code that gets run many times.
01:04:17,480 --> 01:04:24,800 Now I know that sort of people who know this field are expecting me to at some point say,
01:04:24,800 --> 01:04:30,280 we built a just-in-time compiler. Actually, we didn't. We just made the interpreter a
01:04:30,280 --> 01:04:32,240 little more efficient.
01:04:32,240 --> 01:04:34,960 What's a just-in-time compiler?
01:04:34,960 --> 01:04:41,840 That is a thing from the Java world, although it's now applied to almost all programming
01:04:41,840 --> 01:04:44,720 languages, especially interpreted ones.
01:04:44,720 --> 01:04:50,120 So you see the compiler inside Python, not like a just-in-time compiler, but it's a compiler
01:04:50,120 --> 01:04:57,040 that creates bytecode that is then fed to the interpreter. And the compiler, was there
01:04:57,040 --> 01:04:59,920 something interesting to say about the compiler? It's interesting that you haven't changed
01:04:59,920 --> 01:05:02,160 that, tweaked that at all, or much.
01:05:02,160 --> 01:05:10,080 We changed some parts of the bytecode, but not very much. And so we only had to change
01:05:10,080 --> 01:05:15,360 the parts of the compiler where we decided that the breakdown of a Python program in
01:05:15,360 --> 01:05:20,160 bytecode instructions had to be slightly different.
01:05:20,160 --> 01:05:31,480 But that didn't gain us the performance improvements. The performance improvements were like making
01:05:31,480 --> 01:05:40,240 the interpreter faster in part by sort of removing the fat from some internal data structures
01:05:40,240 --> 01:05:49,960 used by the interpreter. But the key idea is an adaptive specializing interpreter.
01:05:49,960 --> 01:05:53,880 Let's go. What is adaptive about it? What is specialized about it?
01:05:53,880 --> 01:05:58,960 Well, let me first talk about the specializing part, because the adaptive part is the sort
01:05:58,960 --> 01:06:07,480 of the second order effect, but they're both important. So bytecode is a bunch of machine
01:06:07,480 --> 01:06:13,480 instructions, but it's an imaginary machine. But the machine can do things like call a
01:06:13,480 --> 01:06:20,960 function, add two numbers, print a value. Those are sort of typical instructions in
01:06:20,960 --> 01:06:31,680 Python. And if we take the example of adding two numbers, actually in Python, the language,
01:06:31,680 --> 01:06:39,200 there's no such thing as adding two numbers. The compiler doesn't know that you're adding
01:06:39,200 --> 01:06:46,600 two numbers. You might as well be adding two strings or two lists or two instances of some
01:06:46,600 --> 01:06:54,240 user defined class that happened to implement this operator called add. That's a very interesting
01:06:54,240 --> 01:07:00,180 and fairly powerful mathematical concept. It's mostly a user interface trick because
01:07:00,180 --> 01:07:08,840 it means that a certain category of functions can be written using a single symbol, the
01:07:08,840 --> 01:07:15,120 plus sign, and sort of a bunch of other functions can be written using another single symbol,
01:07:15,120 --> 01:07:24,480 the multiply sign. So if we take addition, the way traditionally in Python, the add bytecode
01:07:24,480 --> 01:07:34,740 was executed is pointers, pointers, and more pointers. So first we have two objects. An
01:07:34,740 --> 01:07:39,160 object is basically a pointer to a bunch of memory that contains more pointers.
01:07:39,160 --> 01:07:41,360 Pointers all the way down.
01:07:41,360 --> 01:07:48,920 Well not quite, but there are a lot of them. So to simplify a bit, we look up in one of
01:07:48,920 --> 01:07:59,000 the objects, what is the type of that object? And does that object type define an add operation?
01:07:59,000 --> 01:08:05,760 And so you can imagine that there is a sort of a type integer that knows how to add itself
01:08:05,760 --> 01:08:11,880 to another integer. And there is a type floating point number that knows how to add itself
01:08:11,880 --> 01:08:18,840 to another floating point number. And the integers and floating point numbers are sort
01:08:18,840 --> 01:08:27,000 of important, I think, mostly historically because in the first computers, you used the
01:08:27,000 --> 01:08:32,440 sort of, the same bit pattern when interpreted as a floating point number had a very different
01:08:32,440 --> 01:08:34,600 value than when interpreted as an integer.
01:08:34,600 --> 01:08:36,720 Can I ask a dumb question here?
01:08:36,720 --> 01:08:37,720 Please do.
01:08:37,720 --> 01:08:42,520 If you take the basics of int and float and add, who carries the knowledge of how to add
01:08:42,520 --> 01:08:47,920 two integers? Is it the integer? It's the type integer versus...
01:08:47,920 --> 01:08:50,240 It's the type integer and the type float.
01:08:50,240 --> 01:08:57,000 What about the operator? Does the operator just exist as a platonic form possessed by
01:08:57,000 --> 01:08:59,720 the integer?
01:08:59,720 --> 01:09:10,120 The operator is more like, it's an index in a list of functions that the integer type
01:09:10,120 --> 01:09:19,120 defines. And so the integer type is really a collection of functions and there is an
01:09:19,120 --> 01:09:24,240 add function and there's a multiply function and there are like 30 other functions for
01:09:24,240 --> 01:09:32,920 other operations. There's a power function, for example. And you can imagine that in memory
01:09:32,920 --> 01:09:40,160 there is a distinct slot for the add operations. Let's say the add operation is the first operation
01:09:40,160 --> 01:09:46,000 of a type and the multiply is the second operation of a type. So now we take the integer type
01:09:46,000 --> 01:09:54,040 and we take the floating point type. In both cases, the add operation is the first slot
01:09:54,040 --> 01:10:01,040 and multiply is the second slot. But each slot contains a function and the functions
01:10:01,040 --> 01:10:09,960 are different because the add to integers function interprets the bit patterns as integers.
01:10:09,960 --> 01:10:19,360 The add to float function interprets the same bit pattern as a floating point number. And
01:10:19,360 --> 01:10:29,200 then there is the string data type, which again interprets the bit pattern as the address
01:10:29,200 --> 01:10:35,680 of a sequence of characters. There are lots of lies in that story, but that's sort of
01:10:35,680 --> 01:10:37,080 a basic idea.
01:10:37,080 --> 01:10:42,440 I can tell. I can tell the fake news and the fabrication going on here at the table. But
01:10:42,440 --> 01:10:46,720 where's the optimization? Is it on the operators? Is it different inside the integer?
01:10:46,720 --> 01:10:58,720 So the optimization is the observation that in a particular line of code, so now you write
01:10:58,720 --> 01:11:04,560 your little Python program and you write a function and that function sort of takes a
01:11:04,560 --> 01:11:11,200 bunch of inputs and at some point it adds two of the inputs together. Now I bet you
01:11:11,200 --> 01:11:18,840 even if you call your function a thousand times, that all those calls are likely all
01:11:18,840 --> 01:11:26,840 going to be about integers because maybe your program is all about integers. Or maybe on
01:11:26,840 --> 01:11:33,400 that particular line of code where there's that plus operator, every time the program
01:11:33,400 --> 01:11:41,240 hits that line, the variables A and B that are being added together happen to be strings.
01:11:41,240 --> 01:11:47,480 And so what we do is instead of having this single byte code that says, here's an add
01:11:47,480 --> 01:11:52,720 operation and the implementation of add is fully generic. It looks at the object from
01:11:52,720 --> 01:11:57,960 the object, it looks at the type, then it takes the type and it looks up the function
01:11:57,960 --> 01:12:04,120 pointer, then it calls the function. Now the function has to look at the other argument
01:12:04,120 --> 01:12:08,720 and it has to double check that the other argument has the right type. And then there's
01:12:08,720 --> 01:12:15,040 a bunch of error checking before it can actually just go ahead and add the two bit patterns
01:12:15,040 --> 01:12:26,280 in the right way. What we do is every time we execute an add instruction like that, we
01:12:26,280 --> 01:12:37,040 keep a little note of, in the end, after we hit the code that did the addition for a particular
01:12:37,040 --> 01:12:46,320 type, what type was it? And then after a few times through that code, if it's the same
01:12:46,320 --> 01:12:56,360 type all the time, we say, oh, so this add operation, even though it's the generic add
01:12:56,360 --> 01:13:04,440 operation, it might as well be the add integer operation. And add integer operation is much
01:13:04,440 --> 01:13:11,160 more efficient because it just says, assume that A and B are integers, do the addition
01:13:11,160 --> 01:13:20,520 operation, do it right there in line and produce the result. And the big lie here is that in
01:13:20,520 --> 01:13:26,880 Python, even if you have great evidence that in the past it was always two integers that
01:13:26,880 --> 01:13:31,920 you were adding, at some point in the future, that same line of code could still be hit
01:13:31,920 --> 01:13:35,480 with two floating points or two strings, or maybe a string and an integer.
01:13:35,480 --> 01:13:41,800 It's not a great lie, that's just the fact of life. I didn't account for what should
01:13:41,800 --> 01:13:46,080 happen in that case in the way I told the story.
01:13:46,080 --> 01:13:53,000 There is some accounting for that. And so what we actually have to do is when we have
01:13:53,000 --> 01:14:01,680 the add integer operation, we still have to check, are the two arguments in fact integers?
01:14:01,680 --> 01:14:08,920 We applied some tricks to make those checks efficient. And we know statistically that
01:14:08,920 --> 01:14:17,040 the outcome is almost always, yes, they are both integers. And so we quickly make that
01:14:17,040 --> 01:14:22,120 check and then we proceed with the sort of add integer operation. And then there is a
01:14:22,120 --> 01:14:28,320 fallback mechanism where we say, oops, one of them wasn't an integer. Now we're going
01:14:28,320 --> 01:14:35,280 to pretend that it was just the fully generic add operation. We wasted a few cycles believing
01:14:35,280 --> 01:14:42,240 it was going to be two integers and then we had to back up. But we didn't waste that much
01:14:42,240 --> 01:14:47,600 time and statistically most of the time.
01:14:47,600 --> 01:14:54,240 Basically we're sort of hoping that most of the time we guess right, because if it turns
01:14:54,240 --> 01:15:01,920 out that we guessed wrong too often, or we didn't have a good guess at all, things might
01:15:01,920 --> 01:15:09,520 actually end up running a little slower. So someone armed with this knowledge and a copy
01:15:09,520 --> 01:15:14,800 of the implementation, someone could easily construct a counterexample where they say,
01:15:14,800 --> 01:15:20,240 oh, I have a program and now it runs five times as slow in Python 3.11 than it did in
01:15:20,240 --> 01:15:22,500 Python 3.10.
01:15:22,500 --> 01:15:29,400 But that's a very unrealistic program. That's just like an extreme fluke.
01:15:29,400 --> 01:15:31,680 It's a fun reverse engineering task though.
01:15:31,680 --> 01:15:32,680 Oh yeah.
01:15:32,680 --> 01:15:43,000 So there's people like fun, yes. So there's some presumably heuristic of what defines
01:15:43,000 --> 01:15:49,960 a momentum of saying, you know, you seem to be working, adding two integers, not two generic
01:15:49,960 --> 01:15:54,240 types. So how do you figure out that heuristic?
01:15:54,240 --> 01:15:59,400 I think that the heuristic is actually, we assume that the weather tomorrow is going
01:15:59,400 --> 01:16:01,000 to be the same as the weather today.
01:16:01,000 --> 01:16:04,040 So you don't need two days of the weather?
01:16:04,040 --> 01:16:10,600 No. That is already so much better than guessing randomly.
01:16:10,600 --> 01:16:21,120 So how do you find this idea? Hey, I wonder if instead of adding two generic types, we
01:16:21,120 --> 01:16:24,960 start assuming that the weather tomorrow is the same as the weather today. Where do you
01:16:24,960 --> 01:16:32,000 find the idea for that? Because that ultimately, for you to do that, you have to kind of understand
01:16:32,000 --> 01:16:35,040 how people are using the language, right?
01:16:35,040 --> 01:16:40,200 Python is not the first language to do a thing like this. This is a fairly well-known trick,
01:16:40,200 --> 01:16:47,840 especially from other interpreted languages that had reason to be sped up. We occasionally
01:16:47,840 --> 01:16:58,160 look at papers about HHVM, which is Facebook's efficient compiler for PHP. There are tricks
01:16:58,160 --> 01:17:04,200 known from the JVM, and sometimes it just comes from academia.
01:17:04,200 --> 01:17:11,520 So the trick here is that the variable doesn't know what type it is. So this is not a statically
01:17:11,520 --> 01:17:18,080 typed language where you can afford to have a shortcut to saying it's ints.
01:17:18,080 --> 01:17:25,080 This is a trick that is especially important for interpreted languages with dynamic typing,
01:17:25,080 --> 01:17:33,400 because if the compiler could read in the source these x and y that we're adding are
01:17:33,400 --> 01:17:40,900 integers, the compiler can just insert a single add machine code that hardware machine instruction
01:17:40,900 --> 01:17:51,120 that exists on every CPU and ditto for floats. But because in Python, you don't generally
01:17:51,120 --> 01:17:57,200 declare the types of your variables, you don't even declare the existence of your variables.
01:17:57,200 --> 01:18:04,280 They just spring into existence when you first assign them, which is really cool and helps
01:18:04,280 --> 01:18:09,720 those beginners, because there is less bookkeeping they have to learn how to do before they can
01:18:09,720 --> 01:18:17,720 start playing around with code. But it makes the interpretation of the code less efficient,
01:18:17,720 --> 01:18:28,640 and so we're sort of trying to make the interpretation more efficient without losing the super dynamic
01:18:28,640 --> 01:18:32,160 nature of the language. That's always the challenge.
01:18:32,160 --> 01:18:42,000 3.5 got the PEP484 type hints. What is type hinting and is it used by the interpreter,
01:18:42,000 --> 01:18:44,640 the hints, or is it just syntactic sugar?
01:18:44,640 --> 01:18:52,000 So the type hints is an optional mechanism that people can use, and it's especially popular
01:18:52,000 --> 01:18:58,760 with sort of larger companies that have very large code bases written in Python.
01:18:58,760 --> 01:19:02,560 Do you think of it as almost like documentation saying these two variables are this type?
01:19:02,560 --> 01:19:11,320 It's more than documentation. I mean, so it is a sub-language of Python where you can
01:19:11,320 --> 01:19:16,600 express the types of variables. So here's a variable and it's an integer, and here's
01:19:16,600 --> 01:19:21,800 an argument to this function and it's a string, and here is a function that returns a list
01:19:21,800 --> 01:19:22,800 of strings.
01:19:22,800 --> 01:19:25,200 But that's not checked when you run the code.
01:19:25,200 --> 01:19:31,720 Exactly. There is a separate piece of software called a static type checker that reads all
01:19:31,720 --> 01:19:39,960 your source code without executing it and thinks long and hard about what it looks from
01:19:39,960 --> 01:19:47,960 just reading the code that code might be doing and double checks if that makes sense if you
01:19:47,960 --> 01:19:51,760 take the types as annotated into account.
01:19:51,760 --> 01:19:54,320 So this is something you're supposed to run as you develop.
01:19:54,320 --> 01:20:00,200 It's like a linter, yeah. That's definitely a development tool, but the type annotations
01:20:00,200 --> 01:20:09,560 currently are not used for speeding up the interpreter, and there are a number of reasons.
01:20:09,560 --> 01:20:18,320 Many people don't use them. Even when they do use them, they sometimes contain lies where
01:20:18,320 --> 01:20:25,360 the static type checker says everything's fine. I cannot prove that this integer is
01:20:25,360 --> 01:20:33,000 ever not an integer, but at runtime somehow someone manages to violate that assumption
01:20:33,000 --> 01:20:40,120 and the interpreter ends up doing just fine. If we started enforcing type annotations in
01:20:40,120 --> 01:20:47,160 Python, many Python programs would no longer work, and some Python programs wouldn't even
01:20:47,160 --> 01:20:50,320 be possible because they're too dynamic.
01:20:50,320 --> 01:20:57,480 So we made a choice of not using the annotations. There is a possible future where eventually
01:20:57,480 --> 01:21:06,200 three, four, five releases in the future, we could start using those annotations to
01:21:06,200 --> 01:21:15,760 sort of provide hints because we can still say, well, the source code leads us to believe
01:21:15,760 --> 01:21:23,400 that these X and Y are both integers, and so we can generate an add integer instruction,
01:21:23,400 --> 01:21:32,080 but we can still have a fallback that says, oh, if somehow the code at runtime provided
01:21:32,080 --> 01:21:38,400 something else, maybe it provided two decimal numbers, we can still use that generic add
01:21:38,400 --> 01:21:41,880 operation as a fallback, but we're not there.
01:21:41,880 --> 01:21:47,080 Is there currently a mechanism, or do you see something like that where you can almost
01:21:47,080 --> 01:21:55,440 add an assert inside a function that says, please check that my type hints are actually
01:21:55,440 --> 01:22:00,720 mapping to reality, sort of like insert manual static typing?
01:22:00,720 --> 01:22:05,600 There are third-party libraries that are in that business.
01:22:05,600 --> 01:22:09,720 It's possible to do that kind of thing? It's possible for a third-party library to take
01:22:09,720 --> 01:22:12,680 a hint and enforce it?
01:22:12,680 --> 01:22:20,240 Well, what we actually do is, and I think this is a fairly unique feature in Python,
01:22:20,240 --> 01:22:25,000 the type hints can be introspected at runtime.
01:22:25,000 --> 01:22:32,380 So while the program is running, I mean, Python is a very introspectable language.
01:22:32,380 --> 01:22:37,800 You can look at a variable and ask yourself, what is the type of this variable?
01:22:37,800 --> 01:22:44,260 And if that variable happens to refer to a function, you can ask, what are the arguments
01:22:44,260 --> 01:22:46,240 to the function?
01:22:46,240 --> 01:22:51,000 And nowadays, you can also ask, what are the type annotations for the function?
01:22:51,000 --> 01:22:55,920 So the type annotations are there inside the variable as it's at runtime?
01:22:55,920 --> 01:23:00,680 They're mostly associated with the function object, not with each individual variable,
01:23:00,680 --> 01:23:06,100 but you can sort of map from the arguments to the variables.
01:23:06,100 --> 01:23:08,360 And that's what a third-party library can help with.
01:23:08,360 --> 01:23:09,360 Exactly.
01:23:09,360 --> 01:23:15,260 And the problem with that is that all that extra runtime type checking is going to slow
01:23:15,260 --> 01:23:17,680 your code down instead of speed it up.
01:23:17,680 --> 01:23:25,880 I think to reference this sales pitchy blog post that says 75% of developers' time is
01:23:25,880 --> 01:23:30,020 spent on debugging, I would say that in some cases that might be okay.
01:23:30,020 --> 01:23:36,480 It might be okay to pay the cost of performance for the catching of the types, the type errors.
01:23:36,480 --> 01:23:46,200 And in most cases, doing it statically before you ship your code to production is more efficient
01:23:46,200 --> 01:23:48,920 than doing it at runtime piecemeal.
01:23:48,920 --> 01:23:49,920 Yeah.
01:23:49,920 --> 01:23:56,720 Can you tell me about mypy project?
01:23:56,720 --> 01:23:57,880 What is it?
01:23:57,880 --> 01:23:58,880 What's the mission?
01:23:58,880 --> 01:24:03,600 And in general, what is the future of static typing in Python?
01:24:03,600 --> 01:24:11,800 Well, so mypy was started by a Finnish developer, Jukka Latusalo.
01:24:11,800 --> 01:24:14,600 So many cool things out of Finland, I gotta say.
01:24:14,600 --> 01:24:15,680 Just that part of the world.
01:24:15,680 --> 01:24:20,760 I guess people have nothing better to do in those long, cold winters.
01:24:20,760 --> 01:24:21,760 I don't know.
01:24:21,760 --> 01:24:25,600 I think Jukka lived in England when he invented that stuff, actually.
01:24:25,600 --> 01:24:33,880 Mypy is the original static type checker for Python, and the type annotations that were
01:24:33,880 --> 01:24:43,760 introduced with PEP 484 were sort of developed together with the static type checker.
01:24:43,760 --> 01:24:51,120 And in fact, Jukka had first invented a different syntax that wasn't quite compatible with Python.
01:24:51,120 --> 01:24:59,760 And Jukka and I sort of met at a Python conference in, I think, in 2013.
01:24:59,760 --> 01:25:10,400 And we sort of came up with a compromise syntax that would not require any changes to Python,
01:25:10,400 --> 01:25:16,080 and that would let mypy sort of be an add-on static type checker for Python.
01:25:16,080 --> 01:25:18,480 Just out of curiosity, was it like double colon or something?
01:25:18,480 --> 01:25:21,360 What was he proposing that would break Python?
01:25:21,360 --> 01:25:28,480 I think he was using angular brackets for types like in C++ or Java generics.
01:25:28,480 --> 01:25:31,600 Yeah, you can't use angular brackets in Python.
01:25:31,600 --> 01:25:35,520 That would be too tricky for template type stuff.
01:25:35,520 --> 01:25:41,960 Well, the key thing is that we already had a syntax for annotations.
01:25:41,960 --> 01:25:45,480 We just didn't know what to use them for yet.
01:25:45,480 --> 01:25:52,400 So type annotations were just the sort of most logical thing to use that existing dummy
01:25:52,400 --> 01:25:54,000 syntax for.
01:25:54,000 --> 01:26:04,840 But there was no syntax for defining generics directly syntactically in the language.
01:26:04,840 --> 01:26:11,160 mypy literally meant my version of Python, where my refers to Jukka.
01:26:11,160 --> 01:26:22,720 He had a parser that translated mypy into Python by doing the type checks and then removing
01:26:22,720 --> 01:26:30,440 the annotations and all the angular brackets from the positions where he was using them.
01:26:30,440 --> 01:26:37,160 But a preprocessor model doesn't work very well with the typical workflow of Python development
01:26:37,160 --> 01:26:38,160 projects.
01:26:38,160 --> 01:26:39,160 That's funny.
01:26:39,160 --> 01:26:43,720 That could have been another major split if it became successful.
01:26:43,720 --> 01:26:51,400 If you watch TypeScript versus JavaScript, it's like a split in the community over types.
01:26:51,400 --> 01:26:53,360 That seems to be stabilizing now.
01:26:53,360 --> 01:26:55,080 It's not necessarily a split.
01:26:55,080 --> 01:27:02,480 There are certainly plenty of people who don't use TypeScript, but just use the original
01:27:02,480 --> 01:27:08,200 JavaScript notation, just like there are many people in the Python world who don't use type
01:27:08,200 --> 01:27:10,760 annotations and don't use static type checkers.
01:27:10,760 --> 01:27:11,760 No, I know.
01:27:11,760 --> 01:27:17,240 But there is a bit of a split between TypeScript and old school JavaScript, EES, whatever.
01:27:17,240 --> 01:27:23,880 Well, in the JavaScript world, transpilers are sort of the standard way of working anyway,
01:27:23,880 --> 01:27:29,120 which is why TypeScript being a transpiler itself is not a big deal.
01:27:29,120 --> 01:27:33,560 And transpilers, for people who don't know, it's exactly the thing you said with mypy.
01:27:33,560 --> 01:27:38,320 It's the code, I guess you call it pre-processing code that translates from one language to
01:27:38,320 --> 01:27:39,320 the other.
01:27:39,320 --> 01:27:43,400 And that's part of the culture, part of the workflow of the JavaScript community.
01:27:43,400 --> 01:27:44,560 So that's right.
01:27:44,560 --> 01:27:50,640 At the same time, an interesting development in the JavaScript slash TypeScript world at
01:27:50,640 --> 01:27:55,720 the moment is that there is a proposal under consideration.
01:27:55,720 --> 01:28:02,360 It's only a stage one proposal that proposes to add a feature to JavaScript where, just
01:28:02,360 --> 01:28:12,600 like Python, it will ignore certain syntax when running the JavaScript code.
01:28:12,600 --> 01:28:21,360 And what it ignores is more or less a superset of the TypeScript annotation syntax.
01:28:21,360 --> 01:28:29,760 So that would mean that eventually, if you wanted to, you could take TypeScript and you
01:28:29,760 --> 01:28:36,180 could shove it directly into a JavaScript interpreter without transpilation.
01:28:36,180 --> 01:28:41,480 The interesting thing in the JavaScript world, at least the web browser world, the web browsers
01:28:41,480 --> 01:28:49,800 have changed how they deploy and they sort of update their JavaScript engines much more
01:28:49,800 --> 01:28:53,440 quickly than they used to in the early days.
01:28:53,440 --> 01:29:00,840 And so there's much less of a need for transpilation in JavaScript itself, because most browsers
01:29:00,840 --> 01:29:05,760 just support the most recent version of ECMAScript.
01:29:05,760 --> 01:29:11,320 Just on a tangent of a tangent, do you see, if you were to recommend somebody use a thing,
01:29:11,320 --> 01:29:14,280 would you recommend TypeScript or JavaScript?
01:29:14,280 --> 01:29:19,640 I would recommend TypeScript, just because of the strictness of the typing.
01:29:19,640 --> 01:29:28,040 It's an enormously helpful extra tool that helps you sort of keep your head straight
01:29:28,040 --> 01:29:32,120 about what your code is actually doing.
01:29:32,120 --> 01:29:36,640 I mean, it helps with editing your code.
01:29:36,640 --> 01:29:43,080 It helps with ensuring that your code is not too incorrect.
01:29:43,080 --> 01:29:50,800 And it's actually quite compatible with JavaScript, nevermind this syntactic sort of hack that
01:29:50,800 --> 01:29:53,240 is still years in the future.
01:29:53,240 --> 01:30:00,360 But any library that is written in pure JavaScript can still be used from TypeScript programs.
01:30:00,360 --> 01:30:06,080 And also the other way around, you can write a library in TypeScript and then export it
01:30:06,080 --> 01:30:10,780 in a form that is totally consumable by JavaScript.
01:30:10,780 --> 01:30:17,080 That sort of compatibility is sort of the key to the success of TypeScript.
01:30:17,080 --> 01:30:21,680 Yeah, just to look at it, it's almost like a biological system that's evolving.
01:30:21,680 --> 01:30:24,720 It's fascinating to see JavaScript evolve the way it does.
01:30:24,720 --> 01:30:30,120 Well maybe we should consider that biological systems are just engineering systems too,
01:30:30,120 --> 01:30:31,120 right?
01:30:31,120 --> 01:30:32,120 Yes.
01:30:32,120 --> 01:30:35,680 Just very advanced, with more history.
01:30:35,680 --> 01:30:41,360 But it's almost like the most visceral in the JavaScript world because there's just
01:30:41,360 --> 01:30:47,320 so much code written in JavaScript that for its history was messy.
01:30:47,320 --> 01:30:54,040 If you talk about bugs per line of code, I just feel like JavaScript eats the cake, whatever
01:30:54,040 --> 01:30:55,480 the terminology is.
01:30:55,480 --> 01:31:01,920 It beats Python by a lot in terms of the number of bugs, meaning like way more bugs in JavaScript.
01:31:01,920 --> 01:31:08,480 And then obviously the browsers, there's so much active development, it feels a lot more
01:31:08,480 --> 01:31:15,080 like evolution where a bunch of stuff is born and dies and there's experimentation and debates
01:31:15,080 --> 01:31:22,440 versus Python is more, all that stuff is happening, but there's just a longer history of stable
01:31:22,440 --> 01:31:28,480 working giant software systems written in Python versus JavaScript is just a giant,
01:31:28,480 --> 01:31:31,560 beautiful, I would say, mess of code.
01:31:31,560 --> 01:31:37,520 It's a very different culture and to some extent differences in culture are random,
01:31:37,520 --> 01:31:44,920 but to some extent the differences have to do with the environment and the fact that
01:31:44,920 --> 01:31:56,120 JavaScript is primarily the language for developing web applications, especially the client side,
01:31:56,120 --> 01:32:02,720 and the fact that it's basically the only language for developing web applications makes
01:32:02,720 --> 01:32:10,960 that community sort of just have a different nature than the community of other languages.
01:32:10,960 --> 01:32:18,000 Plus the graphical component and the fact that they're deploying it on all kinds of
01:32:18,000 --> 01:32:22,320 shapes of screens and devices and all that kind of stuff, it just creates a beautiful
01:32:22,320 --> 01:32:23,320 chaos.
01:32:23,320 --> 01:32:24,800 Anyway, back to MyPy.
01:32:24,800 --> 01:32:30,480 So what, okay, you met, you talked about a syntax that could work, where does it currently
01:32:30,480 --> 01:32:31,480 stand?
01:32:31,480 --> 01:32:35,160 What's the future of static typing in Python?
01:32:35,160 --> 01:32:42,120 It is still controversial, but it is much more accepted than when MyPy and PEP484 were
01:32:42,120 --> 01:32:43,120 young.
01:32:43,120 --> 01:32:48,960 What's the connection between PEP484 type hints and MyPy?
01:32:48,960 --> 01:32:58,600 MyPy was the original static type checker, so MyPy quickly evolved from Yuka's own variant
01:32:58,600 --> 01:33:09,400 of Python to a static type checker for Python, and sort of PEP484, that was a very productive
01:33:09,400 --> 01:33:17,080 year where many hundreds of messages were exchanged debating the merits of every aspect
01:33:17,080 --> 01:33:19,540 of that PEP.
01:33:19,540 --> 01:33:23,560 And so MyPy is a static type checker for Python.
01:33:23,560 --> 01:33:27,760 It is itself written in Python.
01:33:27,760 --> 01:33:37,400 Most additional static typing features that we introduced in the time since 3.6 were also
01:33:37,400 --> 01:33:45,720 prototyped through MyPy, MyPy being an open source project with a very small number of
01:33:45,720 --> 01:33:47,120 maintainers.
01:33:47,120 --> 01:33:53,400 It was successful enough that people said this static type checking stuff for Python
01:33:53,400 --> 01:33:58,120 is actually worth an investment for our company.
01:33:58,120 --> 01:34:08,840 But somehow they chose not to support making MyPy faster, say, or adding new features to
01:34:08,840 --> 01:34:16,280 MyPy, but both Google and Facebook, and later Microsoft, developed their own static type
01:34:16,280 --> 01:34:17,280 checker.
01:34:17,280 --> 01:34:20,440 I think Facebook was one of the first.
01:34:20,440 --> 01:34:26,840 They decided that they wanted to use the same technology that they had successfully used
01:34:26,840 --> 01:34:36,640 for HHVM, because they sort of, they had a bunch of compiler writers and sort of static
01:34:36,640 --> 01:34:43,640 type checking experts who had written the HHVM compiler, and it was a big success within
01:34:43,640 --> 01:34:45,280 the company.
01:34:45,280 --> 01:34:49,080 And they had done it in a certain way, sort of.
01:34:49,080 --> 01:34:56,520 They wrote a big, highly parallel application in an obscure language named OCaml, which
01:34:56,520 --> 01:35:00,720 is apparently mostly very good for writing static type checkers.
01:35:00,720 --> 01:35:01,720 Interesting.
01:35:01,720 --> 01:35:07,080 I have a lot of questions about how to write a static type checker then.
01:35:07,080 --> 01:35:08,080 That's very confusing.
01:35:08,080 --> 01:35:14,160 Facebook wrote their version and they worked on it in secret for about a year, and then
01:35:14,160 --> 01:35:18,280 they came clean and went open source.
01:35:18,280 --> 01:35:25,920 Google in the meantime was developing something called PyType, which was mostly interesting
01:35:25,920 --> 01:35:32,240 because it, as you may have heard, they have one gigantic monorepo.
01:35:32,240 --> 01:35:36,240 So all the code is checked into a single repository.
01:35:36,240 --> 01:35:37,360 Facebook has a different approach.
01:35:37,360 --> 01:35:44,160 So Facebook developed Pyre, which was written in OCaml, which worked well with Facebook's
01:35:44,160 --> 01:35:48,000 development workflow.
01:35:48,000 --> 01:35:55,800 Google developed something they called PyType, which was actually itself written in Python.
01:35:55,800 --> 01:36:04,040 And it was meant to sort of fit well in their static type checking needs in Google's gigantic
01:36:04,040 --> 01:36:05,400 monorepo.
01:36:05,400 --> 01:36:07,960 So Google has one giant, got it.
01:36:07,960 --> 01:36:14,880 So just to clarify, this static type checker, philosophically, is a thing that's supposed
01:36:14,880 --> 01:36:17,440 to exist outside of the language itself.
01:36:17,440 --> 01:36:20,280 And it's just a workflow, like a debugger for the programmers.
01:36:20,280 --> 01:36:21,280 It's a linter.
01:36:21,280 --> 01:36:24,680 For people who don't know, a linter, maybe you can correct me.
01:36:24,680 --> 01:36:31,520 But it's a thing that runs through the code continuously, pre-processing to find issues
01:36:31,520 --> 01:36:33,520 based on style, documentation.
01:36:33,520 --> 01:36:36,240 I mean, there's all kinds of linters, right?
01:36:36,240 --> 01:36:38,080 You can check that.
01:36:38,080 --> 01:36:39,940 What usual things does a linter do?
01:36:39,940 --> 01:36:46,280 Maybe check that you haven't too many characters in a single line?
01:36:46,280 --> 01:36:52,880 Linters often do static analysis where they try to point out things that are likely mistakes,
01:36:52,880 --> 01:36:57,520 but not incorrect according to the language specification.
01:36:57,520 --> 01:37:01,760 Like maybe you have a variable that you never use.
01:37:01,760 --> 01:37:04,520 For the compiler, that is valid.
01:37:04,520 --> 01:37:11,520 You might be planning to use it in a future version of the code, and the compiler might
01:37:11,520 --> 01:37:12,880 just optimize it out.
01:37:12,880 --> 01:37:16,520 But the compiler is not going to tell you, hey, you're never using this variable.
01:37:16,520 --> 01:37:20,560 A linter will tell you that variable is not used.
01:37:20,560 --> 01:37:25,360 Maybe there's a typo somewhere else where you meant to use it, but you accidentally
01:37:25,360 --> 01:37:26,560 used something else.
01:37:26,560 --> 01:37:30,080 Or there are a number of sort of common scenarios.
01:37:30,080 --> 01:37:39,200 And a linter is often a big collection of little heuristics where by looking at the
01:37:39,200 --> 01:37:44,120 combination of how your code is laid out, maybe how it's indented, maybe the comment
01:37:44,120 --> 01:37:55,100 structure, but also just things like definition of names, use of names, it'll tell you likely
01:37:55,100 --> 01:37:56,680 things that are wrong.
01:37:56,680 --> 01:38:02,040 And in some cases, linters are really style checkers.
01:38:02,040 --> 01:38:09,800 For Python, there are a number of linters that check things like, do you use the PEP-8
01:38:09,800 --> 01:38:14,640 indented naming scheme for your functions and classes and variables?
01:38:14,640 --> 01:38:19,960 Because classes start with an uppercase, and the rest starts with a lowercase.
01:38:19,960 --> 01:38:21,560 There's differences there.
01:38:21,560 --> 01:38:29,120 And so the linter can tell you, hey, you have a class whose first letter is not an uppercase
01:38:29,120 --> 01:38:30,120 letter.
01:38:30,120 --> 01:38:31,720 And I just find it annoying.
01:38:31,720 --> 01:38:36,420 If I wanted that to be an uppercase letter, I would have typed an uppercase letter.
01:38:36,420 --> 01:38:42,420 But other people find it very comforting that if the linter is no longer complaining about
01:38:42,420 --> 01:38:46,320 their code, that they have followed all the style rules.
01:38:46,320 --> 01:38:50,280 Maybe it's a fast way for a new developer joining a team to learn the style rules, right?
01:38:50,280 --> 01:38:51,900 Yeah, there's definitely that.
01:38:51,900 --> 01:39:01,200 But the best use of a linter is probably not so much to sort of enforce team uniformity,
01:39:01,200 --> 01:39:08,560 but to actually help developers catch bugs that the compilers, for whatever reason, don't
01:39:08,560 --> 01:39:09,800 catch.
01:39:09,800 --> 01:39:13,320 And there's lots of that in Python.
01:39:13,320 --> 01:39:22,320 But a static type checker focuses on a particular aspect of the linting, which, I mean, MyPy
01:39:22,320 --> 01:39:27,760 doesn't care how you name your classes and variables.
01:39:27,760 --> 01:39:33,520 But it is meticulous about when you say that it was an integer here, and you're passing
01:39:33,520 --> 01:39:37,680 a string there, it will tell you, hey, that string is not an integer.
01:39:37,680 --> 01:39:39,720 So something's wrong.
01:39:39,720 --> 01:39:44,480 Either you were incorrect when you said it was an integer, or you're incorrect when you're
01:39:44,480 --> 01:39:45,920 passing it a string.
01:39:45,920 --> 01:39:49,640 If this is a race of static type checkers, is somebody winning?
01:39:49,640 --> 01:39:55,880 As you said, it's interesting that the companies didn't choose to invest in this centralized
01:39:55,880 --> 01:39:59,520 development of MyPy.
01:39:59,520 --> 01:40:02,080 Is there a future for MyPy?
01:40:02,080 --> 01:40:03,080 What do you see?
01:40:03,080 --> 01:40:11,080 Will one of the companies win out and everybody uses, like, PyType, whatever Google's is called?
01:40:11,080 --> 01:40:17,680 Well, Microsoft is hoping that Microsoft's horse in that race, called PyWrite, is going
01:40:17,680 --> 01:40:18,680 to win.
01:40:18,680 --> 01:40:20,680 PyWrite, like R-I-G-H-T?
01:40:20,680 --> 01:40:21,680 Correct.
01:40:21,680 --> 01:40:31,040 Yeah, all my word processors tend to typo-correct that as PyWrite, the name of the, I don't
01:40:31,040 --> 01:40:35,440 know what it is, some kind of semi-precious metal.
01:40:35,440 --> 01:40:37,600 Oh, right.
01:40:37,600 --> 01:40:38,600 I love it.
01:40:38,600 --> 01:40:39,600 Okay.
01:40:39,600 --> 01:40:43,880 So, okay, that's the Microsoft hope, but, okay, so let me ask the question a different
01:40:43,880 --> 01:40:44,880 way.
01:40:44,880 --> 01:40:49,280 Is there going to be ever a future where the static type checker gets integrated into the
01:40:49,280 --> 01:40:53,440 language?
01:40:53,440 --> 01:40:59,360 Nobody is currently excited about doing any work towards that.
01:40:59,360 --> 01:41:08,280 That doesn't mean that five or ten years from now the situation isn't different.
01:41:08,280 --> 01:41:19,260 At the moment, all the static type checkers still evolve at a much higher speed than Python
01:41:19,260 --> 01:41:23,040 and its annotation syntax evolve.
01:41:23,040 --> 01:41:26,800 You get a new release of Python once a year.
01:41:26,800 --> 01:41:32,360 Those are the only times that you can introduce new annotation syntax.
01:41:32,360 --> 01:41:40,200 And there are always people who invent new annotation syntax that they're trying to push.
01:41:40,200 --> 01:41:46,960 And worse, once we've all agreed that we are going to put some new syntax in, we can never
01:41:46,960 --> 01:41:54,040 take it back, at least a sort of deprecating an existing feature takes many releases because
01:41:54,040 --> 01:41:59,080 you have to assume that people started using it as soon as we announced it.
01:41:59,080 --> 01:42:01,640 And then you can't take it away from them right away.
01:42:01,640 --> 01:42:07,720 You have to start telling them, well, this will go away, but we're not going to tell
01:42:07,720 --> 01:42:09,320 you that it's an error yet.
01:42:09,320 --> 01:42:11,480 And then later it's going to be a warning.
01:42:11,480 --> 01:42:16,560 And then eventually three releases in the future, maybe we remove it.
01:42:16,560 --> 01:42:26,520 On the other hand, the typical static type checker still has a release like every month,
01:42:26,520 --> 01:42:30,880 every two months, certainly many times a year.
01:42:30,880 --> 01:42:39,320 Some type checkers also include a bunch of experimental ideas that aren't official standard
01:42:39,320 --> 01:42:42,480 Python syntax yet.
01:42:42,480 --> 01:42:50,800 The static type checkers also just get better at discovering things that sort of are unspecified
01:42:50,800 --> 01:42:53,880 by the language, but that sort of could make sense.
01:42:53,880 --> 01:42:59,440 And so each static type checker actually has its sort of strong and weak points.
01:42:59,440 --> 01:43:00,440 So it's cool.
01:43:00,440 --> 01:43:05,200 It's like a laboratory of experiments, Microsoft, Google and all, and you get to see.
01:43:05,200 --> 01:43:06,920 And you see that everywhere, right?
01:43:06,920 --> 01:43:12,000 Because there's not one single JavaScript engine either.
01:43:12,000 --> 01:43:17,440 There is one in Chrome, there is one in Safari, there is one in Firefox.
01:43:17,440 --> 01:43:19,200 But that said, you said there's not interest.
01:43:19,200 --> 01:43:24,320 I think there is a lot of interest in type hinting, right?
01:43:24,320 --> 01:43:25,320 In the PEP484.
01:43:25,320 --> 01:43:28,000 Actually, like how many people use that?
01:43:28,000 --> 01:43:32,920 Do you have a sense how many people use, because it's optional, it's a sugar.
01:43:32,920 --> 01:43:40,440 I can't put a number on it, but from the number of packages that do interesting things with
01:43:40,440 --> 01:43:49,320 it at runtime and the fact that there are like now three or four very mature type checkers
01:43:49,320 --> 01:43:52,480 that each have their segment of the market.
01:43:52,480 --> 01:43:57,320 And oh, and then there is PyCharm, which has a sort of more heuristic based type checker
01:43:57,320 --> 01:44:00,740 that also supports the same syntax.
01:44:00,740 --> 01:44:09,600 My assumption is that many, many people developing Python software professionally for some kind
01:44:09,600 --> 01:44:17,600 of production situation are using a static type checker, especially anybody who has a
01:44:17,600 --> 01:44:27,040 continuous integration cycle probably has one of the steps in their testing routine
01:44:27,040 --> 01:44:34,440 that happens for basically every commit is run a static type checker.
01:44:34,440 --> 01:44:39,800 And in most cases, that will be MyPy.
01:44:39,800 --> 01:44:43,240 So I think it's a pretty popular topic.
01:44:43,240 --> 01:44:50,760 According to this web page, 20 to 30% of Python 3 codebases are using type hints.
01:44:50,760 --> 01:44:51,760 Wow.
01:44:51,760 --> 01:44:53,280 I wonder how they measured that.
01:44:53,280 --> 01:44:55,360 Did they just scan all of GitHub?
01:44:55,360 --> 01:44:57,400 Yeah, that's what it looks like.
01:44:57,400 --> 01:45:03,180 They did a quick, not an all of, but like a random sampling.
01:45:03,180 --> 01:45:04,440 So you mentioned PyCharm.
01:45:04,440 --> 01:45:09,800 Let me ask you the big subjective question.
01:45:09,800 --> 01:45:17,640 What's the best IDE for Python, and you're extremely biased now that you're with Microsoft.
01:45:17,640 --> 01:45:21,400 Is it PyCharm, VS Code, Vim or Emacs?
01:45:21,400 --> 01:45:30,080 Historically, I actually started out with using Vim, but when it was still called VI.
01:45:30,080 --> 01:45:41,160 For a very long time, I think from the early 80s to, I'd say two years ago, I was Emacs
01:45:41,160 --> 01:45:42,160 user.
01:45:42,160 --> 01:45:43,160 Nice.
01:45:43,160 --> 01:45:56,160 Between, I'd say 2013 and 2018, I dabbled with PyCharm mostly because it had a couple
01:45:56,160 --> 01:46:07,520 of features, I mean, PyCharm is like driving an 18-wheeler truck, whereas Emacs is more
01:46:07,520 --> 01:46:15,680 like driving your comfortable Toyota car that you've had for a hundred thousand miles and
01:46:15,680 --> 01:46:19,320 you know what every little rattle of the car means.
01:46:19,320 --> 01:46:23,640 I was very comfortable in Emacs, but there were certain things it couldn't do.
01:46:23,640 --> 01:46:30,920 It wasn't very good at that sort of, at least the way I had configured it.
01:46:30,920 --> 01:46:37,800 I didn't have very good tooling in Emacs for finding a definition of a function.
01:46:37,800 --> 01:46:47,120 When I was at Dropbox exploring a 5 million line Python code base, just grabbing all that
01:46:47,120 --> 01:46:53,120 code for where is there a class foobar, well, it turns out that if you grab all 5 million
01:46:53,120 --> 01:46:58,160 lines of code, there are many classes with the same name.
01:46:58,160 --> 01:47:06,800 And so PyCharm sort of, once you fired it up and once it's indexed, your repository
01:47:06,800 --> 01:47:08,360 was very helpful.
01:47:08,360 --> 01:47:13,960 But as soon as I had to edit code, I would jump back to Emacs and do all my editing there
01:47:13,960 --> 01:47:20,640 because I could type much faster and switch between files when I knew which file I wanted
01:47:20,640 --> 01:47:21,640 much quicker.
01:47:21,640 --> 01:47:26,440 And I never really got used to the whole PyCharm user interface.
01:47:26,440 --> 01:47:32,000 Yeah, I feel torn in that same kind of way because I've used PyCharm off and on exactly
01:47:32,000 --> 01:47:33,760 in that same way.
01:47:33,760 --> 01:47:39,640 And I feel like I'm just being an old grumpy man for not learning how to quickly switch
01:47:39,640 --> 01:47:41,160 between files and all that kind of stuff.
01:47:41,160 --> 01:47:44,760 I feel like that has to do with shortcuts, that has to do with, I mean, you just have
01:47:44,760 --> 01:47:46,720 to get accustomed, just like with touch typing.
01:47:46,720 --> 01:47:49,120 Yeah, you have to just want to learn that.
01:47:49,120 --> 01:47:53,440 I mean, if you don't need it much, you don't need touch typing either.
01:47:53,440 --> 01:47:57,280 You can type with two fingers just fine in the short term, but in the long term, your
01:47:57,280 --> 01:48:02,760 life will become better psychologically and productivity-wise if you learn how to type
01:48:02,760 --> 01:48:04,280 with 10 fingers.
01:48:04,280 --> 01:48:09,480 If you do a lot of keyboard input before everyone, emails and stuff, right?
01:48:09,480 --> 01:48:14,840 Like you look at the next 20, 30 years of your life, you have to anticipate where technology
01:48:14,840 --> 01:48:17,560 is going.
01:48:17,560 --> 01:48:20,240 Do you want to invest in handwriting notes?
01:48:20,240 --> 01:48:21,240 Probably not.
01:48:21,240 --> 01:48:26,080 More and more people are doing typing versus handwriting notes, so you can anticipate that.
01:48:26,080 --> 01:48:28,880 So there's no reason to actually practice handwriting.
01:48:28,880 --> 01:48:31,720 There's more reason to practice typing.
01:48:31,720 --> 01:48:38,000 You can actually estimate, back to the spreadsheet, the number of paragraphs, sentences, or words
01:48:38,000 --> 01:48:41,400 you write for the rest of your life.
01:48:41,400 --> 01:48:42,400 You could probably estimate.
01:48:42,400 --> 01:48:46,320 You go again with the spreadsheet of my life, huh?
01:48:46,320 --> 01:48:47,320 Yes.
01:48:47,320 --> 01:48:51,840 All of that is not actual, like converted to a spreadsheet, but it's a gut feeling.
01:48:51,840 --> 01:48:54,520 Like I have the same kind of gut feeling about books.
01:48:54,520 --> 01:49:00,080 I've almost exclusively switched to Kindle now for ebook readers, even though I still
01:49:00,080 --> 01:49:06,280 love and probably always will the smell, the feel of a physical book.
01:49:06,280 --> 01:49:12,320 And the reason I switched to Kindle is like, all right, well, this is really paving, the
01:49:12,320 --> 01:49:18,920 future is going to be digital in terms of consuming books and content of that nature.
01:49:18,920 --> 01:49:23,880 So you should get, you know, you should let your brain get accustomed to that experience.
01:49:23,880 --> 01:49:27,200 And that same way, it feels like PyCharm or VS Code.
01:49:27,200 --> 01:49:35,180 I think PyCharm is the most sort of sophisticated, featureful Python ID.
01:49:35,180 --> 01:49:41,040 It feels like I should probably at some point very soon switch entire, like I'm not allowed
01:49:41,040 --> 01:49:45,240 to use anything else for Python than this ID or VS Code.
01:49:45,240 --> 01:49:46,240 It doesn't matter.
01:49:46,240 --> 01:49:50,080 But walk away from Emacs for this particular application, because I think I'm limiting
01:49:50,080 --> 01:49:56,000 myself in the same way that using two fingers for typing is limiting myself.
01:49:56,000 --> 01:49:57,160 This is a therapy session.
01:49:57,160 --> 01:50:00,280 This is, I'm not even asking a question.
01:50:00,280 --> 01:50:01,280 But I'm sure a lot of people are thinking this way, right?
01:50:01,280 --> 01:50:04,760 I'm not going to stop you.
01:50:04,760 --> 01:50:13,680 I think that everybody has to decide for themselves which one they want to invest more time in.
01:50:13,680 --> 01:50:22,320 I actually ended up giving VS Code a very tentative try when I started out at Microsoft
01:50:22,320 --> 01:50:25,040 and really liking it.
01:50:25,040 --> 01:50:32,000 And it took me a while before I realized why that was.
01:50:32,000 --> 01:50:38,880 I think that actually the founders of VS Code may not necessarily agree with me on this.
01:50:38,880 --> 01:50:49,360 But to me, VS Code is in a sense the spiritual successor of Emacs, because as you probably
01:50:49,360 --> 01:50:59,200 know as an old Emacs hack, the key part of Emacs is that it's mostly written in Lisp
01:50:59,200 --> 01:51:09,080 and that new features of Emacs usually update all the Lisp packages and add new Lisp packages.
01:51:09,080 --> 01:51:16,800 And oh yeah, there's also some very obscure thing improved in the part that's not in Lisp,
01:51:16,800 --> 01:51:21,560 but that's usually not why you would upgrade to a new version of Emacs.
01:51:21,560 --> 01:51:29,960 There's a core implementation that can read a file and it can put bits on the screen and
01:51:29,960 --> 01:51:33,960 it can manage memory and buffers.
01:51:33,960 --> 01:51:40,000 And then what makes it an editor full of features is all the Lisp packages.
01:51:40,000 --> 01:51:47,040 And of course the design of how the Lisp packages interact with each other and with that base
01:51:47,040 --> 01:51:55,240 layer of the core immutable engine, but almost everything in that core engine in Emacs case
01:51:55,240 --> 01:51:59,580 can still be overridden or replaced.
01:51:59,580 --> 01:52:12,280 And so VS Code has a similar architecture where there is like a base engine that you
01:52:12,280 --> 01:52:13,680 have no control over.
01:52:13,680 --> 01:52:22,240 I mean, it's open source, but nobody except the people who work on that part changes it
01:52:22,240 --> 01:52:24,600 much.
01:52:24,600 --> 01:52:32,840 And it has sort of a package manager and a whole series of interfaces for packages and
01:52:32,840 --> 01:52:38,760 an additional series of conventions for how packages should interact with the lower layers
01:52:38,760 --> 01:52:47,560 and with each other and powerful primitive operations that let you move the cursor around
01:52:47,560 --> 01:52:54,360 or select pieces of text or delete pieces of text or interact with the keyboard and
01:52:54,360 --> 01:52:58,880 the mouse and whatever peripherals you have.
01:52:58,880 --> 01:53:07,480 And so the sort of the extreme extensibility and the package ecosystem that you see in
01:53:07,480 --> 01:53:13,400 VS Code is a mirror of very similar architectural features in Emacs.
01:53:13,400 --> 01:53:20,660 Well, I'll have to give it a serious try because as far as sort of the hype and the excitement
01:53:20,660 --> 01:53:24,560 in the general programming community, VS Code seems to dominate.
01:53:24,560 --> 01:53:32,920 The interesting thing about PyCharm and what is it, PHPStorm, which are these JetBrains
01:53:32,920 --> 01:53:36,560 specific IDs that are designed for one programming language.
01:53:36,560 --> 01:53:41,280 That's interesting too, when an ID is specialized, right?
01:53:41,280 --> 01:53:47,880 They're usually actually just specializations of IntelliJ because underneath it's all the
01:53:47,880 --> 01:53:59,480 same editing engine with different veneer on top, where in VS Code, many things you
01:53:59,480 --> 01:54:09,240 do require loading third-party extensions, in PyCharm it is possible to have third-party
01:54:09,240 --> 01:54:14,320 extensions but it is a struggle to create one.
01:54:14,320 --> 01:54:15,320 Yes.
01:54:15,320 --> 01:54:17,960 And it's not part of the culture, all that kind of stuff.
01:54:17,960 --> 01:54:18,960 Yeah.
01:54:18,960 --> 01:54:24,800 I remember that, it might have been five years ago or so, we were trying to get some better
01:54:24,800 --> 01:54:33,160 MyPy integration into PyCharm because MyPy is sort of Python tooling and PyCharm had
01:54:33,160 --> 01:54:41,560 its own type checking heuristic thing that we wanted to replace with something based
01:54:41,560 --> 01:54:45,080 on MyPy because that was what we were using in the company.
01:54:45,080 --> 01:54:53,280 And for the guy who was writing that PyCharm extension, it was really a struggle to sort
01:54:53,280 --> 01:55:02,200 of find documentation and get the development workflow going and debug his code and all
01:55:02,200 --> 01:55:03,200 that.
01:55:03,200 --> 01:55:06,280 So that was not a pleasant experience.
01:55:06,280 --> 01:55:08,960 Let me talk to you about parallelism.
01:55:08,960 --> 01:55:15,480 In your post titled, Reasoning about AsyncIO Semaphore, you talk about a fast food restaurant
01:55:15,480 --> 01:55:17,280 in Silicon Valley that has only one table.
01:55:17,280 --> 01:55:18,280 Is this a real thing?
01:55:18,280 --> 01:55:20,200 I just wanted to ask you about that.
01:55:20,200 --> 01:55:24,640 Is that just like a metaphor you're using or is that an actual restaurant in Silicon
01:55:24,640 --> 01:55:25,640 Valley?
01:55:25,640 --> 01:55:27,600 It was a metaphor, of course.
01:55:27,600 --> 01:55:29,600 I can imagine such a restaurant.
01:55:29,600 --> 01:55:37,360 So for people who don't, then read the thing you should, but it was a idea of a restaurant
01:55:37,360 --> 01:55:43,440 where there's only one table and you show up one at a time and you're prepared.
01:55:43,440 --> 01:55:47,640 And I actually looked it up and there is restaurants like this throughout the world.
01:55:47,640 --> 01:55:50,640 And it just seems like a fascinating idea.
01:55:50,640 --> 01:55:56,640 You stand in line, you show up, there's one table, they ask you all kinds of questions,
01:55:56,640 --> 01:55:58,200 they cook just for you.
01:55:58,200 --> 01:55:59,200 That's fascinating.
01:55:59,200 --> 01:56:02,520 It sounds like you'd find places like that in Tokyo.
01:56:02,520 --> 01:56:06,720 It sounds like a very Japanese thing, or in the Bay Area, there are pop-up places that
01:56:06,720 --> 01:56:08,600 probably more or less work like that.
01:56:08,600 --> 01:56:10,600 I've never eaten at such a place.
01:56:10,600 --> 01:56:14,400 The fascinating thing is you propose it's a fast food, this is all for a burger.
01:56:14,400 --> 01:56:23,000 It was one of my rare sort of more literary or poetic moments where I thought I'll just
01:56:23,000 --> 01:56:27,160 open with a crazy example to catch your attention.
01:56:27,160 --> 01:56:33,980 And the rest is very dry stuff about locks and semaphores and how a semaphore is a generalization
01:56:33,980 --> 01:56:34,980 of a lock.
01:56:34,980 --> 01:56:37,320 Well, it was very poetic and well delivered.
01:56:37,320 --> 01:56:41,600 And it actually made me wonder if it's real or not, because you don't make that explicit.
01:56:41,600 --> 01:56:46,280 And it feels like it could be true, and in fact I wouldn't be surprised if somebody listens
01:56:46,280 --> 01:56:49,760 to this and knows exactly a restaurant like this in Silicon Valley.
01:56:49,760 --> 01:56:56,920 Anyway, can we step back and can you just talk about parallelism, concurrency, threading,
01:56:56,920 --> 01:56:59,780 asynchronous, all of these different terms.
01:56:59,780 --> 01:57:02,760 What is it, sort of a high philosophical level?
01:57:02,760 --> 01:57:04,560 The fisherman is back in the boat.
01:57:04,560 --> 01:57:12,440 Well, the idea is if the fisherman has two fishing rods, since fishing is mostly a matter
01:57:12,440 --> 01:57:17,120 of waiting for a fish to nibble, well, it depends on how you do it actually.
01:57:17,120 --> 01:57:22,080 But if you had two, if you're doing the style of fishing where you sort of, you throw it
01:57:22,080 --> 01:57:27,640 out and then you let it sit for a while until maybe you see a nibble.
01:57:27,640 --> 01:57:34,640 One fisherman can easily run two or three or four fishing rods, and so as long as you
01:57:34,640 --> 01:57:40,640 can afford the equipment, you can catch four times as many fish by a small investment in
01:57:40,640 --> 01:57:41,880 four fishing rods.
01:57:41,880 --> 01:57:48,000 And so, since your time, you sort of say you have all Saturday to go fishing, if you can
01:57:48,000 --> 01:57:53,020 catch four times as much fish, you have a much higher productivity.
01:57:53,020 --> 01:57:55,440 And that's actually, I think, how deep sea fishing is done.
01:57:55,440 --> 01:58:00,720 You could just have a rod and you put in a hole so you can have many rods.
01:58:00,720 --> 01:58:06,720 Is there an interesting difference between parallelism and concurrency and asynchronous?
01:58:06,720 --> 01:58:09,200 Is there one a subset of the other to you?
01:58:09,200 --> 01:58:10,980 Like, how do you think about these terms?
01:58:10,980 --> 01:58:15,080 In the computer world, there is a big difference.
01:58:15,080 --> 01:58:24,480 When people are talking about parallelism, like a parallel computer, that's usually really
01:58:24,480 --> 01:58:33,440 several complete CPUs that are sort of tied together and share something like memory or
01:58:33,440 --> 01:58:37,480 an IO bus.
01:58:37,480 --> 01:58:46,160 Concurrency can be a much more abstract concept where you have the illusion that things happen
01:58:46,160 --> 01:58:54,360 simultaneously, but what the computer actually does is it spends a little time running some
01:58:54,360 --> 01:58:58,880 this program for a while, and then it spends some time running that program for a while,
01:58:58,880 --> 01:59:02,320 and then spending some time for the third program for a while.
01:59:02,320 --> 01:59:08,640 So, parallelism is the reality and concurrency is part reality, part illusion.
01:59:08,640 --> 01:59:16,000 Yeah, parallelism typically implies that there is multiple copies of the hardware.
01:59:16,000 --> 01:59:20,920 You write that implementing synchronization primitives is hard in that blog post, and
01:59:20,920 --> 01:59:23,780 you talk about locks and semaphores.
01:59:23,780 --> 01:59:27,720 Why is it hard to implement synchronization primitives?
01:59:27,720 --> 01:59:36,300 Because at the conscious level, our brains are not trained to sort of keep track of multiple
01:59:36,300 --> 01:59:39,680 things at the same time.
01:59:39,680 --> 01:59:45,960 Like obviously, you can walk and chew gum at the same time, because they're both activities
01:59:45,960 --> 01:59:51,400 that require only a little bit of your conscious activity.
01:59:51,400 --> 01:59:58,160 But try balancing your checkbook and watching a murder mystery on TV.
01:59:58,160 --> 02:00:03,880 You'll mix up the digits or you'll miss an essential clue in the TV show.
02:00:03,880 --> 02:00:08,760 So why does it matter that the programmer, the human, is bad?
02:00:08,760 --> 02:00:15,080 Because the programmer is, at least with the current state of the art, is responsible for
02:00:15,080 --> 02:00:23,360 writing the code correctly, and it's hard enough to keep track of a recipe that you
02:00:23,360 --> 02:00:28,240 just execute one step at a time.
02:00:28,240 --> 02:00:34,000 Chop the carrots, then peel the potatoes, mix the icing.
02:00:34,000 --> 02:00:41,120 You need your whole brain, when you're reading a piece of code, what is going on?
02:00:41,120 --> 02:00:48,080 Okay, we're loading the number of mermaids in variable A and the number of mermen in
02:00:48,080 --> 02:00:52,080 variable B, and now we take the average or whatever.
02:00:52,080 --> 02:00:57,160 I like how we're just jumping from metaphor to metaphor, I like it.
02:00:57,160 --> 02:01:02,920 You have to keep in your head what is in A, what is in B, what is in C. Hopefully you
02:01:02,920 --> 02:01:05,520 have better names.
02:01:05,520 --> 02:01:08,200 And that is challenging enough.
02:01:08,200 --> 02:01:18,680 If you have two different pieces of code that are sort of being executed simultaneously,
02:01:18,680 --> 02:01:28,040 whether it's using the parallel or the concurrent approach, if A is the number of fishermen
02:01:28,040 --> 02:01:33,520 and B is the number of programmers, but in another part of the code, A is the number
02:01:33,520 --> 02:01:40,220 of mermaids and B is the number of mermen, and somehow that's the same variable.
02:01:40,220 --> 02:01:45,360 If you do it sequentially, if first you do your mermaid merpeople computation and then
02:01:45,360 --> 02:01:51,000 you do your people in the boat computation, it doesn't matter that the variables are called
02:01:51,000 --> 02:01:57,120 A and B and that is literally the same variable, because you're done with one use of that variable.
02:01:57,120 --> 02:02:03,680 But when you mix them together, suddenly the number of merpeople replaces the number of
02:02:03,680 --> 02:02:08,240 fishermen and your computation goes dramatically wrong.
02:02:08,240 --> 02:02:13,680 And there's all kinds of ordering of operations that could result in the assignment of those
02:02:13,680 --> 02:02:17,240 variables and so you have to anticipate all possible orderings.
02:02:17,240 --> 02:02:24,040 And you think you're smart and you'll put a lock around it, and in practice, in terms
02:02:24,040 --> 02:02:31,000 of bugs per thousand lines of code, this is an area where everything is worse.
02:02:31,000 --> 02:02:41,360 So a lock is a mechanism by which you forbid only one chef can access the oven at a time.
02:02:41,360 --> 02:02:42,520 Something like that.
02:02:42,520 --> 02:02:46,640 And then semaphores allow you to do what, multiple ovens?
02:02:46,640 --> 02:02:53,460 That's not a bad idea, because if you're baking cakes and you have multiple people
02:02:53,460 --> 02:02:59,000 all baking cakes but there's only one oven, then maybe you can tell that the oven is in
02:02:59,000 --> 02:03:02,080 use but maybe it's preheating.
02:03:02,080 --> 02:03:08,200 And so you have to, maybe you make a sign that says oven in use and you flip the sign
02:03:08,200 --> 02:03:13,560 over and it says oven is free when you're done baking your cake.
02:03:13,560 --> 02:03:19,360 That's a lock, that's sort of, and what do you do when you have two ovens or maybe you
02:03:19,360 --> 02:03:21,600 have ten ovens?
02:03:21,600 --> 02:03:26,360 You can put a separate sign on each oven or maybe you can sort of, someone who comes in
02:03:26,360 --> 02:03:32,920 wants to see at a glance and maybe there's an electronic sign that says there are still
02:03:32,920 --> 02:03:36,040 five ovens available.
02:03:36,040 --> 02:03:44,560 Or maybe there are already three people waiting for an oven so you can, if you see an oven
02:03:44,560 --> 02:03:49,560 that's not in use, it's already reserved for someone else who got in line first.
02:03:49,560 --> 02:03:53,240 That's sort of what the restaurant metaphor was trying to explain.
02:03:53,240 --> 02:04:00,400 Yeah, and so your now tasks, you're sitting as a designer of Python with a team of brilliant
02:04:00,400 --> 02:04:04,800 core developers and you have to try to figure out to what degree can any of these ideas
02:04:04,800 --> 02:04:06,960 be integrated and not.
02:04:06,960 --> 02:04:13,880 So maybe this is a good time to ask, what is AsyncIO and how has it evolved since Python
02:04:13,880 --> 02:04:14,880 3.4?
02:04:14,880 --> 02:04:25,200 Wow, yeah, so we had this really old library for doing things concurrently, especially
02:04:25,200 --> 02:04:34,880 things that had to do with I.O. and networking I.O. was especially sort of a popular topic.
02:04:34,880 --> 02:04:45,080 And in the Python standard library, we had a brief period where there was lots of development
02:04:45,080 --> 02:04:49,720 and I think it was late 90s, maybe early 2000s.
02:04:49,720 --> 02:04:56,160 And like two little modules were added that were the state of the art of doing asynchronous
02:04:56,160 --> 02:05:02,440 I.O. or sort of non-blocking I.O. which means that you can keep multiple network connections
02:05:02,440 --> 02:05:08,160 open and sort of service them all in parallel like a typical web server does.
02:05:08,160 --> 02:05:12,880 So I.O. is input and output, so you're writing either to the network or reading from a network
02:05:12,880 --> 02:05:16,880 connection or reading and writing to a hard drive, to storage.
02:05:16,880 --> 02:05:17,920 Also possible.
02:05:17,920 --> 02:05:25,800 And you can do the ideas you could do to multiple while also doing computation, running some
02:05:25,800 --> 02:05:28,120 code that does some fancy stuff.
02:05:28,120 --> 02:05:35,000 Yeah, like when you're writing a web server, when a request comes in, a user sort of needs
02:05:35,000 --> 02:05:38,200 to see a particular web page.
02:05:38,200 --> 02:05:43,040 You have to find that page maybe in the database and format it properly and send it back to
02:05:43,040 --> 02:05:44,040 the client.
02:05:44,040 --> 02:05:49,000 And there's a lot of waiting, waiting for the database, waiting for the network.
02:05:49,000 --> 02:05:55,400 And so you can handle hundreds or thousands or millions of requests concurrently on one
02:05:55,400 --> 02:05:56,400 machine.
02:05:56,400 --> 02:06:00,800 Ways of doing that in Python were kind of stagnated.
02:06:00,800 --> 02:06:11,000 And I forget, it might have been around 2012, 2014, when someone for the umpteenth time
02:06:11,000 --> 02:06:18,520 actually said these async chat and async core modules that you have in a standard library
02:06:18,520 --> 02:06:22,960 are not quite enough to solve my particular problem.
02:06:22,960 --> 02:06:25,840 Can we add one tiny little feature?
02:06:25,840 --> 02:06:31,320 And everybody said, no, that stuff is not, you're not supposed to use that stuff.
02:06:31,320 --> 02:06:34,240 Write your own using a third party library.
02:06:34,240 --> 02:06:39,280 And then everybody started a debate about what the right third party library was.
02:06:39,280 --> 02:06:49,320 And somehow I felt that there was actually a cue for, well, maybe we need a better state
02:06:49,320 --> 02:06:56,360 of the art module in the standard library for multiplexing input output from different
02:06:56,360 --> 02:06:57,680 sources.
02:06:57,680 --> 02:07:00,680 You could say that it spiraled out of control a little bit.
02:07:00,680 --> 02:07:07,200 It was at the time, it was the largest Python enhancement proposal that was ever proposed.
02:07:07,200 --> 02:07:09,160 And you were deeply involved with that.
02:07:09,160 --> 02:07:11,760 At the time I was very much involved with that.
02:07:11,760 --> 02:07:15,240 I was like the lead architect.
02:07:15,240 --> 02:07:23,160 I ended up talking to people who had already developed serious third party libraries that
02:07:23,160 --> 02:07:30,040 did similar things and sort of taking ideas from them and getting their feedback on my
02:07:30,040 --> 02:07:34,360 design and eventually we put it in the standard library.
02:07:34,360 --> 02:07:36,760 And after a few years I got distracted.
02:07:36,760 --> 02:07:42,120 I think the thing, the big thing that distracted me was actually type annotations.
02:07:42,120 --> 02:07:49,280 But other people kept it alive and kicking and it's been quite successful actually in
02:07:49,280 --> 02:07:51,440 the world of Python web clients.
02:07:51,440 --> 02:07:56,720 So initially, what are some of the design challenges there in that debate for the PEP?
02:07:56,720 --> 02:07:58,600 And what are some things that got rejected?
02:07:58,600 --> 02:08:02,200 What are some things that got accepted that stand out to you?
02:08:02,200 --> 02:08:07,160 There are a couple of different ways you can handle parallel IO.
02:08:07,160 --> 02:08:12,840 And this happens sort of at an architectural level in operating systems as well, like Windows
02:08:12,840 --> 02:08:17,640 prefers to do it one way and Unix prefers to do it the other way.
02:08:17,640 --> 02:08:24,480 You sort of, you have an object that represents a network endpoint, say a connection with
02:08:24,480 --> 02:08:32,120 a web browser that's your client and say you're waiting for an incoming request.
02:08:32,120 --> 02:08:38,880 Two fundamental approaches are, okay, I'm waiting for an incoming request, I'm doing
02:08:38,880 --> 02:08:45,240 something else, come wake me up or sort of come tell me when something interesting happened,
02:08:45,240 --> 02:08:48,960 like a packet came in on that network connection.
02:08:48,960 --> 02:08:58,000 And the other paradigm is we're on a team of a whole bunch of people with maybe a little
02:08:58,000 --> 02:09:02,240 mind and we can only manage one web connection at a time.
02:09:02,240 --> 02:09:13,400 So I'm just sitting looking at this web connection and I'm just blocked until something comes
02:09:13,400 --> 02:09:19,720 in and then I'm already waiting for it.
02:09:19,720 --> 02:09:25,200 I get the data, I process the data and then I go back to the top and say, no, sort of
02:09:25,200 --> 02:09:27,720 I'm waiting for the next packet.
02:09:27,720 --> 02:09:29,480 Those are about the two paradigms.
02:09:29,480 --> 02:09:36,520 One is a paradigm where there is sort of notionally a thread of control, whether it's an actual
02:09:36,520 --> 02:09:42,800 operating system thread or more an abstraction in async IO, we call them tasks.
02:09:42,800 --> 02:09:50,040 But a task in async IO or a thread in other contexts is devoted to one thing and it has
02:09:50,040 --> 02:09:52,880 logic for all the stages.
02:09:52,880 --> 02:09:59,240 Like when it's a web request, like first wait for the first line of the web request, parse
02:09:59,240 --> 02:10:06,960 it because then you know if it's a get or a post or a put or whatever, or an error.
02:10:06,960 --> 02:10:11,760 Then wait until you have a bunch of lines until there's a blank line, then parse that
02:10:11,760 --> 02:10:18,560 as headers and then interpret that and then wait for the rest of the data to come in if
02:10:18,560 --> 02:10:24,720 there is any more that you expect, that sort of standard web stuff.
02:10:24,720 --> 02:10:29,520 And the other thing is, and there's always endless debate about which approach is more
02:10:29,520 --> 02:10:35,440 efficient and which approach is more error prone, where I just have a whole bunch of
02:10:35,440 --> 02:10:43,280 stacks in front of me and whenever a packet comes in, I sort of look at the number of
02:10:43,280 --> 02:10:48,560 the packet, that there's some number on the packet and I say, oh, that packet goes on
02:10:48,560 --> 02:10:55,400 this pile and then I can do a little bit and then sort of that pile provides my context.
02:10:55,400 --> 02:11:01,800 And as soon as I'm done with the processing, I can forget everything about what's going
02:11:01,800 --> 02:11:06,680 on because the next packet will come in from some random other client and it's that pile
02:11:06,680 --> 02:11:09,280 or it's this pile.
02:11:09,280 --> 02:11:14,240 And every time a pile is maybe empty or full or whatever the criteria is, I can toss it
02:11:14,240 --> 02:11:16,760 away or use it for a new space.
02:11:16,760 --> 02:11:24,520 But several traditional third party libraries for asynchronous IO processing in Python chose
02:11:24,520 --> 02:11:27,360 the model of a callback.
02:11:27,360 --> 02:11:32,440 And that's the idea where you have a bunch of different stacks of paper in front of you
02:11:32,440 --> 02:11:37,600 and every time someone gives you a piece, gives you a new sheet, you decide which stack
02:11:37,600 --> 02:11:40,120 it belongs to.
02:11:40,120 --> 02:11:49,560 And that leads to a certain style of spaghetti code that I find sort of aesthetically not
02:11:49,560 --> 02:11:56,400 pleasing and I was sort of never very successful and I had heard many stories about people
02:11:56,400 --> 02:12:02,960 who were also sort of complaining about that style of coding.
02:12:02,960 --> 02:12:07,960 It was very prevalent in JavaScript at the time at least because it was like how the
02:12:07,960 --> 02:12:12,040 JavaScript event loop basically works.
02:12:12,040 --> 02:12:19,800 And so I thought, well, the task-based model where each task has a bunch of logic, we had
02:12:19,800 --> 02:12:26,080 mechanisms in the Python language that we could easily reuse for that.
02:12:26,080 --> 02:12:33,280 And I thought, I want to build a whole library for asynchronous networking IO and all the
02:12:33,280 --> 02:12:40,160 other things that may need to be done asynchronously based on that paradigm.
02:12:40,160 --> 02:12:45,960 And so I just chose a paradigm and tried to see how far I could get with that.
02:12:45,960 --> 02:12:48,520 And it turns out that it's a pretty good paradigm.
02:12:48,520 --> 02:12:56,480 So people enjoy that kind of paradigm programming for asynchronous IO relative to callbacks.
02:12:56,480 --> 02:12:58,760 Okay, beautiful.
02:12:58,760 --> 02:13:07,120 So how does that all interplay with the infamous GIL, the Global Interpreter Lock?
02:13:07,120 --> 02:13:13,360 Maybe can you say what the GIL is and how does it dance beautifully with async IO?
02:13:13,360 --> 02:13:20,080 The Global Interpreter Lock solves the problem that Python originally was not written with
02:13:20,080 --> 02:13:25,080 either asynchronous or parallelism in mind at all.
02:13:25,080 --> 02:13:30,640 There was no concurrency in the language, there was no parallelism, there were no threads.
02:13:30,640 --> 02:13:37,680 Only a small number of years into Python's initial development, all the new cool operating
02:13:37,680 --> 02:13:46,800 systems like SunOS and Silicon Graphics IRIX and then eventually POSIX and Windows all
02:13:46,800 --> 02:13:53,440 came with threading libraries that let you do multiple things in parallel.
02:13:53,440 --> 02:14:00,800 And there is a certain sort of principle which is the operating system handles the threads
02:14:00,800 --> 02:14:10,520 for you, and the program can pretend that there are as many CPUs as there are threads
02:14:10,520 --> 02:14:13,680 in the program.
02:14:13,680 --> 02:14:18,640 And those CPUs work completely independently.
02:14:18,640 --> 02:14:25,220 And if you don't have enough CPUs, the operating system sort of simulates those extra CPUs.
02:14:25,220 --> 02:14:32,560 On the other hand, if you have enough CPUs, you can get a lot of work done by deploying
02:14:32,560 --> 02:14:34,540 those multiple CPUs.
02:14:34,540 --> 02:14:41,120 But Python wasn't written to do that.
02:14:41,120 --> 02:14:52,180 And so as libraries for multi-threading were added to C, but every operating system vendor
02:14:52,180 --> 02:14:55,640 was adding their own version of that.
02:14:55,640 --> 02:15:00,820 We thought, and maybe we were wrong, but at the time we thought, well, we quickly want
02:15:00,820 --> 02:15:06,980 to be able to support these multiple threads, because they seemed at the time in the early
02:15:06,980 --> 02:15:14,040 90s, when they were new, at least to me, they seemed a cool, interesting programming paradigm.
02:15:14,040 --> 02:15:19,920 And one of the things that Python, at least at the time, felt was nice about the language
02:15:19,920 --> 02:15:29,020 was that we could give a safe version of all kinds of cool new operating system toys to
02:15:29,020 --> 02:15:30,380 the Python programmer.
02:15:30,380 --> 02:15:42,140 Like I remember one or two years before threading, I had spent some time adding networking sockets
02:15:42,140 --> 02:15:43,260 to Python.
02:15:43,260 --> 02:15:48,540 And they were very literal translation of the networking sockets that were in the BSD
02:15:48,540 --> 02:15:51,660 programming system, so Unix BSD.
02:15:51,660 --> 02:15:57,040 But the nice thing was if you were using sockets from Python, then all the things you can do
02:15:57,040 --> 02:16:02,300 wrong with sockets in C would automatically give you a clear error message instead of
02:16:02,300 --> 02:16:06,500 just ending up with a malfunctioning hanging program.
02:16:06,500 --> 02:16:10,420 And so we thought, well, we'll do the same thing with threading.
02:16:10,420 --> 02:16:21,780 But we didn't really want to rewrite the interpreter to be thread safe, because that would be a
02:16:21,780 --> 02:16:28,180 very complex refactoring of all the interpreter code and all the runtime code, because all
02:16:28,180 --> 02:16:32,500 the objects were written with the assumption that there's only one thread.
02:16:32,500 --> 02:16:38,260 And so we said, okay, well, we'll take our losses, we'll provide something that looks
02:16:38,260 --> 02:16:43,860 like threads, and as long as you only have a single CPU on your computer, which most
02:16:43,860 --> 02:16:48,940 computers at the time did, it feels just like threads.
02:16:48,940 --> 02:16:56,420 Because the whole idea of multiple threads in the OS was that even if your computer only
02:16:56,420 --> 02:17:00,820 had one CPU, you could still fire up as many threads as you wanted.
02:17:00,820 --> 02:17:06,460 Well, within reason, maybe 10 or 12, not 5,000.
02:17:06,460 --> 02:17:16,500 And so we thought we had conquered the abstraction of threads pretty well, because multi-core
02:17:16,500 --> 02:17:23,100 CPUs were not in most Python programmers' hands anyway.
02:17:23,100 --> 02:17:29,780 And then, of course, a couple of more iterations of Moore's law, and computers getting faster,
02:17:29,780 --> 02:17:38,500 and at some point, the chip designers decided that they couldn't make the CPUs faster, but
02:17:38,500 --> 02:17:43,980 they could still make them smaller, and so they could put multiple CPUs on one chip.
02:17:43,980 --> 02:17:51,260 And suddenly, there was all this pressure about do things in parallel, and that's where
02:17:51,260 --> 02:17:55,580 the solution we had in Python didn't work.
02:17:55,580 --> 02:18:01,940 And that's sort of the moment that the GIL became infamous.
02:18:01,940 --> 02:18:09,660 Because the GIL was the solution we used to sort of take this single interpreter and share
02:18:09,660 --> 02:18:15,100 it between all the different operating system threads that you could create.
02:18:15,100 --> 02:18:21,820 And so as long as the hardware physically only had one CPU, that was all fine.
02:18:21,820 --> 02:18:27,980 And then as hardware vendors were suddenly telling us all, oh, you got to parallelize,
02:18:27,980 --> 02:18:34,300 everything's got to be parallelized, people started saying, oh, but we can use multiple
02:18:34,300 --> 02:18:40,140 threads in Python, and then they discovered, oh, but actually all threads run on a single
02:18:40,140 --> 02:18:41,140 core.
02:18:41,140 --> 02:18:42,140 Yeah.
02:18:42,140 --> 02:18:48,380 I mean, is there a way, is there ideas in the future to remove the global interpreter
02:18:48,380 --> 02:18:56,780 log GIL, like maybe multiple subinterpreters, some tricky interpreters on top of interpreters
02:18:56,780 --> 02:18:57,780 kind of thing?
02:18:57,780 --> 02:19:04,180 Yeah, there are a couple of possible futures there.
02:19:04,180 --> 02:19:11,660 The most likely future is that we'll get multiple subinterpreters, which each run a completely
02:19:11,660 --> 02:19:14,340 independent Python program.
02:19:14,340 --> 02:19:15,380 Nice.
02:19:15,380 --> 02:19:25,180 But there's still some benefit of sort of faster communication between those programs.
02:19:25,180 --> 02:19:31,580 But it's also managing for you this running a multiple Python programs, so it's hidden
02:19:31,580 --> 02:19:33,740 from you, right?
02:19:33,740 --> 02:19:39,100 It's hidden from you, but you have to spend more time communicating between those programs
02:19:39,100 --> 02:19:46,340 because the sort of, the attractive thing about the multi-threaded model is that the
02:19:46,340 --> 02:19:48,980 threads can share objects.
02:19:48,980 --> 02:19:54,260 At the same time, that's also the downfall of the multi-threaded programming model because
02:19:54,260 --> 02:20:01,460 when you do share objects, you weren't, and you didn't necessarily intend to share them
02:20:01,460 --> 02:20:10,460 or there were aspects of those objects that were not reusable, you get all kinds of concurrency
02:20:10,460 --> 02:20:11,740 bugs.
02:20:11,740 --> 02:20:18,660 And so the reason I wrote that little blog post about semaphores was that concurrency
02:20:18,660 --> 02:20:21,540 bugs are just harder.
02:20:21,540 --> 02:20:30,420 It would be nice if Python had no global interpreter lock and it had so-called free threading,
02:20:30,420 --> 02:20:35,860 but it would also cause a lot more software bugs.
02:20:35,860 --> 02:20:41,940 The interesting thing is that there is still a possible future where we are actually going
02:20:41,940 --> 02:20:51,340 to or where we could experiment at least with that because there is a guy working for Facebook
02:20:51,340 --> 02:21:00,860 who has developed a fork of CPython that he called the no-gill interpreter where he removed
02:21:00,860 --> 02:21:08,220 the gill and made a whole bunch of optimizations so that the single-threaded case doesn't run
02:21:08,220 --> 02:21:18,180 too much slower and multi-threaded case will actually use all the cores that you have.
02:21:18,180 --> 02:21:28,360 And so that would be an interesting possibility if we would be willing as Python core developers
02:21:28,360 --> 02:21:35,460 to actually maintain that code indefinitely.
02:21:35,460 --> 02:21:42,320 And if we're willing to put up with the additional complexity of the interpreter and the additional
02:21:42,320 --> 02:21:45,380 sort of overhead for the single-threaded case.
02:21:45,380 --> 02:21:58,060 And I'm personally not convinced that there are enough people needing the speed of multiple
02:21:58,060 --> 02:22:06,900 threads with their Python programs that it's worth to sort of take that performance hit
02:22:06,900 --> 02:22:09,420 and that complexity hit.
02:22:09,420 --> 02:22:18,300 And I feel that the gill actually is a pretty nice Goldilocks point between no threads and
02:22:18,300 --> 02:22:20,180 all threads all the time.
02:22:20,180 --> 02:22:24,860 But not everybody agrees on that so that is definitely a possible future.
02:22:24,860 --> 02:22:31,700 The sub-interpreters look like a fairly safe bet for 3.12 so say a year from now.
02:22:31,700 --> 02:22:32,700 A year.
02:22:32,700 --> 02:22:37,060 So the goal is to do a new version every year for Python.
02:22:37,060 --> 02:22:42,580 Let me ask you perhaps a fun question but there's a philosophy to it too.
02:22:42,580 --> 02:22:45,780 Will there ever be a Python 4.0?
02:22:45,780 --> 02:22:56,540 Now before you say it's currently a joke and probably not we're gonna go to 3.99 or 3.999.
02:22:56,540 --> 02:23:06,180 Can you imagine possible features that Python 4.0 might have that would necessitate the
02:23:06,180 --> 02:23:09,860 creation of the new 4.0?
02:23:09,860 --> 02:23:16,540 Given the amount of pain and joy, suffering and triumph that was involved in the move
02:23:16,540 --> 02:23:21,180 between version 2 and version 3.
02:23:21,180 --> 02:23:34,020 Yeah well as a community and as a core development team we have a large amount of painful memories
02:23:34,020 --> 02:23:38,900 about the Python 3 transition.
02:23:38,900 --> 02:23:45,820 Which is one reason that sort of everybody is happy that we've decided there's not going
02:23:45,820 --> 02:23:53,860 to be a 4.0 at least not anytime soon and if there is going to be one we'll sort of
02:23:53,860 --> 02:23:57,220 plan the transition very differently.
02:23:57,220 --> 02:24:03,540 Because clearly we underestimated the pain the transition caused for our users in the
02:24:03,540 --> 02:24:13,940 Python 3 case and had we known we could have sort of designed Python 3 somewhat differently
02:24:13,940 --> 02:24:17,780 without making it any worse.
02:24:17,780 --> 02:24:25,660 We just thought that we had a good plan but we underestimated what sort of the users were
02:24:25,660 --> 02:24:28,900 capable of when it comes to that kind of transition.
02:24:28,900 --> 02:24:37,780 By the way I think we talked way before like a year and a half before the Python 2 officially
02:24:37,780 --> 02:24:38,780 end of life.
02:24:38,780 --> 02:24:39,780 End of life.
02:24:39,780 --> 02:24:40,780 Oh yeah.
02:24:40,780 --> 02:24:43,780 What was that, what was your memory of the end of life?
02:24:43,780 --> 02:24:47,140 Did you shed a tear on January 1st 2020?
02:24:47,140 --> 02:24:48,660 Was there, were you standing alone?
02:24:48,660 --> 02:24:54,220 Everyone on the core team had basically moved on years before.
02:24:54,220 --> 02:25:06,540 It was, it was purely, it was a little symbolic moment to signal to the remaining users that
02:25:06,540 --> 02:25:14,620 there was no longer going to be any new releases or support for Python 2.7.
02:25:14,620 --> 02:25:18,980 Did you shed a single tear while looking out over the horizon?
02:25:18,980 --> 02:25:27,260 I'm not, not a very poetic person and I don't shed tears like that but no.
02:25:27,260 --> 02:25:33,700 We actually had planned a party but the party was planned for the Python, the US Python
02:25:33,700 --> 02:25:37,780 conference that year which would never happened of course because of the pandemic.
02:25:37,780 --> 02:25:39,900 Oh, was it like in March or something?
02:25:39,900 --> 02:25:46,160 Yeah, the conference was going to be I think late April that year.
02:25:46,160 --> 02:25:51,780 So that, that was a very difficult decision to cancel it but they did.
02:25:51,780 --> 02:25:57,180 So anyway, if we're going to have a Python 4, we're going to have to have both a different
02:25:57,180 --> 02:26:04,180 reason for, for having that and a different process for managing the transition.
02:26:04,180 --> 02:26:10,380 Can you imagine a possible process that, so, so I think you're implying that if there's
02:26:10,380 --> 02:26:14,700 a 4.0 in some ways it would break back compatibility?
02:26:14,700 --> 02:26:22,620 Well, so here is, here is a concrete thought I've had and I'm not unique but not everyone
02:26:22,620 --> 02:26:27,620 agrees with this so this is definitely a personal opinion.
02:26:27,620 --> 02:26:41,780 If we were to try something like that Noguil Python, my expectation is that it would feel
02:26:41,780 --> 02:26:52,180 just different enough at least for the part of the Python ecosystem that is heavily based
02:26:52,180 --> 02:27:00,140 on C extensions and that is like the entire machine learning, data science, scientific
02:27:00,140 --> 02:27:06,660 Python world is all based on C extensions for Python.
02:27:06,660 --> 02:27:18,340 And so those people would likely feel the pain the most because they, even if we don't
02:27:18,340 --> 02:27:23,660 change anything about the syntax of the language and the semantics of the language when you're
02:27:23,660 --> 02:27:31,620 writing Python code, we could even say, suppose that after Python say 3.19 instead of 3.20
02:27:31,620 --> 02:27:34,180 we'll have 4.0.
02:27:34,180 --> 02:27:41,500 Suppose that's the time when we flip the switch to 4.0 we'll not have a GIL.
02:27:41,500 --> 02:27:43,740 Imagine it was like that.
02:27:43,740 --> 02:27:54,980 So I would probably say that particular year the release that we name 4.0 will be syntactically,
02:27:54,980 --> 02:28:01,540 it will not have any new syntactical features, no new modules in the standard library, no
02:28:01,540 --> 02:28:05,020 new built-in functions.
02:28:05,020 --> 02:28:13,140 Everything will be at the Python level will be purely compatible with Python 3.19.
02:28:13,140 --> 02:28:21,840 However, extension modules will have to make a change, they will have to be recompiled,
02:28:21,840 --> 02:28:35,540 they will not have the same binary interface, the semantics and APIs for some things that
02:28:35,540 --> 02:28:40,500 are frequently accessed by C extensions will be different.
02:28:40,500 --> 02:28:48,700 And so for a pure Python user 4.0 would be a breeze, except that there are very few pure
02:28:48,700 --> 02:28:54,300 Python users left, because everybody who is using Python for something significant is
02:28:54,300 --> 02:29:00,200 using third-party extensions, there are like, I don't know, several hundreds of thousands
02:29:00,200 --> 02:29:06,420 of third-party extensions on the PyPI service.
02:29:06,420 --> 02:29:12,620 And I'm not saying they're all good, but there is a large list of extensions that would have
02:29:12,620 --> 02:29:20,380 to do work and some of those extensions are currently already low on maintainers and they're
02:29:20,380 --> 02:29:23,860 struggling to keep afloat.
02:29:23,860 --> 02:29:29,980 So there you can give a huge heads up to them if you go to 4.0 to really keep developing
02:29:29,980 --> 02:29:30,980 it.
02:29:30,980 --> 02:29:37,420 Yeah, we'd probably have to do something like several years before, who knows, maybe five
02:29:37,420 --> 02:29:45,300 years earlier, like 3.15, we would have to say, and I'm just making the specific numbers
02:29:45,300 --> 02:29:53,180 up, but at some point we'd have to say that Nogail Python could be an option.
02:29:53,180 --> 02:29:57,360 It might be a compile-time option.
02:29:57,360 --> 02:30:04,300 If you want to use Nogail Python, you have to recompile Python from source for your platform
02:30:04,300 --> 02:30:06,500 using your toolset.
02:30:06,500 --> 02:30:13,100 All you have to do is change one configuration variable and then you just run make or configure
02:30:13,100 --> 02:30:15,960 and make and it will build it for you.
02:30:15,960 --> 02:30:23,420 But now you also have to use the Nogail compatible versions of all extension modules you want
02:30:23,420 --> 02:30:25,020 to use.
02:30:25,020 --> 02:30:33,180 And so as long as many extension modules don't have fully functional sort of variants that
02:30:33,180 --> 02:30:40,200 work in the Nogail world, that's not a very practical thing for Python users, but it would
02:30:40,200 --> 02:30:49,540 allow extension developers to test the waters, see what they need to syntactically to be
02:30:49,540 --> 02:30:51,380 able to compile at all.
02:30:51,380 --> 02:30:57,780 Maybe they're using functions that are defined by the Python 3 runtime that won't be in the
02:30:57,780 --> 02:30:59,300 Python 4 runtime.
02:30:59,300 --> 02:31:00,660 Those functions will not work.
02:31:00,660 --> 02:31:07,980 They'll have to find an alternative, but they can experiment with that and sort of write
02:31:07,980 --> 02:31:08,980 test applications.
02:31:08,980 --> 02:31:11,340 And that would be a way to transition.
02:31:11,340 --> 02:31:21,000 And that could be a series of releases where Python 4 is more and more imminent.
02:31:21,000 --> 02:31:28,540 We have supported more and more third-party extension modules to have solid support that
02:31:28,540 --> 02:31:34,500 works for Nogail Python for that new API.
02:31:34,500 --> 02:31:42,660 And then sort of Python 4.0 is like the official moment that the mayor comes out and cuts the
02:31:42,660 --> 02:31:49,820 ribbon and now Python, now the sort of Nogail mode is the default and maybe the only mode
02:31:49,820 --> 02:31:51,900 there is.
02:31:51,900 --> 02:31:58,860 The internet wants to know from Reddit, it's a small and fun question.
02:31:58,860 --> 02:32:08,260 There's many fun questions, but out of the PyPi packages, PyPI packages, do you have
02:32:08,260 --> 02:32:09,260 ones you like?
02:32:09,260 --> 02:32:14,180 Do you, in your opinion, are there must-have PyPi libraries or ones you use all the time
02:32:14,180 --> 02:32:15,180 constantly?
02:32:15,180 --> 02:32:23,060 Oh my, I should really have a standard answer for that question, but like a positive standard
02:32:23,060 --> 02:32:24,060 answer.
02:32:24,060 --> 02:32:30,340 But my current standard answer is that I'm not a big user of third-party packages.
02:32:30,340 --> 02:32:39,260 When I write Python code, I'm usually developing some tooling around building Python itself.
02:32:39,260 --> 02:32:43,640 And the last thing we want is dependencies on third-party packages.
02:32:43,640 --> 02:32:47,140 So I tend to just use the standard library.
02:32:47,140 --> 02:32:50,500 That's where your focus is, that's where your mind is.
02:32:50,500 --> 02:32:56,460 But do you keep an eye on what's out there to understand where the standard library could
02:32:56,460 --> 02:32:58,020 be moving, should be moving?
02:32:58,020 --> 02:33:02,900 It's a good kind of landscape of what's missing from the standard library.
02:33:02,900 --> 02:33:11,900 Well, usually when something's missing from the standard library, nowadays it is a relatively
02:33:11,900 --> 02:33:22,260 new idea and there is a third-party implementation or maybe possibly multiple third-party implementations,
02:33:22,260 --> 02:33:28,180 but they evolve at a much higher rate than they could when they're in the standard library.
02:33:28,180 --> 02:33:37,780 So it would be a big reduction in activity to incorporate things like that in the standard
02:33:37,780 --> 02:33:38,780 library.
02:33:38,780 --> 02:33:45,060 I like that there is a lively package ecosystem and that sort of recent trends in the standard
02:33:45,060 --> 02:33:49,580 library are actually that we're doing the occasional spring cleaning where we're just
02:33:49,580 --> 02:34:05,860 choosing some modules that have not had a lot of change in a long time and that maybe
02:34:05,860 --> 02:34:10,300 would be better off not existing at all at this point because there might be a better
02:34:10,300 --> 02:34:17,620 third-party alternative anyway and we're sort of slowly removing those.
02:34:17,620 --> 02:34:25,420 Often those are things that I sort of, I spiked somewhere in 1992 or 1993.
02:34:25,420 --> 02:34:32,940 If you look through the commit history, it's very sad, like all cosmetic changes like changes
02:34:32,940 --> 02:34:39,060 in the indentation style or the name of this other standard library module got changed
02:34:39,060 --> 02:34:48,420 or nothing of any substance, the API is identical to what it was 20 years ago.
02:34:48,420 --> 02:34:53,740 So speaking of packages, they have a lot of impact on a lot of people's lives.
02:34:53,740 --> 02:34:59,380 Does it make sense to you why Python has become the primary, the dominant language for the
02:34:59,380 --> 02:35:01,020 machine learning community?
02:35:01,020 --> 02:35:07,660 So packages like PyTorch, TensorFlow, scikit-learn and even like the lower level stuff like NumPy,
02:35:07,660 --> 02:35:11,220 SciPy, Pandas, Matplotlib with the visualization.
02:35:11,220 --> 02:35:19,100 Can you like, does it make sense to you why it permeated the entire data science machine
02:35:19,100 --> 02:35:21,220 learning AI community?
02:35:21,220 --> 02:35:27,820 Well it's, part of it is an effect that's as simple as we're all driving on the right
02:35:27,820 --> 02:35:31,700 side of the road, right?
02:35:31,700 --> 02:35:44,060 It's compatibility and part of it is not quite as fundamental as driving on the right side
02:35:44,060 --> 02:35:50,220 of the road, which you have to do for safety reasons, I mean you have to agree on something.
02:35:50,220 --> 02:35:54,900 They could have picked JavaScript or Perl, there was a time in the early 2000s that it
02:35:54,900 --> 02:36:02,180 really looked like Perl was going to dominate like biosciences, because DNA search was all
02:36:02,180 --> 02:36:06,700 based on regular expressions and Perl has the fastest and most comprehensive regular
02:36:06,700 --> 02:36:08,980 expression engine, still does.
02:36:08,980 --> 02:36:15,060 I spent quite a long time with Perl, that was another letting go, letting go of this
02:36:15,060 --> 02:36:19,820 kind of data processing system.
02:36:19,820 --> 02:36:31,340 The reasons why Python became the lingua franca of scientific code and machine learning in
02:36:31,340 --> 02:36:39,820 particular and data science, it really had a lot to do with anything was better than
02:36:39,820 --> 02:36:42,820 C or C++.
02:36:42,820 --> 02:36:50,580 Recently a guy who worked at Lawrence Livermore National Laboratories in the sort of computing
02:36:50,580 --> 02:37:01,580 division wrote me his memoirs and he had his own view of how he helped something he called
02:37:01,580 --> 02:37:10,020 computational steering into existence and this was the idea that you take libraries
02:37:10,020 --> 02:37:19,500 that in his days were written in Fortran, that solved universal mathematical problems
02:37:19,500 --> 02:37:27,300 and those libraries still work, but the scientists that used the libraries, used them to solve
02:37:27,300 --> 02:37:35,380 continuously different specific applications and answer different questions and so those
02:37:35,380 --> 02:37:45,660 poor scientists were required to use say Fortran, because Fortran was the language that the
02:37:45,660 --> 02:37:51,820 library was written in, and then the scientists would have to write an application that sort
02:37:51,820 --> 02:37:59,500 of uses the library to solve a particular equation or set of, answer a set of questions
02:37:59,500 --> 02:38:09,780 and the same for C++, because there's interoperability so the dusty decks are written either in C++
02:38:09,780 --> 02:38:13,460 or Fortran.
02:38:13,460 --> 02:38:22,220 And so Paul Dubois was one of the people who I think in the mid-90s saw that you needed
02:38:22,220 --> 02:38:32,100 a higher level language for the scientists to sort of tie together the fundamental mathematical
02:38:32,100 --> 02:38:37,380 algorithms of linear algebra and other stuff.
02:38:37,380 --> 02:38:46,860 And so gradually some libraries started appearing that did very fundamental stuff with arrays
02:38:46,860 --> 02:38:53,940 of numbers in Python, I mean when I first created Python I was not expecting it to be
02:38:53,940 --> 02:39:00,060 used for arrays of numbers much, I thought that was like an outdated data type and everything
02:39:00,060 --> 02:39:06,780 was like objects and strings and like Python was good and fast at string manipulation and
02:39:06,780 --> 02:39:12,580 objects obviously, but arrays of numbers were not very efficient and the multidimensional
02:39:12,580 --> 02:39:17,940 arrays didn't even exist in the language at all.
02:39:17,940 --> 02:39:25,740 But there were people who realized that Python had extensibility that was flexible enough
02:39:25,740 --> 02:39:33,420 that they could write third-party packages that did support large arrays of numbers and
02:39:33,420 --> 02:39:36,940 operations on them very efficiently.
02:39:36,940 --> 02:39:45,020 And somehow they got a foothold through sort of different parts of the scientific community.
02:39:45,020 --> 02:39:50,140 I remember that the Hubble Space Telescope people in Baltimore were somehow big Python
02:39:50,140 --> 02:39:53,220 fans in the late 90s.
02:39:53,220 --> 02:40:02,020 And at various points small improvements were made and more people got in touch with using
02:40:02,020 --> 02:40:12,000 Python to derive these libraries of interesting algorithms and like once you have a bunch
02:40:12,000 --> 02:40:18,660 of scientists who are working on similar problems, say they're all working on stuff that comes
02:40:18,660 --> 02:40:22,460 in from the Hubble Space Telescope, but they're looking at different things.
02:40:22,460 --> 02:40:27,220 Some are looking at stars in this galaxy, others are looking at galaxies, the math is
02:40:27,220 --> 02:40:33,660 completely different, but the underlying libraries are still the same.
02:40:33,660 --> 02:40:40,380 And so they exchange code, they say, well, I wrote this Python program or I wrote a Python
02:40:40,380 --> 02:40:44,660 library to solve this class of problems.
02:40:44,660 --> 02:40:50,180 And the other guys either say, oh, I can use that library too or if you make a few changes
02:40:50,180 --> 02:40:53,380 I can use that library too.
02:40:53,380 --> 02:41:00,980 Why start from scratch in Perl or JavaScript where there's not that infrastructure for
02:41:00,980 --> 02:41:04,920 arrays of numbers yet, whereas in Python you have it.
02:41:04,920 --> 02:41:12,980 And so more and more scientists at different places doing different work discovered Python
02:41:12,980 --> 02:41:21,020 and then people who had an idea for an important new fundamental library decided, oh, Python
02:41:21,020 --> 02:41:28,900 is actually already known to our users, so let's use Python as the user interface.
02:41:28,900 --> 02:41:33,600 I think that's how, I imagine at least, that's how TensorFlow ended up with Python as the
02:41:33,600 --> 02:41:35,180 user interface.
02:41:35,180 --> 02:41:43,220 Right, but with TensorFlow there's a deeper history of what the community, so it's not
02:41:43,220 --> 02:41:48,100 just like what packages it needs, it's like what the community leans on for programming
02:41:48,100 --> 02:41:55,780 language because TensorFlow had a prior library that was internal to Google, but there's also
02:41:55,780 --> 02:42:02,860 competing machine learning frameworks like Theano, Caffe, that were in Python, there
02:42:02,860 --> 02:42:10,060 was some Scala, some other languages, but Python was really dominating it.
02:42:10,060 --> 02:42:15,100 And it's interesting because there's other languages from the engineering space like
02:42:15,100 --> 02:42:22,740 MATLAB that a lot of people used, but different design choices by the company, by the core
02:42:22,740 --> 02:42:26,100 developers led to it not spreading.
02:42:26,100 --> 02:42:33,660 And one of the choices of MATLAB by MathWorks is to not make it open source, or not having
02:42:33,660 --> 02:42:34,660 people pay.
02:42:34,660 --> 02:42:41,860 It was a very expensive product and so universities especially disliked it because it was a price
02:42:41,860 --> 02:42:45,100 per seat, I remember hearing.
02:42:45,100 --> 02:42:50,780 Yeah, but I think that's not why it failed, or it failed to spread.
02:42:50,780 --> 02:42:55,300 I think the universities didn't like it, but they would still pay for it.
02:42:55,300 --> 02:43:04,180 The thing is, it didn't feed into that GitHub open source packages culture, and that's somehow
02:43:04,180 --> 02:43:08,260 a precondition for viral spreading.
02:43:08,260 --> 02:43:13,460 The hacker culture, like the tinkerer culture, with Python it feels like you can build a
02:43:13,460 --> 02:43:17,700 package from scratch to solve a particular problem and get excited about sharing that
02:43:17,700 --> 02:43:19,220 package with others.
02:43:19,220 --> 02:43:22,340 And that creates an excitement about a language.
02:43:22,340 --> 02:43:28,780 I tend to like Python's approach to open source in particular because it's sort of, it's almost
02:43:28,780 --> 02:43:35,420 egalitarian, there's little hierarchy.
02:43:35,420 --> 02:43:40,060 There's obviously some, because like you all need to decide whether you drive on the left
02:43:40,060 --> 02:43:42,980 or the right side of the road sometimes.
02:43:42,980 --> 02:43:48,900 But there is a lot of access for people with little power, you don't have to work for a
02:43:48,900 --> 02:43:54,340 big tech company to make a difference in the Python world.
02:43:54,340 --> 02:44:02,020 We have affordable events that really care about community and support people, and sort
02:44:02,020 --> 02:44:11,260 of the community is like a big deal at our conferences and in the PSF.
02:44:11,260 --> 02:44:18,020 When the PSF funds events, it's always about growing the community.
02:44:18,020 --> 02:44:23,900 The PSF funds very little development.
02:44:23,900 --> 02:44:35,740 They do some, but most of the money that the PSF forks out is to community fostering things.
02:44:35,740 --> 02:44:41,500 So speaking of egalitarian, last time we talked, four years ago, it was just after you stepped
02:44:41,500 --> 02:44:47,780 down from your role as the benevolent dictator for life, BDFL.
02:44:47,780 --> 02:44:53,340 Looking back, what are your insights and lessons you learned from that experience about Python
02:44:53,340 --> 02:45:00,780 developer community, about human nature, about human civilization, life itself?
02:45:00,780 --> 02:45:08,820 Oh my, I probably held on to the position too long.
02:45:08,820 --> 02:45:21,220 I remember being just extremely stressed for a long time and it wasn't very clear to me
02:45:21,220 --> 02:45:26,780 what was leading, what was causing the stress.
02:45:26,780 --> 02:45:40,380 And looking back, I should have sort of relinquished my central role as BDFL sooner.
02:45:40,380 --> 02:45:42,980 What were the pros and cons of the BDFL role?
02:45:42,980 --> 02:45:48,700 What were the, you not relinquishing it, what are the benefits of that for the community?
02:45:48,700 --> 02:45:50,860 And what are the drawbacks?
02:45:50,860 --> 02:46:02,140 Well, the benefits for the community would be things like clarity of vision and sort
02:46:02,140 --> 02:46:12,740 of a clear direction because I had certain ideas in mind when I created Python.
02:46:12,740 --> 02:46:20,060 And while I sort of let myself be influenced by many other ideas as Python evolved and
02:46:20,060 --> 02:46:30,820 became more successful and more complex and more used, I also stuck to certain principles.
02:46:30,820 --> 02:46:39,180 And it's still hard to say what are Python's core principles, but the fact that I was playing
02:46:39,180 --> 02:46:47,940 that role and sort of always very active grew the community in a certain way.
02:46:47,940 --> 02:46:55,700 It modeled to the community how to think about how to solve a certain problem.
02:46:55,700 --> 02:46:58,900 Well, that was a source of stress, but it was also beneficial.
02:46:58,900 --> 02:47:06,300 It was a source of stress for me personally, but it was beneficial for the community because
02:47:06,300 --> 02:47:16,380 people sort of over time had learned how I was thinking and could predict how I would
02:47:16,380 --> 02:47:23,580 decide about a particular issue and not always perfectly, of course, but there wasn't a lot
02:47:23,580 --> 02:47:29,540 of jerking around like this year we're all, but this year the Democrats are in power and
02:47:29,540 --> 02:47:31,220 we're doing these kinds of things.
02:47:31,220 --> 02:47:36,780 And now the Republicans are in power and they roll all that back and do those kinds of things.
02:47:36,780 --> 02:47:41,620 There is a clear, fairly straight path ahead.
02:47:41,620 --> 02:47:49,460 And so fortunately the successor structure with the steering council has sort of found
02:47:49,460 --> 02:47:58,820 a similar way of leading the community in a fairly steady direction without stagnating.
02:47:58,820 --> 02:48:03,900 And for me personally, it's more fun because there are things I can just ignore.
02:48:03,900 --> 02:48:04,900 Yeah.
02:48:04,900 --> 02:48:05,900 Oh yeah.
02:48:05,900 --> 02:48:07,980 There's a bug in multiprocessing.
02:48:07,980 --> 02:48:13,140 Let someone else decide whether that's important to solve or not.
02:48:13,140 --> 02:48:18,220 I'll stick to typing in the async IO and the faster interpreter.
02:48:18,220 --> 02:48:19,220 Yeah.
02:48:19,220 --> 02:48:23,260 It allows you to focus a little bit more.
02:48:23,260 --> 02:48:27,420 What are interesting differences in culture, if you can comment on, between Google, Dropbox
02:48:27,420 --> 02:48:34,940 and Microsoft from a Python programming perspective, all places you've been to, the positive?
02:48:34,940 --> 02:48:39,900 Is there a difference or is it just about people and there's great people everywhere?
02:48:39,900 --> 02:48:43,260 Or is there culture differences?
02:48:43,260 --> 02:48:48,260 So Dropbox is much smaller than the other two in your list.
02:48:48,260 --> 02:48:52,780 So that is a big difference.
02:48:52,780 --> 02:48:58,940 The set of products they provide is narrower, so they're more focused, smaller code base.
02:48:58,940 --> 02:49:07,420 And Dropbox sort of, at least during the time I was there, had the tendency of sort of making
02:49:07,420 --> 02:49:14,100 a big plan, putting the whole company behind that plan for a year and then evaluate and
02:49:14,100 --> 02:49:21,580 then suddenly find that everything was wrong about the plan and then they had to do something
02:49:21,580 --> 02:49:23,740 completely different.
02:49:23,740 --> 02:49:30,980 And so there was like the annual engineering reorg was sort of an unpleasant tradition
02:49:30,980 --> 02:49:36,020 at Dropbox because like, oh, there's a new VP of engineering and so now all the directors
02:49:36,020 --> 02:49:44,020 are being reshuffled and this guy was in charge of infrastructure one year and the next year
02:49:44,020 --> 02:49:48,660 he was made in charge of, I don't know, product development.
02:49:48,660 --> 02:49:53,540 It's fascinating because like you don't think about these companies internally, but Dropbox
02:49:53,540 --> 02:49:57,580 to me from the very beginning was one of my favorite services.
02:49:57,580 --> 02:50:03,420 There's certain like programs and online services that make me happy, make me more efficient
02:50:03,420 --> 02:50:04,620 and all that kind of stuff.
02:50:04,620 --> 02:50:08,820 But one of the powers of those kinds of services, they disappear.
02:50:08,820 --> 02:50:12,100 You're not supposed to think about how it all works, but it's incredible to me that
02:50:12,100 --> 02:50:20,180 you can sync stuff effortlessly across so many machines so quickly and like don't have
02:50:20,180 --> 02:50:25,220 to worry about conflicts, they take care of the, you know, as a person that comes from
02:50:25,220 --> 02:50:31,500 version repositories and all that kind of stuff or merge is super difficult and just
02:50:31,500 --> 02:50:34,220 keeping different versions of different files is very tricky.
02:50:34,220 --> 02:50:38,620 The fact that they could take care of that is just, I don't know, the engineering behind
02:50:38,620 --> 02:50:43,540 the scenes must be super difficult, both on the compute infrastructure and the software.
02:50:43,540 --> 02:50:51,980 A lot of internal sort of hand-wringing about things like that, but the product itself always
02:50:51,980 --> 02:50:54,700 worked very smoothly and still does.
02:50:54,700 --> 02:50:56,860 But there's probably a lot of lessons to that.
02:50:56,860 --> 02:51:02,260 You can have a lot of turmoil inside on the engineering side, but if the product is good,
02:51:02,260 --> 02:51:03,260 the product is good.
02:51:03,260 --> 02:51:08,420 And don't, maybe don't mess with that either, you know, when it's good, keep, it's like
02:51:08,420 --> 02:51:12,780 with Google, focus on the search and the ads, right?
02:51:12,780 --> 02:51:14,060 And the money will come.
02:51:14,060 --> 02:51:15,060 Yeah.
02:51:15,060 --> 02:51:19,780 And make sure that's done extremely well and don't forget what you do extremely well.
02:51:19,780 --> 02:51:24,140 In what ways do you provide value and happiness to the world?
02:51:24,140 --> 02:51:27,020 Make sure you do that well.
02:51:27,020 --> 02:51:30,180 Is there something else to say about Google and Microsoft?
02:51:30,180 --> 02:51:36,780 Microsoft has had a very fascinating shift recently with a new CEO, what, you know, recent
02:51:36,780 --> 02:51:43,260 CEO with purchasing GitHub, embracing open source culture, embracing the developer culture
02:51:43,260 --> 02:51:44,820 is pretty interesting to see.
02:51:44,820 --> 02:51:47,020 That's like why I joined Microsoft.
02:51:47,020 --> 02:51:53,820 I mean, after retiring and thinking that I would stay retired for the rest of my life,
02:51:53,820 --> 02:51:59,460 which of course was a ridiculous thought, but I was, I was, I was done working for a
02:51:59,460 --> 02:52:05,180 bit and then the pandemic made me realize that work, work can also provide a source
02:52:05,180 --> 02:52:11,980 of fulfillment, keep you out of trouble.
02:52:11,980 --> 02:52:20,060 Microsoft is a very interesting company because it has this incredible, very long and varied
02:52:20,060 --> 02:52:29,980 history and this amazing catalog of products that many of which also date way back.
02:52:29,980 --> 02:52:40,220 I mean, I've been talking to a bunch of Excel people lately and Excel is like 35 years old
02:52:40,220 --> 02:52:46,700 and they can still read spreadsheets that they might find on an old floppy drive.
02:52:46,700 --> 02:52:47,700 Yeah.
02:52:47,700 --> 02:52:48,700 Yeah.
02:52:48,700 --> 02:52:53,180 There's, man, there've been so many incredible tools through the years.
02:52:53,180 --> 02:53:01,300 Excel, one of the great shames of my life is that I've never learned how to use Excel
02:53:01,300 --> 02:53:02,300 well.
02:53:02,300 --> 02:53:05,780 I mean, it just always felt like so many features are there.
02:53:05,780 --> 02:53:08,780 It's similar with IDEs like PyCharm.
02:53:08,780 --> 02:53:14,300 It feels like I converged quickly to the dumbest way to use a thing to get the job done when
02:53:14,300 --> 02:53:17,540 clearly there's so much more power at your fingertips.
02:53:17,540 --> 02:53:18,740 Yeah.
02:53:18,740 --> 02:53:22,300 But there's, I do think there's probably expert users of Excel.
02:53:22,300 --> 02:53:25,660 Oh, Excel is a cash cow, actually.
02:53:25,660 --> 02:53:27,700 Oh, it actually brings in money.
02:53:27,700 --> 02:53:28,780 Oh, yeah.
02:53:28,780 --> 02:53:35,500 A lot of the engineering, sort of, if you look deep inside Excel, there's some very
02:53:35,500 --> 02:53:39,260 good engineering, very, very impressive stuff.
02:53:39,260 --> 02:53:40,260 Okay.
02:53:40,260 --> 02:53:42,940 Now I need to definitely learn Excel a little better.
02:53:42,940 --> 02:53:47,700 I had issues because I'm a keyboard person, so I had issues coming up with shortcuts.
02:53:47,700 --> 02:53:52,660 I mean, Microsoft sometimes, it's changed over the years, but sometimes they kind of
02:53:52,660 --> 02:54:00,540 want to make things easier for you on the surface and therefore make it harder for people
02:54:00,540 --> 02:54:05,220 that like to have shortcuts and all that kind of stuff to optimize their workflow.
02:54:05,220 --> 02:54:09,140 Now Excel is probably, people are probably yelling at me, it's like, no, Excel probably
02:54:09,140 --> 02:54:11,140 has a lot of ways to optimize the workflow.
02:54:11,140 --> 02:54:16,900 In fact, I keep discovering that there are many features in Excel that only exists at
02:54:16,900 --> 02:54:18,660 keyboard shortcuts.
02:54:18,660 --> 02:54:20,060 Yeah.
02:54:20,060 --> 02:54:21,060 That's the sense I have.
02:54:21,060 --> 02:54:23,740 And now, like, I'm embarrassed that it's just...
02:54:23,740 --> 02:54:26,260 You just have to know what they are.
02:54:26,260 --> 02:54:32,980 That's like, there's no logic or reason to the assignment of the keyboard shortcuts,
02:54:32,980 --> 02:54:37,340 because they go back even longer than 35 years.
02:54:37,340 --> 02:54:42,540 Can you maybe comment about such in Adela and how hard it is for a CEO to sort of pivot
02:54:42,540 --> 02:54:45,700 a company towards open source, towards developer culture?
02:54:45,700 --> 02:54:52,220 Is there something you could see about like, what's the role of leadership in such a pivot
02:54:52,220 --> 02:54:54,100 and definition of a new vision?
02:54:54,100 --> 02:55:07,100 I've never met him, but I hear he's just a really sharp thinker, but he also has an incredible
02:55:07,100 --> 02:55:09,180 business sense.
02:55:09,180 --> 02:55:17,700 He took the organization that had very solid pieces, but that was also struggling with
02:55:17,700 --> 02:55:25,220 all sorts of shameful things, especially the Steve Ballmer time, I imagine in part through
02:55:25,220 --> 02:55:32,180 his personal charm and thinking, and of course, the great trust that the rest of the leadership
02:55:32,180 --> 02:55:33,180 has in him.
02:55:33,180 --> 02:55:41,300 He managed to really turn the company around and sort of change it from openly hostile
02:55:41,300 --> 02:55:47,180 to open source, to actively embracing open source.
02:55:47,180 --> 02:55:52,060 And that doesn't mean that suddenly Excel is going to go open source, but that means
02:55:52,060 --> 02:55:56,180 that there's room for a product like VS Code, which is open source.
02:55:56,180 --> 02:55:58,260 Yeah, that's fascinating.
02:55:58,260 --> 02:56:04,780 It gives me faith that large companies with good leadership can grow, can expand, can
02:56:04,780 --> 02:56:09,980 change and pivot and so on and develop, because it gets harder and harder as the company gets
02:56:09,980 --> 02:56:11,820 large.
02:56:11,820 --> 02:56:16,180 You wrote a blog post in response to a person looking for advice about whether with a CS
02:56:16,180 --> 02:56:21,100 degree to choose a nine to five job or to become an entrepreneur.
02:56:21,100 --> 02:56:23,140 It's an interesting question.
02:56:23,140 --> 02:56:29,220 If you just think from first principles right now, somebody has took a few years in programming,
02:56:29,220 --> 02:56:36,860 has loved software engineering, in some sense, creating Python is an entrepreneurial endeavor.
02:56:36,860 --> 02:56:40,740 That's a choice that a lot of people that are good programmers have to make.
02:56:40,740 --> 02:56:48,260 Do I work for a big company or do I create something new?
02:56:48,260 --> 02:56:53,180 Or you can work for a big company and create something new there.
02:56:53,180 --> 02:57:02,280 Yeah, I mean, big companies have individuals who create new stuff that eventually grows
02:57:02,280 --> 02:57:04,380 big all the time.
02:57:04,380 --> 02:57:08,340 And if you're the person that creates a new thing and grows big, you'll have a chance
02:57:08,340 --> 02:57:12,620 to move up quickly in the company to run that thing.
02:57:12,620 --> 02:57:14,700 If that's your aspiration.
02:57:14,700 --> 02:57:22,980 What can also happen is that someone is brilliant engineer and sort of builds a great first
02:57:22,980 --> 02:57:31,780 version of a product and has no aspirations to then become a manager and grow the team
02:57:31,780 --> 02:57:39,180 from five people to 20 people to 100 people to 1000 people and be in charge of hiring
02:57:39,180 --> 02:57:46,560 and meetings and they move on to inventing another crazy thing inside the same company
02:57:46,560 --> 02:57:56,140 or sometimes they found a startup or they move to a different great large or small company.
02:57:56,140 --> 02:57:58,780 There's all sorts of models.
02:57:58,780 --> 02:58:06,580 And sometimes people sort of do have this whole trajectory from engineer buckling down,
02:58:06,580 --> 02:58:16,900 writing code, not nine to five, but more like noon till midnight, seven days a week and
02:58:16,900 --> 02:58:22,540 coming up with a product and sort of staying in charge.
02:58:22,540 --> 02:58:30,500 I mean, if you take Drew Houston, Dropbox's founder, he is still the CEO.
02:58:30,500 --> 02:58:33,940 And at least when I was there, he had not checked out or anything.
02:58:33,940 --> 02:58:43,340 He was a good CEO, but he had started out as the technical inventor or co-inventor.
02:58:43,340 --> 02:58:48,900 And so he was someone who, I don't know if he always aspired that.
02:58:48,900 --> 02:58:52,360 I think when he was 16, he already started a company.
02:58:52,360 --> 02:59:01,500 So maybe he did, but he sort of, it turned out that he did have the personal sort of
02:59:01,500 --> 02:59:06,180 skill set needed to grow and stay on top.
02:59:06,180 --> 02:59:12,460 And other people sort of are brilliant engineers and horrible at management.
02:59:12,460 --> 02:59:16,340 I count myself at least in the second category.
02:59:16,340 --> 02:59:22,620 So your first love and still your love is to be the quote unquote individual contributor.
02:59:22,620 --> 02:59:26,140 So the programmer.
02:59:26,140 --> 02:59:36,620 Do you have advice for a programming beginner on how to learn Python the right way?
02:59:36,620 --> 02:59:42,100 Find something you actually want to do with it.
02:59:42,100 --> 02:59:49,060 If you say, I want to learn skill X, that's not enough motivation.
02:59:49,060 --> 02:59:55,540 You need to pick something and it can be a crazy problem you want to solve.
02:59:55,540 --> 03:00:01,380 It can be completely unrealistic.
03:00:01,380 --> 03:00:11,380 But something that challenges you into actually learning coding in some language.
03:00:11,380 --> 03:00:14,020 And there's so many projects out there you can look for like that.
03:00:14,020 --> 03:00:15,820 That doesn't have to be some big ambitious thing.
03:00:15,820 --> 03:00:18,620 It could be writing a small bot.
03:00:18,620 --> 03:00:24,700 If you're into social media, you can write a Reddit bot or a Twitter bot or some aspect
03:00:24,700 --> 03:00:30,220 of automating something that you do every single day, processing files, all that kind
03:00:30,220 --> 03:00:31,220 of stuff.
03:00:31,220 --> 03:00:39,020 Nowadays, you can take machine learning components and sort of plug those things together.
03:00:39,020 --> 03:00:40,340 So cool stuff with them.
03:00:40,340 --> 03:00:42,020 So that's actually a really good example.
03:00:42,020 --> 03:00:46,420 So if you're interested in machine learning, the state of machine learning is such that
03:00:46,420 --> 03:00:53,260 like a tutorial that takes an hour can get you to start using pre-trained models to do
03:00:53,260 --> 03:00:54,820 something super cool.
03:00:54,820 --> 03:00:58,700 And that's a good way to learn Python because you learn just enough to run this model.
03:00:58,700 --> 03:01:05,260 And that's like a sneaky way to get in there, to figure out how to import stuff, how to
03:01:05,260 --> 03:01:10,220 write basic IO, how to run functions.
03:01:10,220 --> 03:01:13,940 And I'm not sure if it's the best way to learn the basics in Python, but it could be nice
03:01:13,940 --> 03:01:17,820 to just fall in love first and then figure out the basics, right?
03:01:17,820 --> 03:01:24,300 Yeah, you can't expect to learn Python from a one hour video, kind of blanking out on
03:01:24,300 --> 03:01:34,660 the name of someone who wrote a very funny blog post where he said, I see all these ads
03:01:34,660 --> 03:01:40,580 for things like learn Python in 10 days or so.
03:01:40,580 --> 03:01:45,500 And he said, the goal should be learn Python in 10 years.
03:01:45,500 --> 03:01:49,100 That's hilarious, but I completely disagree with that.
03:01:49,100 --> 03:01:55,660 I think the criticism behind that is that the places, just like the blog post from earlier,
03:01:55,660 --> 03:02:00,060 the places that tell you learn Python in five minutes or 10 minutes, they're actually usually
03:02:00,060 --> 03:02:01,180 really bad tutorials.
03:02:01,180 --> 03:02:08,540 So the thing is, I do believe that you can learn a thing in an hour to like get some
03:02:08,540 --> 03:02:11,980 interesting, quick, like it hooks you.
03:02:11,980 --> 03:02:16,180 But it just takes a tremendous amount of skill to be that kind of educator.
03:02:16,180 --> 03:02:20,420 Richard Feynman was able to condense a lot of ideas in physics in a very short amount
03:02:20,420 --> 03:02:21,420 of time.
03:02:21,420 --> 03:02:23,300 But that takes a deep, deep understanding.
03:02:23,300 --> 03:02:29,940 And so yes, of course, the actual, I think the 10 years is about the experience, the
03:02:29,940 --> 03:02:31,660 pain along the way, and there's something fundamental.
03:02:31,660 --> 03:02:33,340 Well, you have to practice.
03:02:33,340 --> 03:02:40,500 You can memorize the syntax, but, well I couldn't, but maybe someone else can, but that doesn't
03:02:40,500 --> 03:02:42,180 make you a coder.
03:02:42,180 --> 03:02:43,180 Yeah.
03:02:43,180 --> 03:02:49,060 Actually, coding has changed in fascinating ways because so much of coding is copying
03:02:49,060 --> 03:02:53,700 pasting from Stack Overflow and then adjusting, which is another way of coding.
03:02:53,700 --> 03:02:58,100 And I don't want to talk down to that kind of style of coding because it's kind of nicely
03:02:58,100 --> 03:02:59,100 efficient.
03:02:59,100 --> 03:03:01,940 But do you know where that is going?
03:03:01,940 --> 03:03:02,940 Code generation?
03:03:02,940 --> 03:03:03,940 No, seriously.
03:03:03,940 --> 03:03:04,940 GitHub Copilot.
03:03:04,940 --> 03:03:06,180 Yeah, Copilot.
03:03:06,180 --> 03:03:08,340 I use it every day and it-
03:03:08,340 --> 03:03:09,340 Really?
03:03:09,340 --> 03:03:15,380 Yeah, it writes a lot of code for me and usually it's slightly wrong, but it still saves me
03:03:15,380 --> 03:03:16,380 typing.
03:03:16,380 --> 03:03:23,260 Because all I have to do is like change one word in a line of text that otherwise it generated
03:03:23,260 --> 03:03:24,260 perfectly.
03:03:24,260 --> 03:03:29,780 And like, how many times are you looking for like, oh, what was I doing this morning?
03:03:29,780 --> 03:03:34,300 I was looking for an begin marker and I was looking for an end marker.
03:03:34,300 --> 03:03:41,340 And so begin is blah, blah, blah, search for begin.
03:03:41,340 --> 03:03:43,420 This is the begin token.
03:03:43,420 --> 03:03:51,380 And then the next line I type E and it completes the whole line with end instead of begin.
03:03:51,380 --> 03:03:53,220 That's a very simple example.
03:03:53,220 --> 03:04:00,540 Sometimes it sort of, if I name my function right, it writes a five or 10 line function.
03:04:00,540 --> 03:04:06,700 And you know Python enough to very quickly then detect the issues.
03:04:06,700 --> 03:04:09,180 So it becomes a really good dance partner then.
03:04:09,180 --> 03:04:15,260 It doesn't save me a lot of thinking, but since I'm a poor typist, I'm very much appreciative
03:04:15,260 --> 03:04:20,180 of all the typing it does for me.
03:04:20,180 --> 03:04:25,780 Much better actually than the previous generation of suggestions that are also still built in
03:04:25,780 --> 03:04:34,560 VS Code, where when you hit like a dot, it tries to guess what the type is of the variable
03:04:34,560 --> 03:04:35,900 to the left of the dot.
03:04:35,900 --> 03:04:42,240 And then it gives you a list, a pop down menu of what the attributes of that object are.
03:04:42,240 --> 03:04:44,620 But Copilot is much, much smoother than that.
03:04:44,620 --> 03:04:49,380 What's fascinating to hear that you use GitHub Copilot.
03:04:49,380 --> 03:04:52,860 Do you think, do you worry about the future of that?
03:04:52,860 --> 03:05:01,740 With the automatic code generation, the increasing amount of that kind of capability, are programmers
03:05:01,740 --> 03:05:05,540 jobs threatened or is there still a significant role for humans?
03:05:05,540 --> 03:05:11,020 Are programmers jobs threatened by the existence of Stack Overflow?
03:05:11,020 --> 03:05:12,420 I don't think so.
03:05:12,420 --> 03:05:14,900 It helps you take care of the boring stuff.
03:05:14,900 --> 03:05:22,340 And you shouldn't try to use it to do something that you have no way of understanding what
03:05:22,340 --> 03:05:24,180 you're doing yet.
03:05:24,180 --> 03:05:31,220 A tool like that is always best when the question you're asking is, please remind me of how
03:05:31,220 --> 03:05:40,820 I do this, which I could do, I could look up how to do it, but right now I've forgotten
03:05:40,820 --> 03:05:47,780 whether the method is called foo or bar or what the shape of the API is, does it use
03:05:47,780 --> 03:05:55,900 a builder object or a constructor or a factory or something else?
03:05:55,900 --> 03:05:57,900 And what are the parameters?
03:05:57,900 --> 03:05:59,700 It serves that role.
03:05:59,700 --> 03:06:06,340 It's like a great assistant, but the creative work of sort of deciding what you want the
03:06:06,340 --> 03:06:10,500 code to do is totally yours.
03:06:10,500 --> 03:06:15,140 What do you think is the future of Python in the next 10, 20, 50 years, 100 years?
03:06:15,140 --> 03:06:16,140 You look forward.
03:06:16,140 --> 03:06:21,620 You ever think about, you ever imagine a future of human civilization or living inside the
03:06:21,620 --> 03:06:27,340 metaverse on Mars, humanoid robots everywhere?
03:06:27,340 --> 03:06:30,500 What part does Python play in that?
03:06:30,500 --> 03:06:37,860 It'll eventually become sort of a legacy language that plays an important role, but that most
03:06:37,860 --> 03:06:42,020 people have never heard of and don't need to know about.
03:06:42,020 --> 03:06:52,180 Just like all kinds of basic structures in biology, like mitochondria.
03:06:52,180 --> 03:06:57,860 So it permeates all of life, all of digital life, but people just build on top of it and
03:06:57,860 --> 03:07:01,300 they only know the stuff that's on top of it.
03:07:01,300 --> 03:07:03,140 You guys, you build layers of abstractions.
03:07:03,140 --> 03:07:12,460 I mean, most programmers nowadays rarely need to do binary arithmetic, right?
03:07:12,460 --> 03:07:14,020 Yeah.
03:07:14,020 --> 03:07:19,780 Or even think about it or even learn about it, or they could go quite far without knowing.
03:07:19,780 --> 03:07:27,060 I started building little digital circuits out of NAND gates that I built myself with
03:07:27,060 --> 03:07:29,400 transistors and resistors.
03:07:29,400 --> 03:07:39,180 So I sort of, I feel very blessed that with that start, when I was a teenager, I learned
03:07:39,180 --> 03:07:47,740 some of the basic, at least concepts that go into building a computer and I sort of
03:07:47,740 --> 03:07:57,600 every part, I have some understanding what it's for and why it's there and how it works.
03:07:57,600 --> 03:08:03,800 And I can forget about all that most of the time, but I sort of, I enjoy knowing, oh,
03:08:03,800 --> 03:08:13,380 if you go deeper, at some point you get to NAND gates and half adders and shift registers.
03:08:13,380 --> 03:08:19,060 When it comes to the point of how do you actually make a chip out of silicon, I have no idea.
03:08:19,060 --> 03:08:21,320 That's just magic to me.
03:08:21,320 --> 03:08:26,040 But you enjoy knowing that you can walk a while towards the lower and lower layers,
03:08:26,040 --> 03:08:27,680 but you don't need to.
03:08:27,680 --> 03:08:28,680 It's nice.
03:08:28,680 --> 03:08:35,640 The other day, as a sort of a mental exercise, I was trying to figure out if I could build
03:08:35,640 --> 03:08:42,600 a flip-flop circuit out of relays.
03:08:42,600 --> 03:08:49,360 I was just sort of trying to remember, oh, how does a relay work?
03:08:49,360 --> 03:08:57,120 Yeah, there's like this electromagnetic force that pulls a switch open or shut.
03:08:57,120 --> 03:09:05,200 And you can have like, it can open one switch and shut another, and you can have multiple
03:09:05,200 --> 03:09:07,360 contacts that go at once.
03:09:07,360 --> 03:09:12,720 And how many relays do I really need to sort of represent one bit of information?
03:09:12,720 --> 03:09:14,600 Can the relay just feed on itself?
03:09:14,600 --> 03:09:22,720 And it was, I don't think I got to the final solution, but it was fun that I could still
03:09:22,720 --> 03:09:27,120 do a little bit of problem solving and thinking at that level.
03:09:27,120 --> 03:09:29,600 And it's cool how we build on top of each other.
03:09:29,600 --> 03:09:34,000 So there's people that are just, you stood on the shoulders of giants and there's others
03:09:34,000 --> 03:09:38,200 who stand on your shoulders and it's a giant, beautiful hierarchy.
03:09:38,200 --> 03:09:44,960 Yeah, I feel I sort of covered this middle layer of the technology stack where it sort
03:09:44,960 --> 03:09:51,600 of peters out below the level of NAND gates.
03:09:51,600 --> 03:09:57,040 And at the top, I sort of, I lose track when it gets to machine learning.
03:09:57,040 --> 03:10:01,280 And then eventually the machine learning will build higher and higher layers that will help
03:10:01,280 --> 03:10:04,720 us understand the lowest layer of the physics.
03:10:04,720 --> 03:10:10,280 And thereby the universe figures out how it itself works.
03:10:10,280 --> 03:10:12,760 Maybe, maybe not.
03:10:12,760 --> 03:10:13,760 Yeah I did.
03:10:13,760 --> 03:10:14,920 I mean, it's possible.
03:10:14,920 --> 03:10:23,240 I mean, if you think of human consciousness, if that's even the right concept, it's interesting
03:10:23,240 --> 03:10:31,320 that sort of we have this super parallel brain that does all these incredible parallel operations
03:10:31,320 --> 03:10:33,480 like image recognition.
03:10:33,480 --> 03:10:40,360 I recognize your face, does huge amount of processing that goes on in parallel.
03:10:40,360 --> 03:10:44,840 There's lots of nerves between my eyes and my brain and the brain does a whole bunch
03:10:44,840 --> 03:10:49,720 of stuff all at once because it's actually really slow circuits, but there are many of
03:10:49,720 --> 03:10:52,760 them that all work together.
03:10:52,760 --> 03:10:59,120 On the other hand, when I'm speaking, everything is completely sequential.
03:10:59,120 --> 03:11:04,160 I have to sort of string words together one at a time.
03:11:04,160 --> 03:11:11,120 And when I'm thinking about stuff, when I'm understanding the world, I'm also thinking
03:11:11,120 --> 03:11:15,760 of everything like one step at a time.
03:11:15,760 --> 03:11:24,240 And so we've sort of, we've got all this incredible parallel circuitry in our brains and eventually
03:11:24,240 --> 03:11:32,080 we use that to simulate a single-threaded, much, much higher level interpreter.
03:11:32,080 --> 03:11:37,000 It's exactly, I mean, that's the illusion of it.
03:11:37,000 --> 03:11:42,880 That's the illusion of it for us, that it's a single sequential set of thoughts and all
03:11:42,880 --> 03:11:50,040 of that came from a single cell through the process of embryogenesis, so DNA is the code.
03:11:50,040 --> 03:11:53,360 DNA holds the entirety of the code.
03:11:53,360 --> 03:12:00,080 The information and how to use that information to build up an organism, the entire like,
03:12:00,080 --> 03:12:01,080 the arms, the legs.
03:12:01,080 --> 03:12:02,080 How is it built?
03:12:02,080 --> 03:12:03,080 Yeah.
03:12:03,080 --> 03:12:04,080 The brain.
03:12:04,080 --> 03:12:07,040 So you don't buy a computer, you buy like a...
03:12:07,040 --> 03:12:10,240 You buy a seed, a diagram.
03:12:10,240 --> 03:12:16,240 And then you plant the computer and it builds itself in almost the same way and then does
03:12:16,240 --> 03:12:25,720 the computation and then eventually dies, it gets stale but gives birth to young computers
03:12:25,720 --> 03:12:30,520 more and more and gives them lessons but they figure stuff out on their own and over time
03:12:30,520 --> 03:12:32,760 it goes on that way.
03:12:32,760 --> 03:12:37,040 And those computers, when they go to college, try to figure out how to program and they
03:12:37,040 --> 03:12:39,240 built their own little computers.
03:12:39,240 --> 03:12:44,280 They're increasingly more intelligent, increasingly higher and higher levels of abstractions.
03:12:44,280 --> 03:12:50,560 Isn't it interesting that you sort of, you see the same thing appearing at different
03:12:50,560 --> 03:13:00,900 levels though because you have like cells that create new cells and eventually that
03:13:00,900 --> 03:13:08,800 builds a whole organism but then the animal or the plant or the human has its own mechanism
03:13:08,800 --> 03:13:17,880 of replication that is sort of connected in a very complicated way to the mechanism of
03:13:17,880 --> 03:13:19,680 replication of the cells.
03:13:19,680 --> 03:13:26,840 And then if you look inside the cell, if you see how DNA and proteins are connected then
03:13:26,840 --> 03:13:34,920 there is yet another completely different mechanism whereby proteins are mass produced
03:13:34,920 --> 03:13:42,680 using enzymes and a little bit of code from DNA and of course viruses break into it at
03:13:42,680 --> 03:13:44,520 that level.
03:13:44,520 --> 03:13:50,560 And while the mechanisms might be different, it seems like the nature of the mechanism
03:13:50,560 --> 03:13:59,680 is the same and it carries across natural languages and programming languages, humans,
03:13:59,680 --> 03:14:04,840 maybe even human civilizations or intelligent civilizations and then all the way down to
03:14:04,840 --> 03:14:07,840 the single cell organisms.
03:14:07,840 --> 03:14:16,080 It is fascinating to see what abstraction levels are built on top of individual humans
03:14:16,080 --> 03:14:25,400 and how you have like whole societies that sort of have a similar self-preservation,
03:14:25,400 --> 03:14:32,760 I don't know what it is, instinct, nature, abstraction as the individuals have and the
03:14:32,760 --> 03:14:33,760 cells have.
03:14:33,760 --> 03:14:37,280 And they self-replicate and breed in different ways.
03:14:37,280 --> 03:14:41,440 It's hard for us humans to introspect it because we were very focused on our particular layer
03:14:41,440 --> 03:14:43,160 of abstraction.
03:14:43,160 --> 03:14:49,360 But from an alien perspective, looking on earth, they'll probably see the higher level
03:14:49,360 --> 03:14:55,560 organism of human civilization as part of this bigger organism of life on earth itself.
03:14:55,560 --> 03:15:01,440 In fact, that could be an organism just alone, just life, life, life on earth.
03:15:01,440 --> 03:15:07,840 This has been a wild, both philosophical and technical conversation, Guido, you're an amazing
03:15:07,840 --> 03:15:08,840 human being.
03:15:08,840 --> 03:15:13,480 You were gracious enough to talk to me when I was first doing this podcast, you're one
03:15:13,480 --> 03:15:18,440 of the earliest, first people I've talked to, somebody I admired for a long time.
03:15:18,440 --> 03:15:21,800 It's just a huge honor that you did it at that time and you do it again.
03:15:21,800 --> 03:15:22,800 You're awesome.
03:15:22,800 --> 03:15:24,160 Thank you, Lex.
03:15:24,160 --> 03:15:27,080 Thanks for listening to this conversation with Guido Van Rossum.
03:15:27,080 --> 03:15:32,200 To support this podcast, please check out our sponsors in the description and now let
03:15:32,200 --> 03:15:37,240 me leave you some words from Oscar Wilde, experience is the name that everyone gives
03:15:37,240 --> 03:15:39,280 to their mistakes.
03:15:39,280 --> 03:15:59,280 Thank you for listening and hope to see you next time.