Kind: captions
Language: en
the following is a conversation with
guido van rossum creator of Python one
of the most popular programming
languages in the world used in almost
any application that involves computers
from web back-end development to
psychology neuroscience computer vision
and robotics deep learning natural
language processing in almost any
subfield of AI this conversation is part
of MIT course on artificial general
intelligence and the artificial
intelligence podcast if you enjoy it
subscribe on YouTube iTunes or your
podcast provider of choice or simply
connect with me on Twitter at lex
friedman spelled FR ID and now here's my
conversation with guido van rossum you
were born in the Netherlands in 1956
your parents and the world around you
was deeply impacted by world war ii as
was my family from the soviet union so
with that context well what is your view
of human nature are some humans
inherently good and some inherently evil
or do we all have both good and evil
within us ouch I did not expect such a
deep one I I guess we all have good and
evil potential in us in a lot of it
depends on circumstances in context out
of that world at least on the Soviet
Union side in Europe sort of out of
suffering out of challenge out of that
kind of set of traumatic events often
emerges beautiful art music literature
in an interview I read or heard you said
you enjoy Dutch literature when when you
were a child can you tell me about the
books that had an influence on you in
your childhood
well as a teenager my favorite writer
was my favorite Dutch author was a guy
named villain Phaedra chemins whose
writing
certainly his early novels were all
about sort of ambiguous things that
happened during World War two I think he
was a young adult during that time and
he wrote about it a lot and and very
interesting very good books I thought I
think in a nonfiction way no it was all
fiction but it was very much set in in
the ambiguous world of resistance
against the Germans where often you
couldn't tell whether someone was truly
in the resistance or really a spy for
the Germans and and some of the
characters in his novels sort of crossed
that line and you never really find out
what exactly happened and in his novels
there's always a good guy and a bad guy
the nature of good and evil is it clear
there's a hero it's no his heroes are
often more his main characters are often
anti-heroes and and and so there they're
not not very heroic they're they're
often they they fail at some level to
accomplish their lofty goals and looking
at the trajectory through the rest of
your life has literature Dutch or
English or translation and an impact
outside the technical world that you
existed in
I still read novels I don't think that
it impacts me that much directly doesn't
impact your work it's just it's uh it's
a separate world my work is is highly
technical and sort of the the world of
art and literature doesn't really
directly have any bearing on it you
don't think there's a creative element
to the design you know some would say
our design of a language is art I'm not
disagreeing with that I'm just saying
that sort of I don't feel direct
influences from more traditional art on
my own creativity right of course you
don't feel doesn't mean it's not somehow
deeply there and your subconscious knows
who knows so let's go back to your early
teens your hobbies were building
electronic circuits building mechanical
models what if you could just put
yourself back in the mind of that young
Guido 12 13 14 was that grounded in a
desire to create a system so to create
something or was it more just tinkering
just the joy of puzzle solving uh I
think it was more the leather actually I
maybe towards the end of my high school
period I felt confident enough that that
I designed my own circuits that were
sort of interesting somewhat
but a lot of that time I literally just
took a model kit and follow the
instructions putting the things together
I mean that I think the first few years
that I build electronics kits I really
did not have enough understanding of
sort of electronics to really understand
what I was doing I mean I could debug it
and I could sort of follow the
instructions very carefully which has
had which has always stayed with me but
I had a very naive model of like how a
transistor works and I don't think that
that in those days I had any
understanding of coils and capacitors
which which actually sort of was a major
problem when I started to build more
complex digital circuits because I was
unaware of the sort of the analog part
of the how they actually work and I
would have things that the scheme the
schematic looked every everything looked
fine and it didn't work and what I
didn't realize was that there was some
megahertz level oscillation that was
throwing the circuit off because I had a
sort of two wires were too close or the
switches were were kind of poorly built
but through that time I think it's
really interesting and instructive to
think about because as echoes of it are
in this time now so in the 1970s the
personal computer was being born
so did you sense in tinkering with these
circuits did you sense the encroaching
revolution and personal computing so if
at that point you're sick we will see
you down and ask you to predict the 80s
and the 90s do you think you would be
able to do so successfully to unroll
this the process that's no I had no clue
I I
remember I think in the summer after my
senior year or maybe it was the summer
after my junior year well at some point
I think when I was 18 I went on a trip
to the Math Olympiad in Eastern Europe
and there was like I was part of the
Dutch team and there were other nerdy
kids that sort of had different
experiences and one of them told me
about this amazing thing called a
computer and I had never heard that word
my own explorations in electronics were
sort of about very simple digital
circuits and I I had sort of I had the
idea that I somewhat understood how a
digital calculator worked hmm and so
there is maybe some echoes of computers
there but I didn't didn't I never made
that connection I didn't know that when
my parents were paying for magazine
subscriptions using punched cards that
there was something called a computer
that was involved that read those cards
and transferred the money between
accounts that was also not really
interested in those things it was only
when I went to university to study math
that I found out that they had a
computer and students were allowed to
use it and there were some you're
supposed to talk to that computer by
programming it what did that feel like
yeah that was the only thing you could
do with it
I think the computer wasn't really
connected to the real world the only
thing you could do was sort of you typed
your program on a bunch of punched cards
you gave the punched cards to the
operator and an hour later the operator
gave you back your printout and so all
you could do was write a program that
did something very abstract and I don't
even remember what my first forays into
programming were
but they were sort of doing simple math
exercises and just to learn how a
programming language worked did you
sense ok first year of college you see
this computer you're able to have a
program and it generates some output did
you start seeing the possibility of this
or was it a continuation of the
tinkering with circuits the did you
start to imagine that one the personal
computer but did you see it as something
that is a tool so got a tool like a word
processing tool maybe maybe for gaming
or something or did you start to imagine
that it could be you know going to the
world of robotics
like you you know the Franklin is that
picture that you could create an
artificial being there's like another
entity in front of you you did not say I
don't think I really saw it that way I
was really more interested in the
tinkering it's maybe not a sort of a
complete coincidence that I ended up
sort of creating a programming language
which is a tool for other programmers
I've always been very focused on the
sort of activity of programming itself
and not so much what happens with with
the program you write right I do
remember and I don't dream it maybe in
my second or third year probably my
second actually someone pointed out to
me that there was this thing called
Conway's Game of Life you're probably
familiar with it I think the seventies I
think yeah he came up with it so there
was a scientific American column by
someone who did a monthly column about
mathematical diversions
I'm also blanking out on the guy's name
it was it was very famous at the time
and I think up to the 90s or so and one
of his columns was about Conway's Game
of Life and he had some illustrations
and he wrote down all the rules
and sort of there was the suggestion
that this was philosophically
interesting that that was why Conway had
called it that and all I had was like
the two pages photocopy of that article
I didn't even remember where I got it
but it spoke to me and I remember
implementing a version of that game for
the batch computer we were using where I
had a whole Pascal program that sort of
read an initial situation from input and
read some numbers that that said do so
many generations and print every so many
generations and then out would come
pages and pages of sort of things kinds
of different kinds and yeah and I
remember much later I've done a similar
thing using Python but I'd sort of that
original version I wrote at the time I
found interesting because I combined it
with some trick I had learned during my
electronics hobbyists times I
essentially first on paper I designed a
simple circuit built out of logic gates
that took nine bits of input which is
the sort of the cell and its neighbors
and produced a new value for that cell
and it's like a combination of a half
adder and some other clipping you know
it's actually a full adder and so I had
worked that out and then I translated
that into a series of boolean operations
on Pascal integers where you could use
the integers as bitwise values and so I
could basically generate 60 bits of a
generation in in like eight instructions
or so
nice I was proud of that it's it's funny
that you mentioned so for people who
don't know Conway's Game of Life is a
there's it's a cellular automata whether
it's single compute units that kind of
look at their neighbors and figure out
what they look like in the next
generation based on the state of their
neighbors and this is deeply distributed
system that it in in concept at least
and then there's simple rules that all
of them follow and somehow out of this
simple rule when you step back and look
at what occurs it's it's beautiful
there's a emergent complexity even
though the underlying rules are simple
there's an emergent complexity now the
funny thing is you've implemented this
and the thing you're commenting on is
you're proud of a hack you did to make
it run efficiently when you're not
commenting on what like this is a
beautiful implementation you're not
commenting on the fact that there's an
emergent complexity that you've you've
you've coded a simple program and when
you step back and you print out those
following generation after generation
that's stuff that you may have not
predicted what happen is happening right
and there was that is that magic I mean
that's the magic that all of us feel
when we program when you when you create
a program and then you run it and
whether it's hello world or show
something on screen if there's a
graphical component for you seeing the
magic in the mechanism of creating that
I think I went back and forth as a
student we had an incredibly small
budget of computer time that we could
use it was actually measured I once got
in trouble with one of my professors
because I had overspent the department's
budget it's a different story but so I I
actually wanted the efficient
implementation because I also wanted to
explore what would happen with a larger
number of generations and a larger sort
of
size of the of the board and so once the
implementation was flawless I would feed
at different patterns and then I think
maybe there was a follow-up article
where there were patterns that that were
like gliders parents that repeated
themselves after a number of generations
but translated one or two positions to
the right or up or something like that
and there were I remember things like
glider guns well you can you can google
Conway's Game of Life is still of people
still go on and over it
for a reason because it's not really
well understood why I mean this is what
Stephen Wolfram is obsessed about yeah
okay so he's just the the we don't have
the mathematical tools to describe the
kind of complexity of the emerges in
these kinds of systems and the only way
to do is to run it I'm not convinced
that that it's sort of a problem that
lends itself to two classic mathematical
analysis no and so one one theory of how
you create an artificial intelligence or
artificial being is you kind of have to
send with a game of life you kind of
have to create a universe and let it run
that creating it from scratch in a
design way in the you know coding up a
Python program that creates a full
intelligence system may be quite
challenging that you might need to
create a universe just like the game of
life is
well you might have to experiment with a
lot of different universes before there
there is a set of rules that doesn't
essentially always just end up repeating
itself in in a trivial way yeah and
analyst Steve wolf from Stephen Wolfram
works with these simple rules says that
it's kind of surprising how quickly find
rules that create interesting things you
shouldn't be able to but somehow you do
and so maybe our universe is laden with
with rules that will create interesting
things that might not look like humans
but yeah you know emergent phenomena
that's interesting may not be as
difficult to create as we think
sure but let me sort of ask at that time
you know some of the world's least in
popular press was kind of captivated
perhaps at least in America by the idea
of artificial intelligence that that
these computers would be able to think
pretty soon and yeah that touch you at
all did that in science fiction or in
reality in uh in anyway I didn't really
start reading science fiction until much
much later
I think as a teenager I I read maybe one
bundle of science fiction stories was in
my background somewhere like in your
thoughts that sort of the using
computers to build something intelligent
always fell to me because I had I felt I
had so much understanding of what
actually goes on inside a computer I I
knew how many bits of memory it had and
how difficult it was to program and sort
of I didn't believe at all that that you
could just build something intelligent
out of that that that would really sort
of satisfy my definition of intelligence
I think the most the most influential
thing that I read in my early 20s was
girlish ABBA that was about
consciousness and that was a big
eye-opener in in some sense in what
sense oh so console yeah so on your own
brain did you do use did you at the time
or do you now see your own brain as a
computer or is there a total separation
of the way so yeah you're very
pragmatically practically know the
limits of memory the limits of this
sequential computing or weakly paralyzed
computing and you just know what we have
now and it's hard to see how it creates
but it's also easy to see it was in the
40s 50s 60s and now at least
similarities between the brain and our
computers oh yeah I mean I I totally
believe that brains are computers in
some sense I mean the rules they they
used to play by are pretty different
from the rules we we can sort of
implement in in our current hardware but
I don't believe in like
a separate thing that infuses us with
intelligence or consciousness or any of
that there's no soul I've been an
atheist probably from when I was 10
years old just by thinking a bit about
math and the universe and then well my
parents were atheists now I know that
you you you could be an atheist and
still believe that there is something
sort of about intelligence or
consciousness that cannot possibly
emerge from a fixed set of rules I am
NOT in that camp
I I totally see that sort of given how
many millions of years evolution took
its time DNA is is a particular machine
that that sort of encodes information
and an unlimited amount of information
in in chemical form and has figured out
a way to replicate itself I thought that
death was maybe it's 300 million years
ago but I thought it was closer to half
a billion years ago that that's sort of
originated and it hasn't really changed
that the sort of the structure of DNA
hasn't changed ever since that is like
our binary code that you're having
hardware I mean the basic programming
language hasn't changed but maybe the
programming itself of has lead it did it
sort of it it happened to be a set of
rules that was good enough to to sort of
develop endless variability and and sort
of the the idea of self-replicating
molecules competing with each other for
resources and and one type eventually
sort of always taking over that happened
before there were any fossils so we
don't know how that exactly happened
but I believe it it's it's clear that
that did happen and can you comment on
consciousness and how you see it because
I think we'll talk about programming
quite a bit we'll talk about you know
intelligence connecting to programming
fundamentally but consciousness
consciousness is this whole lot of other
thing do you think about it often as a
developer of a programming language and
and as a human those those are pretty
sort of separate topics my sort of my
line of work working with programming
does not involve anything that that goes
in the direction of developing
intelligence or consciousness but sort
of privately as an avid reader of
popular science writing I I have some
thoughts which which is mostly that I
don't actually believe that
consciousness is an all-or-nothing thing
I have a feeling that and and I forget
what I read that influenced this but I
feel that if you look at a cat or a dog
or a mouse they have some form of
intelligence if you look at a fish it
has some form of intelligence and that
evolution just took a long time but I
feel that the the sort of the evolution
of more and more intelligence that led
to to sort of the human form of
intelligence follow the evolution of the
senses especially the visual sense I
mean there is an enormous amount of
processing that's needed to interpret a
scene and humans are still better at
that than then computers
yeah and so and and and I have a feeling
that there is a sort of the reason that
that like mammals is in particular
developed the levels of consciousness
that they have and that eventually read
sort of informative going from
intelligence to to self-awareness in
consciousness has to do with sort of
being a robot that has very highly
developed senses as a lot of rich
sensory information coming in so the
it's a really interesting thought that
the that whatever that basic mechanism
of DNA
whatever that basic building blocks are
programming is you if you just add more
abilities more more high resolution
sensors more sensors you just keep
stacking those things on top that
there's basic programming in trying to
survive develops very interesting things
that start to us humans to appear like
intelligence and consciousness yeah so
in in as far as robots go I think that
the self-driving cars have the sort of
the greatest opportunity of developing
something like that
because when I Drive myself
I don't just pay attention to the rules
of the road I also look around and I get
clues from that oh this is a shopping
district oh here's an old lady crossing
the street oh here is someone carrying a
pile of mail there's a mailbox thatthat
should they're gonna cross the street to
reach that mailbox and I slowed down and
I don't even think about that yeah and
and so there is there's so much where
you turn your observations into an
understanding of what utter
consciousnesses are going to do or what
what utter systems in the world are
going to be oh that tree is gone at
fault
yeah I see sort of I see much more of
expect somehow that if anything is going
to become conscious it's going to be the
self-driving car and not the network of
a bazillion computers at in a Google or
Amazon data center that are all
networked together to to do whatever
they do so in that sense so you actually
have like is that's what I work in
autonomous vehicles you highlight a big
gap between what we currently can't do
and what we truly need to be able to do
to solve the problem under that
formulation and consciousness and
intelligence is something that basically
a system should have in order to
interact with us humans as opposed to
some kind of abstract notion of a
consciousness consciousness is something
that you need to have to be able to
empathize to be able to fear the
understand what the fear of death is all
these aspects that are important for
interaction with pedestrians you need to
be able to do basic computation based on
our human desires and flaws sort of yeah
if you if you look at the dog the dog
clearly knows I mean I'm not the dog out
on my brother I have friends who have
dogs the dogs clearly know what the
humans around them are going to do or
the least they have a model of what
those humans are going to do and they
learn the dot some dogs know when you're
going out and they want to go out with
you they're sad when you leave them
alone they cry they're afraid because
they were mistreated when they were
younger we we don't assign sort of
consciousness to dogs or at least not
not all that much but I also don't think
they have none of that so I think it's
it's consciousness and intelligence are
not all or nothing
the spectrum it's really interesting but
in returning to programming languages
and the way we think about building
these kinds of things about building
intelligence building consciousness
building artificial beings I think one
of the exciting ideas came in the 17th
century and with liveness Hobbes decart
where there's this feeling that you can
convert all thought all reasoning all
the thing that we find very special in
our brains you can convert all that into
logic you can formalize it form a
reasoning and then once you formalize
everything all of knowledge and you can
just calculate and that's what we're
doing with our brains is we're
calculating so there's this whole idea
that we that this is possible that this
we're aware of the concept of pattern
matching in the sense that we are aware
of it now add a sort of thought you they
they had discovered incredible bits of
mathematics like Newton's calculus and
they're sort of idealism they're they're
sort of extension of what they could do
with logic and math sort of went along
those lines and they thought there
there's like yeah logic there's there's
like a bunch of rules and a bunch of
input they didn't realize that how you
recognize a face is not just a bunch of
rules but it's a ton of data plus a
circuit that that sort of interprets the
visual clues and the context and
everything else and somehow can
massively parallel pattern match against
stored rules I mean but if I see you
tomorrow here in front of the drop box
office I might recognize you even if I'm
wearing a different shirt yeah but if I
if I see you tomorrow in a coffee shop
in Belmont
I might have no idea
that was you or on the beach or whatever
hey I make those mistakes myself all the
time I see someone that I only know s
like oh this person is a colleague of my
wife's yeah and then I see them at the
movies and I didn't recognize them but
do you see those you call it pattern
matching do you see that rules is unable
to encode that to you you everything you
see all the pieces of information you
look around this room I'm wearing a
black shirt I have a certain height I'm
a human all these you can there's
probably tens of thousands of facts you
pick up moment by moment about this
scene you take them for granted and you
accumulate aggregate them together to
understand the scene
you don't think all that could be
encoded to where at the end of the day
you can just put it all on the table and
calculate oh I don't know what that
means I mean yes in the sense that there
is no there there is no actual magic
there but there are enough layers of
abstraction from sort of from the facts
as they enter my eyes in my ears to the
understanding of the scene that that's I
don't think that that AI has really
covered enough of of that distance it's
like if you take a human body and you
realize it's built out of atoms well
that that is a uselessly reductionist
view right right the body is built out
of organs the organs are built out of
cells the cells are built out of
proteins the proteins are built out of
amino acids the amino acids are built
out of atoms and then you get to quantum
mechanics so that's a very pragmatic
view I mean obviously is an engineer I
agree with that kind of view but I also
you also have to consider the the with
the same harris view of well well
intelligence is just information
processing these just like you said you
take in sensory information you do some
stuff with it and
you come up with actions that are
intelligent that McGee makes it sound so
easy I don't know who Sam Harris is oh
let's philosopher so like this how
philosophers often think right and
essentially that's what the car was is
wait a minute if there is like you said
no magic
so you basically says it doesn't appear
like there is any magic but we know so
little about it that it might as well be
magic so just because we know that we're
made of atoms just because we know we're
made of organs the fact that we know
very little hot to get from the atoms to
organs in a way that's recreate able
means it that you shouldn't get too
excited just yet about the fact that you
figured out that we're made of atoms
right and and and the same about taking
facts as are our sensory organs take
them in and turning that into reasons
and actions that sort of there are a lot
of abstractions that we haven't quite
figured out how to how to deal with
those I mean I so sometimes I don't know
if I can go on a tangent or not I
dragged you back in sure so if I take a
simple program that parses say say have
a compiler it parses a program in a
sense the input routine of that compiler
of that parser is a sense a sensing
Oregon and it builds up a mighty
complicated internal representation of
the program it just saw it doesn't just
have a linear sequence of bytes
representing the text of the program
anymore it has an abstract syntax tree
and I don't know how many of your
viewers or listeners are familiar with
compiler technology but there's fewer
and fewer these days right that's also
true probably people want to take a
shortcut but they're sort of this
abstraction is a data structure that the
compiler then uses to produce outputs
that is relevant like a translation of
the program to machine code that can be
executed by by hardware and then the
data structure gets thrown away when a
fish or a fly sees sort of gets visual
impulses I'm sure it also builds up some
data structure and for the fly that may
be very minimal a fly may may have only
a few I mean in the case of a fly's
brain I could imagine that there are few
enough layers of abstraction that it's
not much more than when it's darker here
than it is here well I can sense motion
because a fly sort of responds when you
move your arm towards it so clearly it's
visual processing is intelligent well
not intelligent but it has an
abstraction for motion and we still have
similar things in in but much more
complicated in our brains I mean
otherwise you couldn't drive a car if
you if you couldn't sort if you didn't
have an incredibly good abstraction for
motion yeah in some sense the same
abstraction for motion is probably one
of the primary sources of our of
information for us we just know what to
do I think we know what to do with that
we've built up other abstractions on top
we've much more complicated data
structures based on that and we build
more persistent data structures sort of
after some processing some information
sort of gets stored in our memory pretty
much permanently and is available on
recall I mean there are some things that
you sort of you're conscious that you're
remembering it like you give me your
phone number
I well at my age I have to write it down
but I could imagine I could remember
those seven numbers or 10 10 digits
and reproduce them in a while if I sort
of repeat them to myself a few times so
that's a fairly conscious form of
memorization on the other hand how do I
recognize your face I have no idea my
brain has a whole bunch of specialized
hardware that knows how to recognize
faces I don't know how much of that is
sort of coded in our DNA and how much of
that is trained over and over between
the ages of 0 and 3 but but but somehow
our brains know how to do lots of things
like that that are useful in our
interactions with with other humans with
without really being conscious of how
it's done anymore right so where are
actual d-day lives we're operating at
the very highest level of abstraction
we're just not even conscious of all the
little details underlying it
there's compilers on top of sec Turtles
on top of turtles or Turtles all the way
down it's compilers all the way down but
that's essentially you see that there's
no magic that's what I what I was trying
to get at I think is with decart started
this whole train of saying that there's
no magic I mean there's always before
well then the cart also have the notion
though that the soul and the body were
were fundamentally separate yeah I think
you had to write in God in there for
political reasons so I don't actually
not historian but there's notions in
there that all of reasoning all of human
thought can be formalized I think that
continued in the 20th century with with
Russell and with with Gaydos
incompleteness theorem this debate of
what what what are the limits of the
things that could be formalized that's
where the touring machine came along and
this exciting idea I mean underlying a
lot of computing that you can do quite a
lot with a computer you can you can
encode a lot of the stuff we're talking
about in terms of recognizing faces and
so on theoretically in an algorithm they
can then run on a computer and in that
context I'd like to ask programming in a
philosophical way so what so what it
what does it mean to program a computer
so you said you write a Python program
or a compiled a C++ program that
compiles to somebody code it's forming
layers your your programming a layer of
abstraction is higher how do you see
programming in that context can it keep
getting higher and higher levels of
abstraction I think and at some point
the higher level of levels of
abstraction will not be called
programming and they will not resemble
what we we call programming at the
moment there will not be source code I
mean there will still be source code
sort of at a lower level of the machine
just like they're still molecules and
electrons and and sort of proteins in
our brains but and and so they're still
programming and and and system
administration and who knows what's
keeping to keep the machine running but
what the machine does is is a different
level of abstraction in a sense and as
far as I understand the way that for
last decade or more people have made
progress with things like facial
recognition or the self-driving cars is
all by endless endless amounts of
training data where at least as a
layperson and I feel myself totally as a
layperson in that field
it looks like the researchers who
publish the results don't necessarily
know exactly how how their algorithms
work and that I often get upset when I
sort of read a sort of a fluff piece
about Facebook in the newspaper or
social networks and they say well Albert
and that that's like a totally different
interpretation of the word algorithm
yeah because for me the way I was
trained or what I learned when I was
eight or ten years old an algorithm is a
set of rules that you completely
understand that can be mathematically
analyzed and and and you can prove
things you can like prove that Aires
Dawson E's sieve produces all prime
numbers and only prime numbers yes so I
don't know if you know how Andre
Carpathia's I'm afraid not
so he's a ahead of hey aya Tesla now but
his Stanford before and he has this
cheeky way of calling this concept
software 2.0 so let me disentangle that
for a second so the so kind of what
you're referring to is the traditional
traditional the the algorithm the
concept of an algo something that's
there is clear you can read it you
understand it you can prove its
functioning it's kind of software 1.0
and what software 2.0 is is exactly what
you described which is you have neural
networks which is a type of machine
learning that you feed a bunch of data
and that neural network learns to do a
function all you specifies the inputs
and the outputs you want and you can't
look inside you can't analyze it all you
can do is train this function to map the
inputs the outputs by giving a lot of
data in that sense programming becomes
getting a lot of cleaning getting a lot
of data that's what programming is in
this well that would be programming 2.0
2.0 to programming 2.0 I I wouldn't call
that programming it's just a different
activity just like building organs out
of cells is not called chemistry well so
let's just set that back and think sort
of more generally of course but you know
it's like as a parent teaching teaching
your kids things can be called
programming in that same sense that
that's
how program has been used you're
providing them data examples use cases
so imagine writing a function not by not
with for loops and clearly readable text
but more saying well here's a lot of
examples of what this function should
take and here's a lot of examples when
it takes those functions it should do
this and then figure out the rest so
that's the 2.0 concept and the this is
the question I have for you is like it's
a very fuzzy way this is a reality of a
lot of these pattern recognition systems
and so on
it's a fuzzy way of quote-unquote
programming what do you think about this
kind of world it should be called
something totally different than
programming it's like if you're a
software engineer does that mean you're
you're designing systems that are very
can be systematically tested evaluated
they have a very specific specification
and then this other fuzzy software 2.0
world machine learning world that's
that's something else totally or is
there some intermixing that it's
possible well the question is probably
only being asked because we we don't
quite know what that software 2.0
actually is and it sort of I think there
is a truism that every task that AI has
has tackled in the past at some point we
realized how it was done and then it was
no longer considered part of artificial
intelligence because it was no longer
necessary to to use that term it was
just oh now he we know how to do this
and a new field of science or
engineering has been developed and I
don't know if sort of every form of
learning or
sort of controlling computer systems
should always be called programming I
said I that I don't know maybe I'm
focused too much on the terminology i
but i expect that that there just will
be different concepts where people with
sort of different education and a
different model of what they're trying
to do will will develop those concepts
yeah and i guess if you could comment
and another way to put this concept is i
think i think the kind of functions that
neural networks provide is things as
opposed to being able to upfront prove
that this should work for all cases you
throw at it all you're able it's the
worst case analysis versus average case
analysis all you're able to say is it's
it seems on everything we've tested to
work 99.9 percent of the time but we
can't guarantee it and it it fails in
unexpected ways but can't even give you
examples of how it fails in unexpected
ways but it's like really good most of
the time yeah but there's no room for
that in current ways we think about
programming programming 1.0 is actually
sort of getting to that point to where
the sort of the ideal of a bug-free
program has been abandoned long ago by
most software developers we only care
about bugs that manifest themselves
often enough to be annoying and we're
willing to take the occasional crash or
outage or incorrect result for granted
because we can't possibly we don't have
enough programmers to make all the code
bug free and it would be an
credibly tedious business and if you try
to throw formal methods at it it gets it
becomes even more tedious so every once
in a while the user clicks on a link in
and somehow they get an error and the
average user doesn't panic they just
click again and see if it works better
the second time which often magically it
does or they go up and they try some
other way of performing their tasks so
that's sort of an end-to-end recovery
mechanism and inside systems there is
all sorts of retries and timeouts and
fall backs and I imagine that that sort
of biological systems are even more full
of that because otherwise they wouldn't
survive do you think programming should
be taught and thought of as exactly what
you just said before I come from is kind
of you're almost denying that fact
always in the insert of basic
programming education the sort of the
program's you're you're having students
right are so small and simple that if
there is a bug you can always find it
and fix it because the sort of
programming as it's being taught in some
even elementary middle schools in high
school introduction to programming
classes in college typically it's
programming in the small very few
classes sort of actually teach software
engineering building large systems I
mean every summer here at Dropbox we
have a large number of interns every
tech company on the west coast has the
same thing these interns are always
amazed because this is the first time in
their life
that they see what goes on in a really
large software development environment
and everything they've learned in
college was almost always about a much
smaller scale and somehow the difference
in scale
makes a qualitative difference in how
you how you do things and how you think
about it if you then take a few steps
back in two decades seventies and
eighties when you're first thinking
about Python or just that world of
programming languages did you ever think
that there would be systems as large as
underlying Google Facebook and Dropbox
did you when you were thinking about
Python I was actually always caught by
surprise by
yeah pretty much every stage of
computing so maybe just because uh you
spoken in other interviews but I think
the evolution of programming languages
are fascinating it's especially because
it leads from my perspective towards
greater and greater degrees of
intelligence
I learned the first programming language
I played with in in Russia was with the
turtle logo logo yeah and if you look I
just have a list of programming
languages all of which I've known played
with a little bit and they're all
beautiful in different ways from Fortran
COBOL Lisp Algol 60 basic logo and C as
a few the object-oriented came along in
the 60s Simula Pascal small talk all of
that lean all the classics the classics
yeah the classic hits write scheme built
that's built on top of Lisp on the
database side SQL C++ and all that leads
up to Python Pascal - and that's before
Python MATLAB these kind of different
communities different languages so he
talked about that world I know that
Python came out of ABC which actually
never knew that language I just having
researched this conversation went back
to ABC and it looks remarkably it it has
a lot of annoying qualities but
underneath those like all caps and so on
but underneath that there's elements of
Python that are quite if they're already
there that's where I got all the good
stuff
all the good stuff so but in that world
you're swimming these programming
languages were you focused on just the
good stuff in your specific circle but
did you have a sense of what what is
everyone chasing you said that every
programming language is built to scratch
an itch mm-hmm
were you aware of all the itches in the
community and if not or if yes I mean
what H we trying to scratch with Python
well I'm glad I wasn't aware of all the
itches because I would probably not have
been able to do anything I mean if
you're trying to solve every problem at
once you saw nothing well yeah that it's
it's too overwhelming and so I had a
very very focused problem I wanted a
programming language that set somewhere
in between shell scripting and C and now
arguably there is like one is higher
level one is lower level and Python is
sort of a language of an intermediate
level although it's still pretty much at
the high level and no I was I was
thinking about much more about I want a
tool that I can use to be more
productive as a programmer in a very
specific environment and I also had
given myself a time budget for the
development of the tool and that was
sort of about three months for both the
design like thinking through
what are all the features of the
language syntactically and semantically
and how do i implement the whole
pipeline from parsing the source code to
executing it so I think both were the
timeline and the goals it seems like
productivity was at the core of it as a
goal so like for me in the 90s and the
first decade of the 21st century I was
always doing machine learning AI
programming for my research was always
in C++ and then and then the other
people who are a little more mechanical
engineering Electrical Engineering our
MATLAB II they're a little bit more
MATLAB focus those are the world and
maybe a little bit Java too but people
who are more interested in and
emphasizing the object oriented nature
of things so but then in last 10 years
or so especially with a calming of
neural networks and these packages are
built on Python to interface with with
neural networks I switch to Python and
it's just I've noticed a significant
boost that I can't exactly because I
don't think about it but I can't exactly
put into words why I'm just except much
much more productive just being able to
get the job done much much faster so how
do you think whatever that qualitative
difference is I don't know if it's
quantitative it could be just a feeling
I don't know if I'm actually more
productive but how do you think about
Layar yeah well that that's right I
think there's elements let me just speak
to one aspect that I think those affect
that productivity is C++ was I really
enjoyed creating performant code and
creating a beautiful structure where
everything that you know this kind of
going into this especially with the
newer newer standards of templated
programming of just really creating this
beautiful formal structure that I found
myself spending most of my time doing
that as opposed to get
you parsing a file and extracting a few
key words or whatever the task was
trying to do so what is it about Python
how do you think of productivity in
general as you were designing it now
sort of through the decades last three
decades what do you think it means to be
a productive programmer and how did you
try to design it into the language there
are different tasks and as a programmer
it's it's useful to have different tools
available that sort of are suitable for
different tasks so I still write C code
I still write shellcode but I write most
of my things in Python why do I still
use those other languages because
sometimes the task just demands it and
well I would say most of the time the
task actually demands a certain language
because the task is not write a program
that solves problem x from scratch but
it's more like fix bug in existing
program X or add a small feature to an
existing large program but even if if
you sort of if you're not constrained in
your choice of language by context like
that there is still the fact that if you
write it in a certain language then you
sort of you you have this balance
between how long does it time does it
take you to write the code and how long
does the code run and when you're in
sort of in the face of exploring
solutions you often spend much more time
writing the code than running it because
every time you've sort of you've run it
you see that the output is not quite
what you wanted and you spend some more
time Cody
and a language like Python just makes
death iteration much faster because
there are fewer details there is a large
library sort of there are fewer details
that that you have to get right before
your program compiles and runs there are
libraries that do all sorts of stuff for
you so you can sort of very quickly take
a bunch of existing components put them
together and get your prototype
application running just like when I was
building electronics I was using a
breadboard most of the time so I had
this like sprawl out circuit that if you
shook it it would stop working because
it was not put together very well but it
functioned and all I wanted was to see
that it worked and then move on to the
next next schematic or design or add
something to it once you've sort of
figured out oh this is the perfect
design for my radio or light sensor or
whatever then you can say okay how do we
design a PCB for this how do we solder
the components in a small space how do
we make it so that it is robust against
say voltage fluctuations or mechanical
disruption I mean I know nothing about
that when it comes to designing
electronics but I know a lot about that
when it comes to to writing code so the
initial initial steps are efficient fast
and there's not much stuff that gets in
the way but you're kind of describing
from a like Darwin described the
evolution of species right you're you're
observing of what is about true about
Python now if you take step back if the
art of if the act of creating languages
is art and you had three months to do it
and initial steps and ha so you just
specified a bunch of goals sort of
things that you observe about Python
perhaps you had those goals but how do
you create the rules the syntactic
structure the the features that result
in those so I have in the beginning and
I have follow-up questions about through
the evolution of Python 2 but in the
very beginning when you're sitting there
creating the lexical analyzers or
whatever evolution was still a big part
of it because I I sort of I said to
myself I don't want to have to design
everything from scratch I'm going to
borrow features from other languages
that I like Oh interesting so you
basically exactly you first observe what
you like yeah and so that's why if
you're 17 years old and you want to sort
of create a programming language you're
not going to be very successful at it
because you have no experience with
other languages whereas I was in my
let's say mid-30s I had written parsers
before so I had worked on the
implementation of ABC I had spent years
debating the design of ABC with its
authors its with its designers I had
nothing to do with the design it was
designed fully as it was ended up being
implemented when I joined the team but
so you borrow ideas and concepts and
very concrete sort of local rules from
different languages like the indentation
and certain other syntactic features
from ABC but I chose to borrow string
literals and how numbers work from C and
various other things so in then if you
take that further so yet you've had this
funny sounding but I think surprisingly
accurate and or at least practical title
of a benevolent dictator for life for
quite you know for last three decades
whatever or no not the actual title but
functionally speaking so you had to make
decisions design decisions can you maybe
let's take Python - there's a Python
releasing Python 3 as an example mm-hmm
it's not backward-compatible - Python -
in ways that a lot of people know so
what was that deliberation discussion
decision like we have what was the
psychology of that experience
do you regret any aspects of how that
experiments undergone that else yeah so
it was a group process really it at that
point even though I was be DFL in nine a
name and and certainly everybody sort of
respected my my position as the creator
and and the current sort of owner of the
language design I was looking at
everyone else for feedback sort of
Python 300 in some sense was sparked by
other people in the community pointing
out oh well there are a few issues that
sort of bite users over and over can we
do something about that and for Python
three we took a number of those Python
wards as they were called at the time
and we said can we try to sort of make
small changes to the language that
address those warts and we had sort of
in the past we had always taken
backwards compatibility very seriously
and so many Python warts in earlier
versions had already been resolved
because they could be resolved while
maintaining backwards compatibility or
sort of using a very gradual path of
evolution of the language in a certain
area and so we were stuck with a number
of warts that were
widely recognized as problems not like
road blocks but nevertheless sort of
things that some people trip over and
you know that that's always the same
thing that that people trip over when
they trip and we could not think of a
backwards compatible way of resolving
those issues but it's still an option to
not resolve the issues and so yes for
for a long time we had sort of resigned
ourselves to well okay the language is
not going to be perfect in this way and
that way that way and we sort of certain
of these I mean there are still plenty