Kind: captions
Language: en
the following is a conversation with
charles isbell and michael littman
charles is the dean of the college of
competing at georgia tech
and michael is a computer science
professor at brown university
i've spoken with each of them
individually on this podcast
and since they are good friends in real
life we all thought it would be fun
to have a conversation together quick
mention of each sponsor
followed by some thoughts related to the
episode thank you to
athletic greens the all-in-one drink
that i start
every day with to cover all my
nutritional bases
ate sleep a mattress that cools itself
and gives me yet another reason to enjoy
sleep master class online courses from
some of the most
amazing humans in history and cash app
the app i use to send money
to friends please check out the sponsors
in the description to get a discount
and to support this podcast as a side
note let me say that
having two guests on the podcast is an
experiment that i've been meaning to do
for a while
in particular because down the road i
would like to occasionally
be a kind of moderator for debates
between
people that may disagree in some
interesting ways if you have suggestions
for who you would like to see debate
on this podcast let me know as with all
experiments of this kind it is a
learning process
both the video and the audio might need
improvement
i realized i think i should probably do
three or more cameras next time as
opposed to just two
and also try different ways to mount the
microphone for the
third person also after recording this
intro
i'm going to have to go figure out the
thumbnail
for the video version of the podcast
since i usually put the guest's
head on the thumbnail and now there's
two heads
and two names to try to fit into the
thumbnail it's a kind of bin packing
problem
which in uh theoretical computer science
happens to be an np hard problem
whatever i come up with if you have
better ideas for the thumbnail let me
know as well
and in general i always welcome ideas
how this thing can be improved
if you enjoy it subscribe on youtube
review it with five stars on apple
podcast
follow on spotify support on patreon or
connect with me on twitter
and lex friedman and now here's my
conversation
with charles isbell and michael littman
you'll probably disagree about this
question
but what is your biggest would you say
disagreement
about either something profound and very
important or something completely not
important at all
i don't think we have any disagreements
at all ah i'm not sure that's true
we walked into that one didn't we yeah
so one thing that you sometimes mention
is that and we did this one on air too
as it were
whether or not machine learning is
computational statistics
it's not but it is well it's not
and in particular and more importantly
it is not just computational statistics
so what's missing in the picture what
all the rest of it
what's missing that which is missing oh
because well you can't be wrong now well
it's not just the statistics he doesn't
even believe this
we've had this conversation before if it
were just the statistics then
we would be happy with where we are but
it's not just the statistics that's why
it's computational statistics or if it
were just the computation i agree that
machine learning is not just statistics
it is not just this we can agree on that
nor is it just computational statistics
it's computational statistics it is
computational what is the computational
and computational statistics
does this take us into the realm of
computing it does but i think
perhaps the way i can get him to admit
that uh he's wrong
is that it's about rules it's about
rules
it's about symbols it's about all these
other things statistics it's not about
rules i'm going to say statistics is
about rules but it's not just the
statistics right it's not just a random
variable that you choose and you have a
probability i think you have a narrow
view of statistics
okay well then what would be the broad
view of statistics that would still
allow it to be statistics and not say
history that would make computational
statistics okay
well okay so i i had my first sort of
research mentor
a guy named tom landauer taught me to do
some statistics
right sure and and i was annoyed all the
time because the statistics would say
that what i was doing was not
statistically significant
and i was like but but but and basically
what he said to me is
statistics is how you're going to keep
from lying to yourself
which i thought was really deep it is a
way
to keep yourself honest in a particular
way i agree with that
yeah and so you're trying to find rules
i'm just kind of bringing back to rules
wait wait wait could you possibly try to
define
rules even regular statisticians
non-computational statisticians
do spend some of their time evaluating
rules right applying statistics to try
to understand is this you know is this
does this rule capture this does this
not capture
i mean like hypothesis testing kind of
or like confidence intervals like
like have like more like hypothesis like
i feel like the word statistic literally
means like a summary
like a number that summarizes other
numbers right but i think the field of
statistics actually applies that idea
to things like rules to understand
whether or not a rule is
valid the software engineering
statistics
no programming languages statistics no
because i think there's a very it's
useful
to think about a lot of what ai and
machine learning is or certainly should
be
as software engineering uh as
programming languages
just if to put it in language that you
might understand
in the hyper parameters beyond the
problem the hyper parameters has too
many syllables for me to understand
the hyperparameters of uh that's better
that goes around it right it's the
decisions you choose to make it's the
metrics you choose to use it's the loss
you want to say the practice
of machine learning is different than
the practice of statistics like the
things you have to worry about and how
you worry about them are different
therefore they're different right at a
very little i mean at the very least
it's that that much is true it doesn't
mean that statistics computational or
otherwise aren't important
i think they are i mean i do a lot of
that for example
but i think it goes beyond and i think
that we could think about game theory in
terms of statistics but i don't think
it's very
as useful to do i mean the way i would
think about it or a way i would think
about it
is this way chemistry is just
physics but i don't think it's as useful
to think about chemistry as being just
physics it's useful to think about it as
chemistry the level of abstraction
really matters here so i think it is
there are contexts in which it is useful
that way right so finding that
connection is actually helpful and i
think that's when i when i emphasize the
computational statistics thing i think
i think i want to befriend statistics
and not absorb them here's the here's
the a way to think about it beyond what
i just said right so
what would you say and i want you to
think back to a conversation we had a
very long time ago what would you say is
the difference between
say the early 2000's icml and what we
used to call nips nerfs
was there a difference a lot of the
particularly on the machine learning
that was done there
icmo was around that long oh yeah so
iclear is the new conference
newish uh yeah i guess so and i see him
i was
around the 2000 oh i see male predates
that
i well i think my most cited icml paper
is from 94. yeah
michael knows this better than me
because of course he's significantly
older than i am but the point is
yeah what is the difference what was the
difference between icml and nureps
in the late 90s early 2000s i don't know
what everyone else's perspective would
be but i had a particular perspective at
that time
which is i felt like icml was more of a
of a computer science
place and that nips nerfs was more of an
engineering place
like the kind of math that happened at
the two places
as a computer scientist i felt more
comfortable with the icml math
and the nurbs people would say that
that's because i'm dumb
and that's such an engineering thing to
say so i agree with that part of it but
i
do a little differently actually i had a
nice conversation with tom dietrich
about this
in public on twitter just a couple days
ago i put it a little differently which
is that icml was
machine learning done by uh computer
scientists
and uh nurbs was machine learning done
by computer scientists
trying to impress statisticians
which was weird because it's the same
people at least by the time i started
paying attention
but it just felt very very different and
i think that that perspective of whether
you're trying to impress the
statisticians or you're trying to
impress the programmers is actually very
different and has real impact on
what yeah what you choose to worry about
and what kind of uh
outcomes you come to so i think it
really matters in computational
statistics is a means to an end it is
not an end in some sense
um and i think that really matters here
in the same way that i don't think
computer science is just engineering or
just science or just math or whatever
but
okay so i'd have to now agree that now
we agree on everything
yes yes the important thing here is that
you know my opinions may have changed
but not the fact that i'm right
i think is what what we just came to
right now my opinions may have changed
and not the fact that i'm wrong
that's right i lost me i'm not i think i
lost myself there too but anyway
this happens to us sometimes we're sorry
how does neural networks change
this just to even linger on this topic
change this idea of statistics
how big of a pi statistics is within the
machine learning thing
like because it sounds like hyper
parameters and also just the role of
data
you know this people are starting to use
this terminology software 2.0
which is like the act of programming as
a as a
like you're a designer in the
hyperparameter space of neural networks
and you're also the collector and the
organizer and the cleaner
of the data and that's part of the
programming
uh so how did on the
versus icml topic what's the role of
neural networks and redefining
the size and the role of machine
learning i can't i can't wait to
hear what michael thinks about this but
um i would add one well but
that's true i'll force myself to i think
the the
there's one thing i would add to your
description which is the kind of
software engineering part is what does
it mean to debug for example
but this is a difference between uh the
kind of computational statistics view
of machine learning and the
computational view of machine learning
which is i think one is worried about
the equation as it were and
by the way this is not a value judgment
i just think it's about
perspective but the kind of questions
you would ask when you start asking
yourself what does it mean to program
and develop and build the system
it's a very computer sciencey view of
the problem
i mean when if you get on data science
twitter and econ twitter
you actually hear this a lot with the
uh you know the economist and the data
scientist complaining about the machine
learning people well
you know it's just statistics and i
don't know why they don't don't see this
but they're not even asking the same
questions they're not
thinking about it as a kind of
programming problem
and i think that that really matters
just asking this question i actually
think it's a little different from
programming and hyper parameter space
and sort of collecting
the data but i do think that that
immersion really matters so i'll give
you a quick
a quick example the way i think about
this so i teach machine learning michael
and i have
co-taught a machine learning class which
has now reached i don't know 10 000
people at least over the last several
years or somewhere there's abouts and
my machine learning assignments are of
this form so the super
the first one is something like
implement these five algorithms you know
k
n and s you know svms and boosting and
decision trees and
neural networks and maybe that's it i
can't remember and when i say implement
i mean steal the code
i am completely uninterested you get
zero points for getting the thing to
work
i don't want you spending your time
worrying about uh getting the corner
case right
of you know what happens when you are
trying to normalize distances and the
points on the thing and so you divide by
zero i'm not interested in that right
steal the code however you're going to
run those algorithms
on two data sets the data sets have to
be interesting
what does it mean to be interesting well
data says interesting if it reveals
differences between algorithms which
presumably are all the same
because they can represent whatever they
can represent and two data sets are
interesting together if they
show different differences as it were
and you have to analyze them you have to
justify their interestingness and you
have to analyze in a whole bunch of ways
but all i care about is the data in your
analysis not the programming and i
occasionally end up in these long
discussions with students well
i don't really i copy and paste the
things that i've said the other 15
000 times it's come up which is they go
but the only way to learn
really understand is to code them up
which is a very
programmer software engineering view of
the world if you don't program it you
don't understand it
which is by the way i think is wrong in
a very specific way
but it is a way that you come to
understand because then you have to
wrestle with the algorithm
but the thing about machine learning is
it's not just sorting numbers where in
some sense the data doesn't matter what
matters is
well does algorithm work on these
abstract things and one less than the
other in machine learning
the data matters it does it matters more
than almost anything
and not everything but almost anything
and so
as a result you have to live with the
data and don't get distracted by
the algorithm per se and i think that
that focus on the data
and what it can tell you and what
question it's actually answering for you
as opposed to the question you thought
you were asking is a key and important
thing
about machine learning and is a way that
computationalists as opposed to
statisticians bring a particular
view about how to think about the
process the statisticians by contrast
bring
i i think i'd be willing to say a better
view about the kind of formal
math that's behind it and what an actual
number
ultimately is saying about the data and
those are both important but they're
also different
i didn't really think of it this way is
to build intuition about
the role of data the different
characteristics of data by having two
data sets that are different
and they reveal the differences in the
differences that's that's a really
fascinating that's a really interesting
educational
approach the students love it but not
right away
no they love it later i love it at the
end not at the beginning
not even not even immediately after i
feel like there's a deep
profound lesson about education there
yeah
that uh you can't listen to students
about
whether what you're doing is the right
or the wrong thing
well as a wise uh michael litman once
said to me
about children which i think applies to
teaching is you have to give them
what they need without bending to their
will
and students are like that you have to
figure out what they need you're a
curator your whole job is to curate
and to present because on their own
they're not going to necessarily know
where to search so you're providing
pushes in some direction and learn space
and you have to give them what they need
in a way that keeps them engaged enough
so that they eventually discover
what they want and they get the tools
they need to go and learn other things
what's your view let me put on my
russian hat
which believes that life is like russian
hats by the way if you have one i would
like
those are ridiculous yes but in a
delightful way but sure
what do you think is the role of uh we
talked about balance a little bit
what do you think is the role of
hardship in education
like i think the biggest things i've
learned
like what made me fall in love with math
for example
is by being bad at it until i got good
at it
so like like struggling with a problem
which increased the level of joy i felt
when i finally figured it out
and it always felt with me with teachers
especially modern discussions of
education how can we make
education more fun more engaging more
all those things
or from my perspective it's like you're
maybe missing the point
that education that life is suffering
education is supposed to be hard and
that actually what
increases the joy you feel when you
actually learn something
is that ridiculous do you like to see
your students suffer
okay so this may be a point where we
differ i'd suspect not
i'm gonna do go on well what would your
answer be i wanna hear you first
okay well i would i was gonna not answer
the question
do you know what this dude is i wasn't
gonna hear them suffering no no no no no
i was i was gonna say that there's
i think there's a distinction that you
can make in the kind of suffering right
so
i think you can be in a mode where
you're you're suffering
in a hopeless way versus you're
suffering in a hopeful way
right where you're like you can see
that if you that you still have you can
still imagine getting
to the end right and as long as people
are in that mindset where they're
struggling but
it's not a hopeless kind of struggling
that's
that's productive i think that's really
helpful but it's struggling like if you
break their will
if you leave them hopeless no that don't
sure some people are gonna whatever lift
themselves up by their bootstraps but
like
mostly you give up and certainly it
takes the joy out of it and
you're not going to spend a lot of time
on something that brings you no joy
so it's it's it is a bit of a delicate
balance right you have to thwart people
in a way that they still believe that
there's a way through
right so that's a that we strongly agree
actually so i think
well first off struggling and suffering
aren't the same thing right
being poetic oh no no i actually
appreciate the poetry and
i one of the reasons i appreciate it is
that they are often the same thing and
often quite different right so
you can struggle without suffering you
can certainly suffer and suffer
suffer pretty easily you don't
necessarily have to struggle to suffer
so i think that
you want people to struggle but that
hope matters
you have to they have to understand that
they're going to get through it on the
other side
and it's very easy to confuse the two
i actually think brown university has a
very just
philosophically has a very different
take on the relationship with their
students particularly undergrads from
say
a place like georgia tech which is which
universities better
uh well i have my opinions on that i
mean remember charles said
it doesn't matter what the facts are i'm
always right the correct thing
is that it doesn't matter they're
different um but
clearly he went to a school
like the school where he is as an
undergrad i went to a school
specifically the same school though it
was it changed a bit in the in the
intervening years
brown or georgia tech no i was talking
about georgia tech and i went yeah
and i went to an undergrad place that's
a lot like the place where i work now
and so it does seem
like we're more familiar with these
models there's a similarity between
brown and yellow
yeah there's a i think that i think
they're quite similar yeah and duke
duke has some similarities too but it's
got a little southern
draw you've kind of worked here you sort
of worked at universities that are like
the places where
you learned and
the same would be true for me are you
uncomfortable uh
venturing outside the box is that what
you're saying
journeying out what i'm saying yeah
charles is definitely he only goes to
places that have institute in the name
right it has worked out that way well
academic places anyway
well no i was a visiting scientist at
upenn or visiting
visiting something at upenn oh wow i
just i just understood
your joke which one
five minutes later i like to set these
sort of time bombs
the institute is in the uh uh that
charles only goes to places that have
institute
in the name so i guess georgia
i forget that georgia tech is georgia
institute of technology the number of
people who refer to it as georgia tech
university is large and incredibly
irritating
that's one of the few things that
generally gets under my schedule but
like schools like georgia tech and mit
have as part of the ethos like there is
i want to say there's a there's an
abbreviation
that someone taught me like i htfp
something like that like there's a
there's a
there's an expression which is basically
i hate being here which they say
so proudly and that is definitely not
the ethos at brown like brown is
there's a little more pampering and
empowerment and stuff and it's not like
we're gonna crush you and you're gonna
love it
so yeah i think there's a i think the
ethos
are different mm-hmm that's interesting
yeah we had drone proofing
what's that trump graduate from georgia
tech this is a true thing feel free to
look it up
uh if you a lot of schools have this by
the way
no actually georgette was barely the
first brandeis has it had it
i feel like georgia tech was the first
in the look first of all
it was it was the first time i think um
had the first time
stop that first masters in computer
science actually right online masters
well that too but way back in the 60s um
nsf yeah yeah
you're the first information and
computer science master's degree in the
country
um but the uh georgia tech it used to be
the case in order to graduate from
georgia tech uh you had to take a drown
proofing class
where effectively they threw you water
tied you up
if you didn't drown you got to graduate
i believe so
there were certainly versions of it but
i mean luckily they ended it just before
i had to graduate because otherwise
would have never graduated
it wasn't going to happen uh i want to
say 84 or 83 someone around then they
they ended it but uh yeah you used to
have to prove you could tread water for
some ridiculous amount of time are you
two yeah you couldn't graduate no it was
more than two hours two minutes
okay it was in a bathtub
it was in a pool but it was a real thing
but that idea that you know push you
fully clothed yeah fully clothed i don't
think i bet it was that and not tied up
because like who needs to learn how to
swim when you're tied
nobody but who needs to learn when to
swim when you're actually falling into
the water dressed that's a real thing
i think your facts are getting in the
way with a good story oh that's fair
that's fair i didn't
think all right so they didn't tell you
what the narrative mattered but whatever
it was you had to it was called drown
proofing for a reason the point of the
story
michael uh is struggle it it's well no
but that's good it doesn't
bring it back to struggle that's a part
of what georgia tech has always been and
we struggle with that by the way
uh about what we want to be as things go
but you you sort of how much can you
be pushed without breaking and you come
out of the other end stronger right
there
there's this saying we said when i was
an undergrad there which is georgia tech
building tomorrow the night before
right kind of idea that you know
give me something impossible to do and
i'll do it in a couple of days because
that's what i just spent the last four
or five
or six years that ethos definitely stuck
to you
having now done a number of projects
with you you definitely will do it the
night before that's not entirely true
there's nothing wrong with waiting until
the last minute the secret is knowing
when the last minute is
right that's brilliant that's
brilliantly put yeah that yeah that's
that is a definite
charles statement that i am trying not
to embrace
and i appreciate that because you helped
move my last minute that's the social
construct that we converge together what
the definition of last minute is and we
we figure that out all together in fact
mit
you know i'm sure a lot of universities
have this but mit has like mit time that
yeah everyone has always agreed together
that
there is such a concept and everyone
just keeps showing up like 10 to 15
to 20 depending on the department late
to everything
so there's like a weird drift that
happens it's kind of fascinating yeah
we're five minutes
five minutes in fact the classes will
say you know well this is no longer true
actually
but it used to be a class was started
eight but actually started 805
yeah it ends at nine actually ends at 8
55. uh everything's five minutes off and
nobody expects anything to start until
five minutes after the half hour or
whatever it is
it still exists it hurts my head well
let's rewind the clock
back to the 50s and 60s when you guys
met
how did you i'm just kidding i don't
know but what can you tell the story of
how you met so
you've like the internet and the world
kind of knows you as
as as connected in some ways
in terms of education of teaching the
world that's
that's like the public facing thing but
how did you as human beings
and as collaborators
meet i think there's two stories one is
how we met
and the other is how we got to know each
other
i'm not gonna say fellaini i'm gonna say
that we came to understand that we
had some common something yeah it's
funny because on the surface i think
we're
we're different in a lot of ways but
there's something yeah i mean that's
just consonant there you go afternoon
so i will tell the story of how we met
and i'll let michael tell the story of
how we
okay all right okay so here's how we met
um i was already at that point it was
18t labs
there's a long interesting story there
but anyway i was there and uh
michael was coming to interview he was a
professor at duke at the time but
decided for reasons that he wanted to be
in new jersey
uh and so that would mean uh bell lab
slash att labs
uh and we were doing interview
interviews very much like academic
interviews uh and so i had to be there
uh we all had to meet with him
afterwards and so on one on one
but it was obvious to me that he was
gonna be hired
like no matter what because everyone
loved him they were just talking about
all the great stuff he did and
oh he did this great thing and you just
won something at triple a i think or
maybe you got 18 papers in triple either
but
i got the best paper award at your play
for the crosswords right exactly
so that it all happened and everyone was
going on and on and on about actually
tinder was saying incredibly nice things
about you really yes so he can be very
grumpy yes that's very that's nice to
hear he was grumpily saying very nice
things oh that's that makes sense and
that does make sense so you know so
it was going to come so why were we why
was i meeting him i had something else i
had to do i came here what it was yeah
it probably involved commenting he
remembers meeting me as inconveniencing
his afternoon
so he came so eventually came to my
office i was in the middle trying to do
something i can't remember what and he
came and he sat down and for
reasons that are purely accidental
despite what michael thinks
my desk at the time was set up in such a
way that had
sort of an l shape and the chair on the
outside was always lower than the chair
that i was in
and you know the kind of point was the
only reason i think that was on purpose
is because you told me it was on purpose
i don't remember that anyway the thing
is that you know it kind of his guest
chair was really low so that he could
yeah he could look down at everybody the
idea was just to simply create a nice
environment that you were asking for a
mortgage and i was going to say no that
was a
very simple idea here anyway so we sat
there and we just talked for a little
while and i think he got the impression
that i didn't like him
that wasn't true strongly the talk was
really good
by the way it was terrible and after
right after the talk i said to my host
michael kearns who ultimately was
my boss i'm a huge fan i'm a friend and
a huge fan of michael yeah yeah he is a
remarkable
person um i i after my talk today
i went into this i went back at ball
he's good at that
basketball no but basketball racquetball
squash which is not
racquetball yes squash no and i hope you
you hear that michael
you mean like your parents as a game not
his skill level because i'm pretty sure
he's all right there's some
competitiveness there but the point
is that it was like the middle of the
day i had full day of interviews like i
met with people but then in the middle
of the day i gave
a job talk and then um and then there
was going to be more interviews but
i i pulled michael aside and i said
i think it's in both of our best
interests if i just leave now
because that was so bad that it's just
be embarrassing if i have to talk to any
more people like you look bad for having
invited me
like it's just let's just forget this
ever happened
so i don't think the talk went well it's
one of the most michael littman set of
sentences i think i've ever heard
he did great or at least everyone knew
he was great so maybe it didn't matter
i was there i remember the talk and i
remember him being very much the way i
remember him now
in any given week so it was good and we
met and we talked about stuff he thinks
i didn't like him but because he was so
grumpy
must been the chair thing the chair
thing and the low voice i think
the like obviously and that like that
like slight like
skeptical look yeah i have no idea what
you're talking about
well i probably didn't have any idea
what you were talking about
anyway i liked him he asked me questions
i answered questions i felt bad about
myself it was a normal day
then he left and then he left and that's
how you tell me can we take it
and then i got hired and i was in the
group can we take a slight tangent on
that on this topic of
it sounds like uh maybe you could speak
to the bigger picture
it sounds like you're quite
self-critical who charles
no you oh i think i can i can do better
i can do better i'll
try me again i'll i'll do better
yeah that was like a like a three out of
ten responses
so let's try to work it up to five and
six uh you know i remember
uh marvin minsky said uh on on a video
interview
something that the key to success in
academic research is to hate everything
you do
for some reason i think i followed that
because i hate everything he's done
[Laughter]
uh it's a good line that's a success
maybe that's a keeper but um but do you
do find that resonates with you at all
in how you think about talks and so on
i would say it differently it's not
really that's such an mit view of the
world though
so i remember i i remember talking about
this when uh as a student you know
you were basically told uh i will clean
it up for the purpose of the podcast
um uh my work is crap my work is crap my
work is crap my work is crap then you
like go to a conference or something
like everybody else's work is crap
everybody else is working crap and you
feel better and better about it yeah
uh relatively speaking and then you sort
of keep working on it
i don't hate my work that resonates with
me yes i've never hated my work but i
have
i have been dissatisfied with it
and i think being dissatisfied being
okay with the fact that you've taken a
positive step the derivative is positive
maybe even the second derivative is
positive that's important because that's
a part of the the hope right
but you have to but i haven't gotten
there yet if that's not there that i
haven't gotten there yet
then you know it's hard to it's hard to
move forward i think so i buy that
which is a little different from hating
everything that you do yeah i mean
there's
there's things that i've done that i
like better than i like myself
so it's separating me from the work
essentially so i think i am very
critical of myself
but sometimes the work i'm really
excited about and sometimes i think it
doesn't happen right away so i found the
work that i've
liked that i've done most of it
i liked it in retrospect more when i was
far away from it in time
i have to be fairly excited about it to
get done
no excited at the time but then happy
with the result or but years later or
even i might go back you know what
that actually turned out to be yeah that
turned out to matter or oh gosh it turns
out i've been thinking about that
it's actually influenced all the work
that i've done since without realizing
it
but that guy was smart yeah that guy had
a future
yeah yeah he's going places
i think there's so yeah so i think
there's something to it i think there's
something to the idea you've got to
you know hate what you do but it's not
quite hate it's just being unsatisfied
and different people motivate themselves
differently i don't happen to motivate
myself with self-loathing
i happen to motivate myself so you're
able to sit back and
be proud of in retrospect of the work
you've done
well and it's easier when you can
connect with other people because then
you can be proud of them
a lot of the people yeah and then the
questions you can still safely hate
yourself
it's a win-win michael or at least win
lose which is what you're looking for
oh wow there's so many brilliant lines
in this
there's levels uh so how did you
actually meet me
yeah so my the way i think about it is
because we didn't do much
research together at 18t but um but then
we all got laid off
so so that was that by the way i decided
to interrupt but that was like
one of the most magical places
historically speaking
they did not appreciate what they had
and how do we uh i feel like there's a
profound lesson in there too uh
how do we get it like what was why was
it so magical is just the coincidence of
history
or is there something special some
really good managers and people who
really believed in
machine learning as this is going to be
important
um let's get the the people who are
thinking about this in creative and
and insightful ways and put them in one
place and
stir yeah but even beyond that right it
was
it was bell labs at its heyday and even
when we were there which i think was
past it
to be clear he's gotten to be at bell
labs i never got to be at bell labs
i joined after that yeah i should have
been 91 as a grad student
so i was there for a long time um every
summer except
twice i worked for companies that had
just stopped being better labs right
bell core and then att labs so about
labs was
several locations or for the for the
research or is it what like
jerseys are involved somehow they're all
in jersey yeah they're all over the
place but they're in a couple places
murray hill was the bell labs um
so you you had you had an office in mary
hill at one point in your career
yeah and i i played ultimate frisbee on
the cricket pitch at bell labs at murray
hill
uh and then it became 18t labs when
split off with loose during what we
called uh tri-vestiture supposedly
better than michael koren's ultimate
frisbee yeah oh yeah
okay but i think that one's not boasting
i think that i think charles plays a lot
of ultimate and i don't think mike i was
yes but but that wasn't the point the
point is yes yes
sorry okay i have played on a
championship winning ultimate frisbee
team
or whatever ultimate team with charles
so i know
how good he is he's really good how good
i was anyway when i was younger but the
thing is i know how young he was when he
was yeah that's true
that was true so much younger than now
he's old enough yeah i'm older michael
is a much
was a much better basketball player than
i was michael kearns yes no not michael
i'm very clear so you don't know how
terrible i am
but you have a probably pretty good
guess that you're not as good as michael
kearns
he's tall and and he cared about it very
outlet he's very good he's
probably competitive i love hanging out
with michael anyway but we were talking
about something else although i no
longer remember what it was what were we
talking about
but also labs so so uh this was kind of
cool about what was magical about it
the first thing you have to know is that
bell labs was an arm of the government
right because att was an army of
government
it was a monopoly uh and you know every
month you paid a little
thing on your phone bill which turned
out was a tax for like all the research
that bell labs was doing
and you know they invented transistors
and the laser and whatever else is that
big bang or whatever the
cosmic background radiation yeah they
did all that stuff they had some amazing
stuff with directional microphones by
the way i got to go in this room
um where they they had all these panels
and everything
and we would talk and one another and he
moved some panels around and then
he would have me step two steps to the
left and i couldn't hear a thing he was
saying because nothing was bouncing off
the walls
and then he would shut it all down and
you could hear your heartbeat yeah
deeply disturbing to hear your heart
beat you can feel it i mean you can feel
it now there's so much all this sort of
noise around anyway bill labs is about
pure research
it was a university in some sense the
purest sense of a university
but without students so it was all the
faculty working with one another
and students would come in to learn they
would come in for three or four months
you know during the summer and they
would go away
but it was just this kind of wonderful
experience i could walk out my door
in fact i would often have to walk out
my door and deal with rich sutton and
michael kearns yelling at each other
about whatever it is they were yelling
about
the proper way to prove something or
another and i could just do that and
dave mcallister and evan
and peter stone and and all of these
other people including
satinder and then eventually michael and
it was just a place where you could
think
thoughts and it was okay because so long
as once every 25 years or so
somebody invented a transistor it paid
for everything else you could afford to
take the risk
and then when that all went away it
became harder
and harder and harder to justify it as
far as the folks who were very far away
were concerned
and there was such a fast turnaround
among middle management
on the atnt side that you never had a
chance to really build the relationship
at least people like us didn't have a
chance to
to build relationships so when the
diaspora happened um
it was amazing right yeah everybody left
and i think everybody ended up at a
great place and
made a huge made a continued to do
really good work with with machine
learning but it was a wonderful place
and people will ask me you know what's
the best job you you've ever had
and as a professor anyway the answer
that i would give is
um well probably
bell labs in some very real sense and i
would never have a job like that again
because bell labs doesn't exist anymore
and you know microsoft research is great
and google does good stuff and you can
pick ibm
you can tell if you want to but bell
labs was magical it was around for it
was an important time
and it represents a a high water mark in
in basic research in the u.s is there
something you could say about the
physical proximity and the chance
collisions
like we live in this time of the
pandemic where everyone
is maybe trying to see the silver lining
and accepting the remote nature of
things
is is there one of the things that
people like faculty
that i talk to miss is the
the procrastination like the chance to
like everything is about meetings that
are supposed to be there's not a chance
to just
uh you know talk about comic book or
whatever like go into discussion that's
totally pointless
so it's funny you say this because
that's how we met matt
it's exactly that so i'll let michael
say that but i'll just add one thing
which is just that uh
you know research is a social process
and it helps to have
random social interactions even if they
don't feel social at the time that's how
you get things done
one of the great things about the a lab
when i was there i
don't quite know what it looks like now
once they moved buildings but we had
entire walls that were whiteboards and
people would just get up there and they
were just right
and people would walk up and you'd have
arguments and you'd explain things to
one another
and you got so much out of the freedom
to do that you had to be
okay with people challenging every
freaking word you said which i would
sometimes find
deeply irritating but most of the time
it was it was quite useful but the sort
of pointlessness and the interaction was
in some sense the point at least for me
yeah i mean you
i think offline yesterday i mentioned
josh tannenbaum and he's
very much he put he's a man he's such an
inspiration
in in the child like
way that he pulls you in on any topic it
doesn't even have to be about machine
learning
it could or or the brain he'll just pull
you into a closest
writable surface which is uh still you
can find whiteboards at mit everywhere
and and just like uh like basically
cancel all meetings and talk for a
couple hours about some
some aimless thing and it it feels like
the whole world the time
space continuum kind of warps and that
becomes the most important thing
and then it's just it's so true it's
it's
definitely something worth missing in
this in this world where everything's
remote
there's some magic to the physical
presence whenever i wonder myself
whether mit really is as great as i
remember it i just go talk to josh yeah
you know that's funny is
there's a few people in this world that
carry the
the best of what particular institutions
stand for right and it's uh it's josh
i mean i i don't i my guess is he's
unaware of this
that's the point that the masters are
not
aware of their mastery so how do we all
meet
yes but but first a tangent no
how did you meet me so i'm not sure what
you were thinking of but my
when it started to dawn on me that maybe
we had a longer-term bond
was after we all got laid off and
you had decided at that point that there
we were still paid we were given an
opportunity to like do job search and
kind of make a transition
but it was clear that we were done and
i would go to my office to work and you
would go to my office to keep me from
working
that was that was my recollection of it
and you had decided that there was no
really no point in working for the
company because the company our
relationship with the company was
was done yeah but remember i felt that
way beforehand it wasn't about the
company it was about the set of people
there doing really cool things and it
always
always been that way but we were working
on something together oh yeah
yeah that's right oh so at the very end
we all got laid off but then
our boss came to our boss's boss came to
us
because our boss was michael kearns and
he had jumped ship
brilliantly like perfect timing like
things like right before the ship was
about to sink
he was like gotta go and and and
landed perfectly because michael kearns
because michael king
and um leaving the rest of us to go like
this is fine and then it was clear that
wasn't fine and we were all
toast so we had this sort of long period
of time but then our boss figured out
okay wait maybe we can save a couple of
these people
if we can have them do something really
useful
and uh the useful thing was we were
going to make a
basically an automated assistant that
could help you with your calendar you
could like
tell it things and it would it would
respond appropriately it would just kind
of integrate across
all sorts of your personal information
and so me and charles and peter stone
were this were set up as the crack team
to actually solve this problem
uh other people maybe were too
theoretical that they thought and
and but we could actually get something
done so we sat down to get something
done and there wasn't time
and it wouldn't have saved us anyway and
so it all kind of went downhill
but the interesting i think coda to that
is that our boss's boss is a guy named
ron brockman
and he when he left at t
because we were all laid off he went to
darpa
started up a program there that became
kalo
which is the program from which siri
sprung
which is a digital assistant that helps
you with your calendar and a bunch of
other things
um it really you know in some ways got
its start
with me and charles and peter trying to
implement this vision that ron brockman
had that
he ultimately got implemented through
his role at darpa
so when i'm trying to feel less bad
about having been laid off from
what is possibly the greatest job of all
time
i think about well we kind of helped
birth siri
that's something and he did other things
too but
the we got to spend a lot of time in his
office and
talk about we got to spend a lot of time
in my office yeah
yeah yeah and so uh so then we went on
our merry way
everyone went to different places
charles landed at georgia tech which was
uh what he always dreamed he would do
and so
um that worked out well yeah um i
came up with a saying at the time which
is luck favors the charles
it's kind of like luck favors the
prepared but charles like
like he'd wish something and then it
would basically happen just the way he
wanted it was
it was inspirational to see things go
that way things worked out and we stayed
in touch
and then um i think it really helped
when you were working on i mean you kept
me in the loop for things like threads
and the work that you were doing at
georgia tech but then
when they were starting their online
master's program he knew that i was
really excited about
moocs and online teaching and he's like
i have a plan and i'm like tell me your
plan he's like i can't tell you the plan
yet because they were
deep in in negotiations between georgia
tech and udacity to make this happen
and they didn't want it to leak so
charles would kept teasing me about it
but wouldn't tell me what was actually
going on and eventually it was announced
and he said i would like you to teach
the machine learning course with me
i'm like that can't possibly work um but
it was a great idea and it was
it was super fun it was a lot of work to
put together but it was it was really
great
and was that the first time you thought
about first of all
was it the first time you got seriously
into teaching
i mean you know i'm trying to get the
feeling right i'll tell you this is
already
after you jump to so like there's a
little bit of
jumping around in time yeah sorry about
it there's a pretty big jump in time so
like the moocs
thing so charles got to georgia tech and
he i mean maybe charles maybe this is a
trick
in 2002. he got to georgia tech in 2002
and um but then and worked on things
like revamping the curriculum the
undergraduate curriculum so that it had
some kind of
semblance of modular structure because
computer science was
at the time moving from a fairly narrow
specific set of topics to touching a lot
of other parts
of of of intellectual life and the
curriculum
was supposed to reflect that and so um
charles played a big role in
in kind of redesigning that and then and
for my and for my
my labors i ended up his associate dean
right he got to become an associate dean
of in charge of educational stuff well
this would be a valuable lesson if
you're
good at something uh they will give you
responsibility to do more of that thing
well until you don't show confidence
don't show confidence if you
well you know what the responsibility
here's what they say yeah
the reward for good work is more work
the reward for bad work is less work
which i don't know depending about what
you're trying to do that week
one of those is better than the other
well one of the problems with the word
work sorry to interrupt
is that it's seems to be an antonym
in this particular language we have the
opposite of happiness
but it seems like they're they're like
that's one of you know we talked about
balance it's uh it's always like
work-life balance it always rubbed me
the wrong way as
a terminology i know it's just words
right the opposite of work is play
but yeah ideally work is play oh i can't
tell you how much time i'd spend
certainly i was about labs except for a
few very key moments
uh as a professor i would do this too i
was just saying cannot believe they're
paying me to do that um because it's fun
it's something that i would i would do
for a hobby
if i could anyway uh so that sort of
worked out i'm sure you want to be
saying that
when this is being recorded as a dean
that is not true at all
i need a raise yes but but i think here
with with this that even though a lot of
time passed
you know michael and i talked almost
every well we texted almost every day
during the period charles at one point
took me
there was the icml conference the
machine learning conference was in
atlanta i was the chair the general
chair of the conference
charles was my publicity chair or
something like that or
something fundraising champion sure yeah
um but he decided it'd be really funny
if he didn't actually show up for the
conference in his own home city
uh so he didn't but he did at one point
picked me up at the conference in his
tesla and drove me to the atlanta mall
and forced me to buy an iphone because
he didn't like
how it was to text with me and thought
it would be better for him
if i had an iphone the text would be
somehow smoother
and it was and it was and it is and his
life is better and my life is better and
so
death but but it was yeah charles
forcing me to get an iphone
so that he could text me more
efficiently i thought that was an
interesting moment it works for me
anyway so we kept talking the whole time
and then eventually we did the
we did the teaching thing and it was
great and there's a couple of reasons
for that by the way one is
i really wanted to do something
different like you've got this medium
here
people claim it can change things what's
a thing that you could do in this medium
that you could not do otherwise
besides edit right i mean what could you
do and and being able to do something
with another person was that kind of
thing it's very hard
i mean you can take turns but teaching
together having conversations is very
hard right
so that was a cool thing the second
thing it gave me an excuse to do more
stuff with him yeah i always thought
he makes it sound brilliant um and it is
i guess but it's at the time it really
felt like
i've got a lot to do charles is saying
and it would be great if michael could
teach
the course and i could just hang out
yeah just kind of coast on that
well that's what the second class was
more like that because the second time
that was explicit because
the first class it was at least half so
the structure
so that was that was kind of true yeah
that was sort of true for 7642 which is
the reinforcement learning class because
that was really his class you started
with reinforcement
no we started with i did the intro
machine learning 7641
uh which is supervised learning
unsupervised learning and
reinforcement learning and decision
making cram all that in there the kind
of assignments that we talked about
earlier
and then eventually about a year later
we did a follow-on 7642 which is
reinforcement learning and decision
making
the first class was based on something
i'd been teaching at that point for well
over a decade
and the second class was based on
something michael had been teachers
actually i learned quite a bit
teaching that class with him but he
drove most of that but the first one i
drove most it was all my material
although i had stolen that material
originally from slides i found online
from michael who had originally stolen
that material from i guess slides he
found online probably from andrew moore
because the jokes were the same
anyway at least some of the at least
when i found the slides some of the
stuff
yes every machine learning class taught
in the early 2000's stole from
andrew moore a particular joke or two
the at least the structure now i did
and he did actually a lot more with
reinforcement learning and such and game
theory and those kinds of things but
you know we all sort of know this world
no no no no i mean
teaching that class the coverage was
different than than what other people
were starting most people were just
doing supervised learning and maybe a
little bit of
you know clustering and whatnot but we
took it all the way to a lot of it
just comes from tom mitchell's book oh
no yeah except well half of it comes
from tom mitchell's book right i mean
the other half doesn't
this is what this is why it's all
readings right because certain things
weren't invented when tom wrote
okay that's true right uh but it was it
was quite good but there's a reason for
that besides
you know just i wanted to do it i wanted
to do something new and i wanted to do
something with him
which is a realization which is despite
what you might believe
he's an introvert and i'm an introvert
or i'm on the edge of it
or being an introvert anyway but both of
us i think
um enjoy the energy of the crowd right
there's something about
talking to people and bringing them into
whatever we find interesting that is
empowering energy energizing or whatever
and i found the idea
of staring alone uh at a computer screen
and then talking off of materials
less inspiring than i wanted it to be
and i had in fact done a mooc for
udacity on
algorithms and it was a week
in a dark room talking at the screen
writing on the little pad and i didn't
know this was happening but they had
watched the
crew had watched some of the videos
while like in the middle of this and
they're like
something's wrong you're you're
sort of shutting down um and i think a
lot of it was i'll make
jokes and no one would laugh yeah and i
felt like the crowd hated me now of
course there was no crowd
so like it wasn't rational yeah but it's
little each time i tried it and i got no
reaction it just was taking
the the the energy out of my performance
out of my
presentation fantastic metaphor for grad
school anyway by working together
we could play off each other and have it
and and have it keep the energy up
because you can't
you with you can't let your guard down
for a moment with charles he'll just
he'll just
overpower you i have no idea what's
wrong with but we would work really well
together i thought and we knew each
other so i knew that we could we could
sort of make it work plus i was the
associate dean so
they had to do what i told him to do we
had to do that we had to make it work
and so it worked out very well i thought
um well enough that we
with great power comes great power
that's right and we became smooth and
curly
and uh that's when we we we did the
the um the uh overfitting thriller video
yeah we took yeah yep that's a thing
so what okay can we just like like uh
smooth and curly where was that so okay
so
that happened it was completely
spontaneous these are the nicknames you
go by yeah so
uh students call us he was he was
lecturing
so the way that we structured the
lectures is one of us is the lecturer
and one of us is basically the student
and so the he was lecturing on the
lecturer prepares all the materials
comes up with the quizzes
and then the student comes in not
knowing anything so it's you know just
like being on campus
yeah uh and i was doing game theory in
particular
the prisoner's delivery's dilemma and so
he needed to set up a little prisoner's
dilemma grid so he drew it and i could
see what he was drawing
and the the prisoner's dilemma consists
of two players two parties so he decided
he would make little cartoons of the two
of us and so
there was uh two criminals right that
were deciding whether or not to rat each
other out
um one of them he drew as you know a
circle with a smiley face
and a kind of goatee thing smooth head
and the other one with all sorts of
curly hair and he said this is smooth
and curly i said smooth and curly he
said no smoove
with a v it's very important that it
happened v and then talk actually
and the students really the students
really took to that like they've really
they found that relatable
he started singing smooth criminal by
michael jackson yeah yeah yeah and
that's
those those names stuck so that so we
now have a video series the
an episode our kind of first actual
episode should be coming out today
um smoove and curly on video
where the two of us discuss uh west
episodes of westworld we watch westworld
and we're like
huh what does this say about computer
science and ai
and we've never we did not watch it i
mean i know it's on season three or
whatever we have
as of this recording it's on season
three and uh that's now two episodes
total yeah i think it was three
what do you think about westworld two
episodes in so i can tell you guys
so far yeah i'm just guessing what's
going to happen next
it seems like bad things are going to
happen with the robot's uprisings
so i no i have not i mean you know i
vaguely remember a movie existing so i
assume it's it's related to that
but that was more my time than your time
charles that's right because you're much
older than i i think the important thing
here
is that uh it's narrative right it's all
about telling a story that's the whole
driving thing
but the idea that they would give these
reveries that they would make
people they would make them remember
remember the awful things that happened
that happened
who could possibly think that was good i
got a i mean i don't know i've only seen
the first two episodes or maybe the
third one i think i've only said you
know what it was you know what the
problem is
that the robots were actually designed
by hannibal lecter that's true
they were so like what do you think is
going to happen it's a bad thing
it's clear that things are happening and
characters are being introduced and we
don't yet know anything
but still i was just struck by how it's
all driven by narrative and story and
there's all these implied things like
programming hap the programming
interface is talking to them about
what's going on in their heads
which is both i mean artistically it's
probably useful to film it that way but
think about how it would work in real
life that just seems very crazy but
there was
we saw in the second episode there's a
screen you could see things they were
wearing like
in the world it was quite interesting to
just kind of ask this question
so far i mean i assume it veers often to
never neverland at some point
but uh we can't answer that question i'm
also a fan of a guy named alex garland
he's a director of ex machina
and he is the first i wonder if kubrick
was like this actually
is he like studies
what would it take to program in ai
systems like he's he's curious enough to
go into that direction
on the west wall side i felt there was
more emphasis on the narratives than
like actually
asking like computer science questions
yeah like how would you
build this how would you uh and how
would you debug it i still think
to me that's the key issue they were
terrible debuggers
yeah and well they said specifically so
we make a change and we put it out in
the world and that's bad because
something terrible could happen
like if you're putting things out in the
world and you're not sure whether
something terrible is going to happen
you're probably your process is probably
i just feel like there should have been
someone whose sole job it was to walk
around and poke his head
and say what could possibly go wrong
just over and over again
i would have loved if there was an and i
did watch a lot more i'm not giving
anything away
i would have loved it if there was like
an episode where like
like the new intern is like debugging a
new model or something and like it just
keeps failing and
they're like all right and then it's
more turns into like a
episode of silicon valley or something
like that yes versus like
all this ominous ai systems that are
constantly like
threatening the the fabric of this world
that's been created
yeah yeah and you know the other the
this this reminds me of something that
so i agree with that that actually be
very cool at least well for the small
percentage of people who care about
debugging
systems but the other thing is debugging
the series
it falls into think of the sequels fear
of the debate oh my gosh
and anyway so a nightmare show it's a
it's a horror movie
i think that's where we lose people by
the way early on is the people who
either decide either figure out
debugging or think debugging is terrible
this is where we lose people in computer
science this is part of the struggle
versus
suffering right you you get through it
and you kind of get the skills of it or
you're just like this is dumb and this
is a dumb way to do anything and
i think that's when we lose people but
um i
well i'll leave it at that but i think
that i think that that there's something
really really neat about framing it that
way but what i
don't like about all of these pro all of
these things and i love text mocking and
by the way i thought the
ending was very depressing um but again
one of the things after talking to alex
about
he says that the thing that nobody
noticed he put in
is uh the at the end spoiler alert
the the robot turns and looks at the
camera
and smiles right briefly and to him
he thought that his definition of
passing
the touring the general version of the
turing test or the consciousness test
is smiling for no one hmm
oh like like
not oh you know it's it's like the
chinese room kind of
experiment it's not always trying to act
for others right but just on your own
being able to have a relationship with
the actual experience
and just like take it in i don't know he
said like nobody noticed
i mean the magic of it i had this vague
feeling that i remember the smile but
now you now you just put the memory in
my head so probably not but i do think
that
that's interesting although by looking
at the camera you are smiling for the
audience right you're breaking the
fourth wall
it seems i mean well that's a that's a
limitation in the medium but i
i like that idea but here's the problem
i have with all of those movies all of
them
um is that but i know why it's this way
and i enjoy those movies
um and westworld is uh it sets up the
problem of ai
as succeeding and then having something
we cannot control
but it's that's not the bad part of ai
the bad part of ai is the stuff we're
living through now right it's the
using the data to make decisions that
are terrible it's not the intelligence
that's going to go out there and surpass
us and you know take over the world or
you know lock us into a room to starve
to death slowly
over multiple days it's instead uh the
the tools that we're building that are
allowing us
to make the terrible decisions we would
have less efficiently made before
right you know computers are very good
at making us more efficient
including being more efficient at doing
terrible things and and that's the part
of the ai we have to worry about it's
not the
you know true intelligence that we're
going to build sometime in the future
probably long after we're around
um but you know i i i i just
i think that whole framing of it sort of
misses
the point even though it is inspiring
and i was inspired by those ideas right
that i got into this in part because i
wanted to build something like that
philosophical questions were interesting
me but but you know that's not where the
terror comes from the terror comes from
the every day
and you can construct situations in the
subtlety of the interaction between ai
and
the human like with the with social
networks
all the stuff you're doing with uh
interactive artificial intelligence
but you know i i feel like cal 9000 came
a little bit closer to that when it's
in 2001 space odyssey because it felt
like
uh a personal assistant you know it felt
like closer to the ai systems we have
today and
and the real things we might actually
encounter which is
over relying uh on in some fundamental
way on our
like dumb assistance or on social
networks like over
offloading too much of us onto uh
you know onto things that require
internet and power and so on and thereby
becoming powerless
as a stand-alone entity and then when
that thing starts to misbehave
in some subtle way it creates a lot of
problems
and those problems are dramatized when
you're in space because
you don't have a way to walk away well
as the man said
um once you once we started making the
decisions for you it stopped being your
world
right that's the matrix michael in case
you don't i didn't generally i don't
remember
but on the other hand i could say no
because isn't that what we do with
people anyway
you know this kind of the shared
intelligence that is humanity is relying
on other people constantly to i mean we
hyper-specialize right
as individuals we're still generally
intelligent we make our own decisions in
a lot of ways but we leave most of this
up to other people and
that's perfectly fine and by the way
everyone doesn't necessarily
share our goals sometimes they seem to
be quite against us sometimes we
make decisions that others would see as
against our own interests and yet we
somehow manage it
manage to survive i'm not entirely sure
why an ai
would actually make that worse
or even different really you mentioned
the matrix
do you think we're living in a
simulation it does
feel like a thought game more than a
real scientific question well i'll tell
you why like i think it's an interesting
thought experiment see what you think
from a computer science perspective
it's a good experiment of how difficult
would it be
to create a sufficiently realistic world
that us humans would enjoy being in
it that that's almost like if we're
living in a simulation then i don't
believe that we were put in the
simulation i believe that
it's just physics playing out and we
came out of that
like i don't i don't i don't think so
you think you have to build the universe
yeah i think
the universe itself we can think of that
as a simulation and in fact
what i try sometimes i try to think
about to understand
what it's like for a computer to
start to think about the world i try to
think about the world
things like quantum mechanics where it
doesn't feel very natural to me at all
and it really strikes me as i don't
understand this thing that we're living
in
it has there's weird things happening in
it that don't feel natural to me at all
now if you want to call that as the
result of a simulator
okay i'm fine with that but like i don't
know the bugs in the simulation
there's the bugs i mean the interesting
thing about the simulation is that it
might have bugs i mean that that's the
thing that i the but there would be bugs
for the people in the simulation they're
just that's just
reality unless you were fair enough to
know that there was a bug
but i i think back to the matrix yeah
the way you put this
i don't think that we live in a in a
simulation created for us
okay i would say that i think that's
interesting i've actually never thought
about it that way i mean
you the way you asked the question
though is could you create a world that
is
enough for us humans it's an
interestingly sort of self-referential
question because
the beings that created the simulation
probably have not created a simulation
that's realistic for them
but we're in the simulation and so it's
realistic for us
so we could create a simulation that is
fine for the
people in the simulation as it were that
would not necessarily be fine for us as
the creators of the simulation
but well you can you can forget i mean
when you go into the
if you play video games in virtual
reality you can
if with some suspension of disbelief or
whatever yeah uh
it becomes a world it becomes the world
even like in brief moments
you forget that another world exists i
mean that's what like good stories do
they pull you in and the question is is
it possible to pull
you know our brains are limited is it
possible to pull the brain in
to where we actually stay in that world
longer and longer longer and longer
and like not only that but we don't want
to leave
and so especially this is the key thing
about the
developing brain is if we journey into
that world early on in life
often how would you even know yeah yeah
so i
but like from a video game design
perspective from a west world
perspective it's
i think i think it's an important thing
for even
uh computer scientists to think about
because it's clear that video games
are getting much better and virtual
reality
although it's been ups and downs just
like artificial intelligence
it feels like virtual reality will be
here in a very impressive form if we
were to fast forward 100 years into the
future
in a way that might change society
fundamentally like
if i were to i'm very limited in
predicting the future as all of us are
but if i were to try to predict
like in which way i'd be surprised to
see the world
100 years from now it'd be
that or impressed it'd be that we're all
no longer living in this physical world
that we're all living in a virtual world
you really need to be calculating god by
sawyer it's a
you'll read it in a night it's a very
easy read but it's
assuming you're that kind of reader but
it's a it's a good story
and it's kind of about this but not in a
way that it appears
and i uh really enjoyed
the thought experiment um i think it's
pretty sure it's robert sawyer but
anyway he's he's
apparently canadian's top science
fiction writer which is why this story
mostly takes place in toronto
uh but it's a it's a very good uh it's a
very good sort of
story that sort of imagines this
very different kind of simulation
hypothesis sort of thing from say um
the egg for example you know you know
i'm talking about the short story
um by the guy who did the martian
who wrote the martian mm-hmm you know
matt damon
no the book so we had this whole
discussion
that michael doesn't uh doesn't partake
in this exercise of reading
yeah he doesn't seem to like it which
seems very strange to me considering how
much he has to read
i read all the time i used to read 10
books at every week when i was a
when i was in sixth grade or whatever i
was a lot of it science fiction a lot of
it
a lot of history but i i love to read
but anyway you should recalculating god
i think you'll
you'll it's very easy read like i said i
think you'll enjoy
sort of the ideas that it presents yeah
i think the the thought experiment is
quite interesting
uh one thing i've noticed about people
growing up now
i mean we'll talk about social media but
video games is a much bigger bigger and
bigger and bigger part of their lives
and then the video games have become
much more realistic i think
it's possible that the three of us
are not uh and maybe the two of you are
not familiar exactly with the numbers
we're talking about here
the number of people it's bigger than
movies right it's it's
it's huge i used to do a lot of the
narrative computational narrative stuff
i understand that economists can
actually see the impact of video games
on the labor market that there are there
there's fewer young men of a certain age
participating in like paying jobs than
you'd expect and and that they trace it
back to video games i mean the problem
with star trek
was not warp drive or teleportation
it was the holodeck like if you have the
holodeck
that's it that's it you go in the
holodeck you never come out i mean
it just never made once i saw that i
thought okay well
so this is the end of humanity as we
know it right they've invented the
holiday
because that feels like the singularity
not some agi or whatever
it's some possibility to go into another
world
that can be artificially made better
than this one
and slowing it down so you live forever
or speeding it up so you appear to live
forever or
making the decision of when to die and
then
most of us will just be old people on
the porch yelling at the kids these days
in their virtual reality
[Music]
but they won't hear us because they've
got headphones on so
i mean rewinding back to moocs
is there lessons that you've uh speaking
of kids these days
uh that was a transition all right i'll
edit i'll fix it and post
yeah that's charles's favorite phrase
fix it in post fix it in post exit in
post they said all when we were
recording all the time whenever
the editor didn't like something or
whatever i would say we'll fix it in
post
he hated that yeah he hated that more
than anything because charles's way of
saying i'm not going to do it again
[Laughter]
you know you're on your own for this one
but it always got fixed in post
exactly right so uh is there something
you've learned about
i mean it's interesting to talk about
moocs is there something you've learned
about the process of education about
thinking about the present i think
there's two lines
of conversation to be had here is the
future of education
in general that you've learned about and
more presciently is the
education in the times of covid yeah
well the second thing in some ways
matters more than the first
um for at least in my head for the not
just because it's happening now but
because
um i think it's it's reminded us of a
lot of things coincidentally today
there's an article out by a good friend
of mine um who's also a professor of
georgia tech but more importantly a
writer and editor at the atlantic i
named ian bogus
um and the title is something like
americans will sacrifice anything for
the college experience
and it's about why we went back to
college and why
people wanted us to go back to college
and it's not you know greedy presidents
trying to get the last dollar from
someone it's because they want to go to
college and what they're paying for is
not the classes what they're paying for
is the college
experience it's not the education it's
being there i've believed this for a
long time
that we continually make this mistake of
people want to go back to college as
being people want to go back to class
they don't they want to go back to
campus they want to move away from home
they want to do all those things that
people experience it's a rite of passage
it's a
it's a identity if i can if i can steal
some of them
ian's words here and i think that's
right and i think what we've learned
through kovid
is it has made it the disaggregation was
not the disaggregation of the education
from the place
universe the university place and that
you can get the best anywhere you want
to
it turns out there's lots of reasons why
that is not necessarily true the
disaggregation
is having it shoved in our faces that
the reason to go again that the reason
to go to college
is not necessarily to learn it's to have
the college experience
and that's very difficult for us to
accept even though we behaved that way
most of us when we were undergrads you
know a lot of us didn't go to every
single class
we learned and we got it and we look
back on it and we're happy we had the
learning experience as well obviously
particularly us because this is
the kind of thing that we do and my
guess is that's true of
the vast majority of your audience but
that doesn't mean the
i'm standing in front of you telling you
this is the thing that people are
excited about um and that's why they
want to be there primarily why they want
to be there
so to me that's what coveted has forced
us to deal with
um even though i think we're still all
in deep denial about it
and hoping that it'll go back to that
and i think about 85
of it will we'll be able to pretend that
that's really the way it is again and
we'll forget the lessons of this
but technically what will come out of it
or technologically will come out of it
is
a way of providing a more dispersed
experience through online education and
these kinds of
remote things that we've learned and
we'll have to come up with new ways to
engage
them in the experience of college which
includes not just the parties or
whatever kids do but the learning part
of it so that they actually come out for
five or six years later
with having actually having actually
learned something so
um i think the world will be radically
different afterwards and i think
technology
will matter for that just not in the way
that the people who are building the
technology originally
imagined it would be and i think this
would have been true
even without covid but covet has
accelerated that
reality so it's happening in two or
three years or five years as opposed to
10 or 15.
that was an amazing answer that i did
not understand
so it was passionate and and i but i
don't know i just didn't
no i'm not trying to criticize it i
think i'm i don't think i'm getting it
so
you mentioned disaggregation so what's
that well so
you know the power the power of
technology that if you go on the west
coast and hang out long enough is all
about we're going to disaggregate these
things together the books from the
bookstore you know that kind of a thing
and then suddenly amazon controls the
universe right and technology is a
disrupter
right and people have been predicting
that for uh higher education for a long
time but certainly
so is this is this the sort of idea like
students can aggregate on a campus
someplace and then take classes
over the network anywhere yeah this is
what people thought was going to happen
or at least people claimed it was going
to happen right that you know because my
daughter is essentially doing that now
she's on one campus but learning in a
different campus sure and kobe makes
that possible
right um okovi makes that um league all
but avoidable
right but the idea originally was that
you know you and i were going to create
this machine learning class and it was
going to be great and then no one else
would
be the machine learning class everyone
takes right that was never going to
happen but
you know something like that but i feel
like you didn't address that so why why
why is it that why cue why i don't think
that will be the thing that happens the
college experience maybe i maybe i
missed what the college experience was i
thought it was
peers like people hanging around a large
part of it is peers
well it's peers and independence yeah
but none of that
you can do classes online for all of
that no no no no
because no definitely we're social
people right so you want to be able to
take the classes that also has to be
part of an experience
it's in a context in the context of the
university and by the way it actually
matters
that georgia tech really is different
from brown
i see because then students can choose
the kind of experience they think is
going to be best for them
okay i think we're giving too much
agency to the students in making an
informed decision
okay but the truth but yes they will
make choices and they will have
different experiences and some of those
choices will be made for them
some of them will be choices they're
making because they think it's this that
or the other i just don't want to say
i don't want to give the idea not
homogenous yes it's certainly not
homogeneous
right i mean georgia tech is different
from brown brown is different
from pick your favorite state school in
iowa
iowa state okay which i guess is my
favorite state school in iowa sure
but you know these are all different
they have different contexts and a lot
of those contexts
are they're about history yes but
they're also about the location of where
you are
uh they're about the larger group of
people who are around you whether you're
in athens georgia and you're basically
the only thing that's there
as a university you're responsible for
all the jobs or whether you're at
georgia state university which is an
urban campus
where you're surrounded by you know six
million people uh in your campus where
it ends and begins in the city ends it
begins we we don't know
it actually matters whether you're a
small campus or a large campus
why is it that if you go to georgia tech
you're like
forever proud of that and you like say
that to people
at dinner with like bars and whatever
and if you
not you know if you get a degree in an
online university somewhere you don't
that's not a thing that comes up at a
bar
well it's funny you say that so the
students who take our online masters
by several measures are more loyal than
the students who come on campus
certainly for the master's degree
the reason for that i think and you'd
have to ask them but based on
my conversations with them i feel
comfortable saying this is because
this didn't exist before i mean we talk
about this online masters and that it's
reaching you know 11 000 students and
that's an amazing thing and we're
admitting everyone we believe we can
succeed we got a
60 acceptance rate it's amazing right
it's also a 6600
degree the entire degree costs 6 600 7
000 depending on how long you take
a dollar degree as opposed to 46 000
cost you to come on campus
so that feels and i can do it while i'm
working full-time and i've got a family
and a mortgage and all these other
things
so it's an opportunity to do something
you wanted to do but you didn't think
was possible
without giving up two years of your life
as well as all the money and everything
else the life that you had built
so i think we created something that's
had an impact
but importantly we gave a set of people
opportunities they otherwise didn't feel
they had so i think people feel very
loyal about that my biggest piece of
evidence for that besides the surveys
is that we have somewhere north of 80
students
might be 100 at this point who graduated
but come back in ta for this class for
basically minimum wage
even though they're working full-time
because they believe they believe
in sort of having that opportunity and
they want to be a part of something now
will they
will generation 3 feel this way 15 years
from now will people have that same
sense i don't know but right now
they they kind of do and so it's not the
online it's
it's a matter of feeling as if you're a
part of something right we're all very
tribal
yeah right um and i think there's
something
very tribal about being a part of
something
like that being on campus makes that
easier going through a shared experience
makes it
easier it's harder to have that shared
experience if you're alone
looking at a computer screen we can
create ways to make that is it possible
it is that's the question is it still is
the intuition to me
and it was at the beginning when i saw
something like
the online master's program is
that this is going to replace
universities and it won't replace
universities better but like where is it
why because it's living in a different
part of the ecosystem
right the people who are taking it are
already adults they've gone through
their undergrad experience
their i think their their goals have
shifted from when they were 17.
um they have other things that that are
going right but it does do something
really important something very social
and very important right
you know this whole thing about um you
know don't build the sidewalks just
leave the grass and the students
or the people will walk and you put the
sidewalks where they create paths kind
of things
yeah um their architects apparently
believe that's the right way to do
things
the metaphor here is that we
we created this environment we didn't
quite know how to think about
the social aspect but you know we didn't
have time to solve all
do all the social engineering right um
the students did it themselves they
created
um you know these groups like on google
plus they're like 30 something groups
created in the first year because
somebody had these google plus um and
they created these groups
and they divided up in ways that made
sense we live in the same state or we're
working on the same things we have the
same background or whatever
and they created these social things we
sent them t-shirts
and they were we have all these great
pictures of students putting on their
t-shirts as they travel around the world
i climb to this mountaintop i'm putting
this t-shirt on
i'm a part of this they were they were
part of them they created the social
environment on top of the social network
and the social media that existed
uh to create this sense of belonging and
being a part of something they found a
way to do it
right and i think they had other
it scratched an itch that they had but
they had scratched some of that itch
that might have required they'd be
physically in the same place
long before right so i think
yes it's possible and it's more than
possible it's necessary
but i don't think it's going to replace
the university as we know it the
university as we know it will change
but there's just a lot of power and the
kind of rite of passage kind of going
off to yourself now maybe there'll be
some other rite of passage that will
happen
right that's the best drive or somewhere
else you can separate so the university
is such a fascinating
uh mess of things so just even the
faculty position
is a fascinating mess like it doesn't
make any sense it it stabilizes itself
but like why are the world-class
researchers
spending a huge amount of time of their
time teaching
and service like you're doing like three
jobs yeah
and and i mean it turns
it's maybe an accident of history or
human evolution i don't know it seems
like the people who are really good at
teaching
are often really good at research there
seems to be a parallel
there but like it doesn't make any sense
that you should be doing that
at the same time it also doesn't seem to
make sense
that your place where you party
is the same place where you go to learn
calculus or whatever the
but it's a safe space safe space for
everything yeah relatively speaking it's
a safe space now by the way i feel
the need very strongly to point out that
we are living in a very particular weird
bubble
right most people don't go to college
and by the way the ones who do go to
college
they're not 18 years old right they're
like 25 or something i forget the
numbers you know
the places where we've been where we are
uh they look like whatever we think the
traditional movie version
of universities are but for most people
it's not that way at all by the way most
people who drop out of college it's
entirely
for financial reasons right the the so
you know we were talking about a
particular experience
um and so for that set of people which
is very small but larger than it was
a decade or two or three or four
certainly ago
i don't think that will change my
concern
which i think is kind of implicit in
some of these questions is that
somehow we will divide the world up
further uh
into the people who get to have this
experience and get to have the network
and they sort of benefit from it
and everyone else while increasingly
requiring that they have more and more
credentials in order to get a job
as a barista right you got to have a
master's degree in order to to work at
starbucks
i mean we're going to force people to do
these things but they're not going to
get to have that experience and there'll
be a small group of people who do who
continue to you know
positive feedback et cetera et cetera i
worry a lot about that
which is why for me um and by the way
here's an answer to your question about
faculty which is why to me that you have
to focus on
access and the mission i think the
reason whether it's good bad or strong i
mean i agree it's strange
but i think it's useful to have the
faculty member particularly large r1
universities where
we've all had experiences uh that
you tie what they get to do and with the
fundamental
mission of the university and let the
mission drive what i hear when i talk to
faculty is
they love their phd students because
they're creating they're reproducing
basically right and it lets them do
their research and multiply
but they understand that the mission is
the undergrads
and so they will do it without complaint
mostly
because it's a part of the mission and
why they're here and they have
experiences with it themselves and that
it was important to get them get them
where they were going the people tend to
get squeezed in that by the way are the
master students
right who are neither the phds who are
like us nor the undergrads we
we have already bought into the idea
that we we have to teach though
that's increasingly changing anyway i
think tying that mission in really
matters and it gives you a way to
unify people around making it an actual
higher calling
education feels like more of a higher
calling to me than than even research
because education you cannot treat it as
a hobby
if you're going to do it well but but
that's the that's the pushback
on this whole system is that you should
education be a full-time job right and
like it's almost like
research is a distraction from that
yes although i think most of our
colleagues many of our colleagues would
say that research is a job and education
is
the distraction right but that's the
beautiful dance it seems to be that
that tension in itself is seems to work
seems to bring out the best in uh
in the faculty or like that but i will
point out two things
one thing i'm going to point out the
other thing i want michael to point out
because i think michael is much closer
to the
to the to sort of the the ideal
professor
in some sense than i am well you're the
platonic sense of a performance
i don't know what he meant by that but
he's he is a dean so he has a different
experience i'm giving him
i'm giving him time to think of the
profound thing he's going to that's good
but let me
let me point this out which is that we
have lecturers in the college of
computing where i am
uh there's 10 or 12 of them depending on
your account as opposed to the
90 or so tenure track faculty those 10
lecturers who
only teach well they don't only teach
they also do service they some of them
do research as well but
primarily they teach they teach 50
over 50 of our credit hours and we teach
everybody
right so they're doing not just
they're doing more than eight times the
work of the tenure track faculty
uh by just if more closer to nine or ten
and that's including our grad courses
right so they're doing this they're
teaching more they're touching more
more than anyone and they're beloved for
it i mean
so we recently had a survey we do these
alumni everyone does these alumni
surveys you hire someone from the
outside to do whatever and
and i was really struck by something you
saw these really cool numbers i'm not
going to talk about it because you know
it's all internal confidential stuff but
one thing i will talk about
is there was a single question we asked
our alum and these are people who
graduated you know born in the 30s and
40s
all the way up to people who graduated
last week right um
well that's great okay good um
and here's the question name this a
single person who had a
strong positive impact on you something
like that i think it was
special impact yeah special impact on
you and then so they got all the answers
from people and they created a word
cloud
there was clear word cop created by
people who don't do word clouds for a
living because they had one person whose
name like appeared
nine different times like philip phil dr
phil you know but whatever
but they got all this and i looked at it
and i noticed something really cool
the five people from the college of
computing
i recognized were in that cloud and
um four of them were lecturers
the people who teach two of them
relatively
modern both were chairs of our division
of computing instruction one just
one retired one is going to retire soon
and the other two were lecturers i
remembered
from the 1980s um two of those four
by the way the fifth person was charles
that's not important the thing is
i i don't tell people that but the two
of those people are teaching the words
are named after
thank you michael two of those are our
teaching awards are named after right
so when you ask students alumni people
who are now 60 70 years old even
you know who touch them they say the
dean of students they say the big
teachers who taught the big introductory
classes that got me into it there's a
guy named
richard bark who's on there who's who's
you know i who's known as a great
teacher
uh the the phil adler guy who um who uh
i probably just said his last name wrong
but i know the first name is phil
because he kept showing up over and over
again
uh famous adler is what it said okay
good but different people spelled it
differently so he appeared multiple
times right
so he was a uh clearly he was a
professor in
um the business school um but when you
read about himself i went to read a box
i was curious who he was you know it's
all about his teaching and the students
that he touched right
so whatever it is that we're doing and
we think we're doing that's important or
why we think the universities function
the people who go through it yeah they
remember the people who are kind to them
the people who
taught them something and they do
remember it they remember it later
i think that's important that's where
the mission matters
yeah not to completely lose track of the
fundamental problem of how do we replace
uh the the party aspect of universities
uh before we go to the what makes the
platonic professor
do you uh do you think
like what in your sense is the role of
moocs in this whole picture
during covid like are we should we
desperately be clamoring to get back on
campus
or is this a stable place to be for a
little while i don't know i know that
it's that
it's the online teaching experience and
learning experience has been
really rough i think that that people
find it to be a struggle in a way that's
not
a happy positive struggle that when you
got through it you just feel like glad
that it's over as opposed to i've
achieved something so you know i worry
about that but
um you know i worry about just even
before this happened i worry about
lecture
teaching is how how well is that
actually really working
as far as a way to do education as a way
to
inspire people i mean all the data that
i'm aware of
seems to indicate and this kind of fits
i think with charles's story
is that people respond
to connection right they actually feel
if they're if they feel connected to the
person teaching the class
they're more likely to go along with it
they're more they're more able to retain
information they're more motivated to be
involved in the class in some way and
and that really matters it people can
mean to the human
themselves yeah so can't you do that
actually perhaps more effectively
online like you mentioned science
communication so i
i literally i think learned linear
algebra from gilbert strang
by watching mit open courseware when i
was in drugs like
and he was a personality he was a bit
like a you know tiny
in his tiny little world of math there's
a bit of a rock star right so you kind
of look up
to that uh to that person can't that
replace
the in-person education it can help
i will point out something i can't share
the numbers but the we have surveyed our
students
and even though they have feelings about
what i would interpret as connection
i like that word um in the different
modes of classrooms
there's no difference between how they
how well they think they're learning
for them the thing that makes them
unhappy
is the situation they're in and i think
the last lack of connection it's not
whether they're learning anything they
seem to think they're learning something
anyway
right in fact they seem to think they're
learning it equally well
uh presumably because the faculty are
putting in
or the instructors more generally
speaking are putting in
uh the energy and effort to try to make
certain that they're
what they've curated can be expressed to
them in a useful way but the connection
is missing and so there's huge
differences in
what they prefer and as far as i can
tell what they prefer is more connection
not less that connection just doesn't
have to be physically in a classroom i
mean look
you know i used to teach 348 students in
a machine learning class on campus you
know why
that was the biggest classroom on campus
they're sitting in a theater they're
sitting in theater seats i'm literally
on a stage looking down on them
and talking to them right there's no
i mean we're not sitting down having a
one-on-one conversation
reading each other's body language
trying to communicate and going we're
not doing any of that
so you know if you're on the if you're
past the third row it might as well be
online anyway is the kind of thing that
people
said daphne has actually said some
version of this um that online starts on
the third row or something like that
and i think that's that's not yeah i
like it i think it captures something
important
but people still came by the way they
even the people who had access to our
material would still come to class
i mean there's a certain element about
looking to the person next to you yeah
it's just like their presence there
their their boredom
and like when the parts are boring and
their excitement when the parts are
exciting
like and sharing in that like unspoken
kind of uh
yeah communication like in part the
connection is with the other people in
the room
watching watching the circus on tv
alone is ever been to a movie theater
and been the only one there at a comedy
it's not as funny as when you're in a
room full of people all laughing
well you need maybe you need just
another person it's like
as opposed to many maybe maybe there's
some kind of well there's different
kinds of connection right
and there's different kinds of comedy
well in the sense as we're learning
today
i wasn't sure if that was going to land
but um just the idea that
that different jokes i i've
i've now done a little bit of stand up
and so different jokes work in different
size crowds too right where sometimes if
you know if it's a
big enough crowd then even a really
subtle joke can
take root someplace and then that cues
other people and it kind of
there's a whole statistics of i did this
terrible thing to my brother so when i
was really young
i decided that my brother was only
laughing at sitcoms when i laughed
like he was taking cues from me so i
like purposely didn't laugh
just to see if i was right laugh at
non-funny things yes i really want to do
both sides i did both sides
and uh and at the end of it i told him
what i did
he was very upset about this yeah and
from that day on
he's he lost his sense of humor no no no
no well yes but from that day on he
he he laughed on his own he stopped
taking cues from me so i want to say
that you know it was a good thing that i
did but yes yes you
was making that man's life yes but it
was mostly me but it's true though it's
true right that people
i i think you're right but okay so
that's where does that get us that gets
us the idea
that i mean certainly movie theaters are
a thing right where people like to be
watching together
even though the people on the screen
can't aren't really co-present with the
people in the audience the
audience is co-present with themselves
by the way and that point it's
it's an open question that's being
raised by this whether movies will no
longer be a thing because netflix's
the audience is growing so that's it's a
it's a very parallel question for
education
will move and theaters still be a thing
right in 2000
but i think i think the argument is that
there is a feeling of being in the crowd
that isn't replicated by being at home
watching it and that there's value in
that
and then i think just but it scales
better on
right i feel like we're having a
conversation
about whether concerts will still exist
after
the invention of the record or the cd or
wherever it is right
you're right concerts are dead um well
okay
i think the joke is only funny if you
say it before
now right yeah like three years ago it's
like well no obviously
i'll wait to publish this until we have
a vaccine
you know we'll fix it in post but i
think the the important thing is
the virus bust concerts changed right
first of all movie theaters weren't this
way right in like the 60s and 70s they
weren't like this like blockbusters
were basically what jaws and star wars
created blockbusters right
before then there weren't like the whole
summer shared summer experience didn't
exist
in our lifetimes right certainly you
were well into adulthood by the time
this was true right
so it's just a very different it was
it's very different so what the what
we're we've been experiencing in the
last 10 years is not like
the majority of human history but more
importantly concerts right
concerts mean something different most
people don't go to concerts
anymore like there's an age where you
care about it you sort of stop doing it
you keep listening to music or whatever
so i think that's a
painful way of saying that um
it will change it's not the same things
are going away replace is too strong of
a word
but it will change it has to i actually
like to push back
i wonder because i think you're probably
just throwing that your intuition out
oh i won't wait and turn it's possible
that concerts
more people go to concerts now but
obviously much more people listen to
well just dumb
when they then before there was records
uh
it's it's possible to argue that uh if
you look at the data
uh that it just expanded the pie of what
music listening means so it's possible
that like universities grow
in the parallel where the theaters grow
but also more people get to watch movies
more people get to
like be educated yes i i hope that yeah
and to this extent that we can grow the
pie
and have education be not just something
you do for four years when you're done
with your other education but it'd be a
more lifelong
thing that would have tremendous
benefits especially as the
the economy and the world change rapidly
like people need opportunities to stay
abreast of these changes and so
i don't know i could i could it's all
part of the the ecosystem it's all to
the good i mean
you know i'm not gonna have an argument
about whether we
we lost fidelity when we went from
laserdisc to dvds or
record players to cds i mean i'm i'm
willing to grant that that is true
but convenience matters
and the ability to do something that you
couldn't do otherwise because that
convenience
matters and you can tell me i'm only
getting 90 of the experience
but i'm getting the experience i wasn't
getting it before or wasn't lasting as
long or it wasn't
i mean this just seems this just seems
straightforward to me it's gonna
it's going to change it is for the good
that more people get access
and it is our job to do two separate
things one to educate them
and make access available that's our
mission but also for
very simple selfish reasons we need to
figure out how to do it better so that
we individually stay in business
we can do both of those things at the
same time they are not in
they may be intentioned but they are not
mutually exclusive
so you've educated some scary number of
people
so you've seen a lot of people succeed
find their
path to life is there advice that you
can give
to a young person today
about computer science education
about education in general about life
about uh whatever the
journey that one takes in there maybe in
their teens in their early twenties
instead of in those underground years as
you try to
go through the essential process of
partying and not going to classes
and yet somehow trying to get a degree
if you get to the point
where you're you're you're far enough up
in the in the
hierarchy of needs that you can actually
make decisions like this
then find the thing that you're
passionate about and pursue it
and sometimes it's the thing that drives
your life and sometimes it's secondary
and you'll do other things because
you've got to eat right
you got a family you got to feed you've
got people you have to help or whatever
i and i understand that and it's not
easy for everyone but
um always take a moment or two to
pursue the things that you love the
things that
bring passion and happiness to your life
and if you don't i know that sounds
corny but i genuinely believe it and
if you don't have such a thing then
you're lying to yourself you have such a
thing you just
have to find it and it's okay if it
takes you a long time to get there
rodney dangerfield became a comedian in
his 50s
i think certainly wasn't his 20s and
lots of people failed for a very long
time before getting to
where they were going um you know i try
to have
hope and it it wasn't obvious i mean you
know we
you and i talked about the experience
that i had
um a long time ago with with a
particular police officer was it my
first one
it wasn't my last one um but you know in
my view
i wasn't supposed to be here after that
and i'm here so it's all gravy
so you might as well go ahead and grab
life as you can because of that that's
that's sort of how i see it
while recognizing again the delusion
matters right allow yourself to be
deluded
allow yourself to believe that it's all
going to work out just don't be so
deluded that you
you miss the obvious and and you're
going to be fine
it's going to be there it's going to be
there it's going to work out
what do you think i like to say choose
your parents wisely
because that has a big impact on your
life
yeah i mean you know i mean there's a
whole lot of things that you don't get
to pick
um and and and whether you get to have
you know one kind of life or a different
kind of life can depend a lot on things
out of your control
but i really do believe in the in the
passion excitement thing
my i was talking to my mom on the phone
the other day and
essentially what came out is that
computer science
is really popular right now and and and
i get to be a professor teaching
something that's very
uh attractive to people and she's she
was like
trying to give me some appreciation for
how
forsightful i was for choosing this line
of work as if somehow i knew that this
is what was going to happen
in 2020 uh but that's not how it went
for me at all like i studied computer
science because i was
just interested it was just so
interesting to me i
didn't i didn't think it would be
particularly lucrative
yeah and i've done everything i can to
keep to keep it as unlucrative as
possible
yeah um some of my you know some of my
friends and colleagues have
have have not done that and i pride
myself on my ability to
just to remain unrich but um
but but i think but but i do believe
that that
like i'm glad i mean i'm glad that it
worked out for me it could have been
like
oh what i was really fascinated by is
this particular kind of engraving that
nobody cares about
but um so i got lucky and the thing that
i cared about happened to be a thing
that other people
eventually cared about but i don't think
i would have had a fun time
choosing anything else like this was the
thing that kept me
interested and engaged well one thing
that people
tell me especially around as an early
undergraduate
and the internet is part of the problem
here
is they say they're passionate about so
many things
how do i choose a thing which is a
harder thing
for me to know what to do with is there
any i mean
don't you know i mean you know
a long time ago i walked down the
hallway and i took a left turn
yeah i could have taken a right turn and
my world
could be better or it could be worse i
have no idea i have no way of knowing is
there anything about this particular
hallway that's relevant or you're just
in general choices yeah you were on the
left it sounds like you regret
not taking the right oh no not at all
you brought it up well
because there was a turn there on the
left was michael lemon's office right
i mean these sorts of things happen
right yes but here's the right by the
way it was just a blank wall
it wasn't a huge choice it would have
really hurt he tried first
no but it's it's true right that you
know i i think about ron brockman right
i
i went i took a trip i wasn't supposed
to take
and i ended up talking to to um
on about this and i ended up going down
this entire path uh that allowed me to i
think get tenure
but by the way i decided to say yes to
something that didn't make any sense and
i went down this educational path
but it would have been you know who
knows right maybe if i hadn't done that
i would be a billionaire right now i'd
be elon musk my life could be so much
better
my life could also be so much worse you
know you just got to feel that sometimes
you have decisions you're going to make
you cannot know what's going to you
should think about it right some things
are clearly smarter than other things
you've got to play the odds a little bit
but in the end if you've got multiple
choices or lots of things you think you
might love go with the thing that you
actually
love the thing that jumps out at you and
sort of pursue it for a little while the
worst thing that will happen is you took
a left turn instead of a right turn
and you ended up merely happy beautiful
so so accepting so taking the step and
just accepting
accepting that that don't like question
questions
life is long and there's time
to actually pursue every once in a while
uh you have to put on a leather suit
and make a thriller video every once in
a while
yeah uh i was told that you actually
danced but that part was edited out i
don't dance
there was a thing where we did do the uh
yeah the zombie thing yeah we did do the
zombies yeah
but that wasn't edited out it just
wasn't able to put into the final thing
i'm quite happy but there was a reason
for that too right like i wasn't wearing
something right there was a reason for
that i can't remember what it was no i
love this suit
is that what it was i can't remember
anyway the right thing happened
exactly you took the left turn and then
the third of the right ended up being
the right thing
so a lot of people ask me that are a
little bit
tangential to the programming the
computing world
and they're interested to learn
programming like all kinds of
disciplines that are outside
of the particular discipline of computer
science what advice
do you have for people that want to
learn how to program or want to
either taste this little
skill set or discipline or try to see if
it can be used somehow in their own life
what stage of life are they in
uh it feels one of the magic things
about the internet
of the people that write me is i don't
know
because my answer is different for for
my daughter is taking ap computer
science right now
hi johnny um she's uh she's amazing and
doing amazing things and
my son's beginning to get interested and
i'll be really curious where he takes it
i think he's
his mind actually works very well for
this sort of thing and she's doing great
but one of the things i have to tell her
all the time she points well i want to
make a rhythm game
so i want to go for two weeks and then
build a rhythm game show me how to build
a rhythm game
and start small learn the building
blocks and hours take the time
have patience eventually you'll build a
rhythm game i was in grad school
when i suddenly woke up one day over the
royal east
um and i thought wait a minute i'm a
computer scientist i should be able to
write pac-man in an afternoon and i did
not with great graphics it was actually
a very cool game i had to figure out how
the ghost moved and everything and i did
it in an afternoon in pascal
on an old apple 2gs um but if i had
started out trying to build pac-man
i think it probably would have ended
very poorly for me luckily back then
there weren't you know these magical
devices we call phones and
software everywhere to give me this
illusion that i could create something
by myself from the basics inside of a
weekend like that
i mean that was a culmination of years
and years and years right before i
decided i should be able to write this
and i could
so you know my advice if you're early on
is
you know you've got the internet there
are lots of people there to give you the
information
find someone who cares about this
remember they've been doing it for a
very long time
take it slow learn the little pieces get
excited about it and then
keep the big projects you want to build
in mind you'll get there soon enough
because
as a wise man once said life is long
sometimes it doesn't seem that long but
it is long
and you'll have enough time to to build
it all out
it all the information is out there but
start small
you know generate different object
numbers that's not exciting but
it'll program well there's only one
programming language it's lisp
but if you have to pick a programming
language i guess
in today's what would i do i guess i do
python is basically doing this but with
better syntax blasphemy yeah
see with c syntax how about that so
you're going to argue that c syntax is
better than
anything anyway also i'll go i'm going
to answer python despite
tell me tell your story about the
somebody's dissertation that had a
lisp program in it it was so funny this
is a this is dave's dave's listening to
him he was like dave mcallister who was
a professor at mit for a while and
and then he came in our in our girl labs
now he's at um now he's a technology
technical institute of chicago uh
a brilliant guy uh such an interesting
guy anyway his thesis
uh it was a theorem proverb
and he decided to have as an appendix
uh his actual code which of course was
in list because of course it was
it's like the last 20 pages are just
right parentheses
it's just wonderful it's like they
that's programming right there just like
pages of
pages of right parentheses anyway lisp
is the only real language but i
understand that that's not necessarily
the place where you start
python is just fine nah python is good
if you're you know of a certain age if
you're really young and trying to figure
it out
graphical languages that let you kind of
see how the thing works and that's fine
too they're all fine it almost doesn't
matter but there are people who spend
a lot of time thinking about how to
build
languages that get people in the
questions are you trying to get in
and figure out what it is or do you
already know what you want
and that's why i asked you what stage of
life people are in because if you're
different stages of life you
you would you would attack it
differently the the answer to that
question of which language keeps
changing i mean there's some
value to exploring uh a lot of people
write to me about julia there's there's
these like more modern languages they
keep being invented rust and
and kotlin and there's stuff that uh
for people who love functional languages
like lisp
there apparently there's echoes of that
but much better
in the modern languages and it's
worthwhile to uh
especially when you're learning
languages it feels like it's okay to try
one that's not
like the popular one oh yeah but you
know you i think you get that
you get that way of thinking almost no
matter what language
and if you if you push far enough like
it can be assembly language
but you need to push pretty far before
you start to hit the really deep
concepts that you would get sooner in
other languages
but like i don't know computation is
kind of computation
it's kind of touring equivalent is kind
of computation and so it's so
it matters how you express things but
you have to build out that mental
structure
in your mind and you i don't i don't
think it super matters which language
i mean it matters a little because some
things are just at the wrong level of
abstraction i think assembly's at the
wrong level abstraction for someone
coming in new
um i think that if you start someone
coming in new
yes for frameworks big frameworks or or
quite a bit um you got to get to the
point where i want to learn any language
means i just pick up a reference book
and
i think of a project and i go through it
in the weekend right you got it you got
to get there you're right though
the languages that are designed for that
are it almost doesn't matter pick the
ones that people have
built tutorials and infrastructure
around to help you get kind of
kind of eased into it because it's hard
i mean we i did this little experiment
once
um i was teaching intro to cs in the
summer as a favor
uh which is anyway i was using
memories i was teaching introduces a
favor it was very funny because i'd go
in every single time and i would think
to myself
how am i possibly going to fill up an
hour and a half talking about for loops
right
and there wasn't enough time it took me
a while to realize this right
there are only three things right
there's reading from a variable writing
to a variable in conditional branching
everything else is syntactic sugar right
the syntactic sugar matters but that's
it
and when i say that's it i don't mean
it's simple i mean it's hard
like conditional branching loops
variable
those are really hard concepts so you
shouldn't be discouraged by this here's
a simple experiment
i'm going to ask you a question now you
ready x equals three
okay y equals four
okay what is x three what is y
four y equals gonna mess this up no
that's oh it's easier
y equals x y equals x what is y
uh three that's right x equals 7
what is y that's one of the trickiest
things to get
for programmers that there's a memory
and the variables
are pointing to a particular thing in
memory and sometimes the languages hide
that from you and they
bring it closer to the way you think
mathematics works right so in fact
mark gosdale who worries about these
sorts of things or used to worry about
these sorts of things anyway
had this kind of belief that actually
people when they see these statements x
equals something y equals something y
equals x
that you have now uh made a mathematical
statement that y and x are the same
which you can if you just put like an
anchor in front of it yes but
people that's not what you're doing yeah
right i thought
and i kind of asked the question and i i
think had some evidence for this
hardly a study is that most of the
people who didn't know the answer
weren't sure about the answer they had
used spreadsheets
ah and so it's a it's a na it's it's
it's you know um it's by it's by
reference
or by name really right and so depending
upon what you think
they are you get completely different
answers the fact that i could go
or one could go two thirds of the way
through a semester
and people still hadn't figured out in
their heads when you say y equals x what
that meant
tells you it's actually hard because all
those answers are possible in fact
when you said oh if you just put an
ampersand in front of it i mean that
doesn't make any sense for an intro
class and of course
a lot of language don't even give you
the ability to think about it in terms
of ampersand
do we want to have a 45-minute
discussion about the difference between
equal eq
and equal in lisp yeah i know you do
[Laughter]
but you know you could do that it's this
is actually really hard stuff
so you shouldn't be it's not too hard we
all do it but you shouldn't be
discouraged it's why you should start
small
so that you can figure out these things
you have the right model in your head
so that when you write the language you
can execute it and build the machine
that you you want to build right yeah
the funny thing about programming
on those very basic things is the the
the very basics are not often made
explicit
which is actually what drives everybody
away from
basically any discipline but program is
just another one like even a simpler
version of the equal sign that i kind of
forget
is in mathematics equals is not
assignment
yeah right like i think basically every
single programming language with just a
few
handful of exceptions equals his
assignment
and you have some other operator for uh
equality
yeah and you know even that like
everyone kind of knows it
once you started doing it but like you
need to say that explicitly or you just
realize it like yourself
otherwise you'll be you might be stuck
for you said like half a semester you
could be stuck for quite a long time
and i think also part of the programming
is being okay in that state of confusion
for a while it's it's to the debugging
point it's like
i just wrote two lines of code why
doesn't this work and staring at that
for like
hours and trying to figure out
and then every once in a while you just
have to restart your computer and
everything works again
and then and then you just kind of stare
into the
void with the tear slowly rolling down
your eye by the way the fact that they
didn't get this
actually had no impact on i mean they
were still able to redo their
assignments right
because it turns out their
misunderstanding wasn't being revealed
to them
yes by the problem sets we were
found actually yeah i wrote a um a
program a long time ago
actually for my master's thesis and uh
in c
plus i think or c i guess we'll see and
it was uh all memory management and
terrible
um and it wouldn't work for a while
and it was some kind of it was clear to
me that it was overwriting memory
and i just couldn't i was like look i
got a paper
time for this so i basically declared a
variable
at the front in the main that was like
400k
just an array and it worked because
wherever i was scribbling over memory
it would scribble into that space and it
didn't matter and so i never figured out
what the bug was
but i did create something to sort of
deal with it to work around it
and it you know that's crazy that's
crazy it was okay because that's what i
wanted
but i knew enough about memory managed
to go you know management to go you know
i'm just going to create an empty array
here and hope that that deals with the
scribbling memory problem
and it did that takes a long time to
figure out and by the way the language
you first learned probably this garbage
collection anyway so you're not even
going to come up across it you're going
to come across that problem
so we talked about the the minsky idea
of hating everything you do
and hating yourself so let's
end on a question that's gonna make both
of you very uncomfortable
okay which is what is your charles
what's your favorite
thing that you're grateful for about
michael
and michael what is your favorite thing
that you're grateful for about
charles well that answer is actually
quite easy
his friendship he's still the easy
i did yeah i'll tell you what i hate
about charles that steals my good
answers
the thing i like most about charles he
sees the world in it in a
similar enough but different way that i
it's sort of like having
another life it's sort of like i get to
experience things that i wouldn't
otherwise
get to experience because i would not
naturally gravitate to them that way
and so he just he just shows me a whole
other world it's awesome
yeah the the inner product is not zero
for sure it's not quite one point seven
maybe
just enough that you can learn
just enough that you can learn that's
the definition of friendship the inner
product is 0.7 yeah
i think so that's the answer to life
really charles sometimes believes in me
when i have not
believed in me he can he also sometimes
works as an outward confidence
that he has so much so much confidence
and
self i don't know aware comfortableness
okay let's go with that um that i feel
better
a little bit if he if he thinks i'm okay
then maybe i'm not as bad as i think i
am
at the end of the day luck favors the
charles
it's a huge honor to talk with you thank
you so much for taking this time
wasting your time with me it was an
awesome conversation you guys are an
inspiration to
a huge number of people and to me so
really enjoyed this thanksgiving i
enjoyed it as well thank you so much
and by the way if luck favors the
charleston is certainly the case that
i've been very lucky to know
i'm going to edit that part out
thanks for listening to this
conversation with charles isabel and
michael littman
and thank you to our sponsors athletic
greens
super nutritional drink ate sleep
self-cooling mattress master class
online
courses from some of the most amazing
humans in history
and cash app the app i use to send money
to friends
please check out the sponsors in the
description to get a discount
and to support this podcast if you enjoy
this thing
subscribe on youtube review it with five
stars napa podcast
follow on spotify support it on patreon
connect with me on twitter
at lex friedman and now let me leave you
some words from desmond tutu
don't raise your voice improve your
argument
thank you for listening and hope to see
you next time