Kind: captions
Language: en
the following is a conversation with
Melanie Mitchell she's the professor of
computer science at Portland State
University and an external professor at
Santa Fe Institute she has worked on and
written about artificial intelligence
from fascinating perspectives including
adaptive complex systems genetic
algorithms and the copycat cognitive
architecture which places the process of
analogy making at the core of human
cognition from her doctoral work with
her advisers Douglas Hofstadter and John
Holland - today she has contributed a
lot of important ideas to the field of
AI including her recent book simply
called artificial intelligence a guide
for thinking humans this is the
artificial intelligence podcast if you
enjoy it subscribe on YouTube give it
five stars on Apple podcast supported on
patreon or simply connect with me on
Twitter at Lex Friedman spelled Fri D ma
n I recently started doing ads at the
end of the introduction I'll do one or
two minutes after introducing the
episode and never any ads in the middle
that can break the flow of the
conversation I hope that works for you
it doesn't hurt the listening experience
I provide time stamps for the start of
the conversation but it helps if you
listen to the ad and support this
podcast by trying out the product the
service being advertised
this show is presented by cash app the
number one finance app in the App Store
I personally use cash app to send money
to friends but you can also use it to
buy sell and deposit Bitcoin in just
seconds cash app also has a new
investing feature you can buy fractions
of a stock say $1 worth no matter what
the stock price is brokerage services
are provided by cash app investing a
subsidiary of square and member s IBC
I'm excited to be working with cash app
to support one of my favorite
organizations called first best known
for their first robotics and Lego
competitions they educate and inspire
hundreds of thousands of students in
over 110 countries and have a perfect
rating and charity navigator which means
that donated money is used to maximum
effectiveness when you get cash app from
the App Store or Google Play and use
code Lex podcast you'll get ten dollars
in cash up will also donate ten dollars
the first which again is an organization
that I've personally seen inspire girls
and boys to dream of engineering a
better world and now here's my
conversation with Melanie Mitchell
the name of your new book is artificial
intelligence subtitle a guide for
thinking humans the name of this podcast
is artificial intelligence so let me
take a step back and ask the old
Shakespeare question about roses and
what do you think of the term artificial
intelligence for our big and complicated
and interesting field I'm not crazy
about the term I think it has a few
problems because it it's means so many
different things to different people and
intelligence is one of those words that
isn't very clearly defined either
there's so many different kinds of
intelligence degrees of intelligence
approaches to intelligence John McCarthy
was the one who came up with the term
artificial intelligence and what from
what I read he called it that to
differentiate it from cybernetics which
was another related movement at the time
and he later regretted calling it
artificial intelligence Herbert Simon
was pushing for calling it complex
information processing which got nixed
but you know probably is equally vague I
guess is it the intelligence or the
artificial in terms of words that it's
the most problematic you would you say
yeah I think it's a little of both but
you know it has some good size because I
personally was attracted to the field
because I was interested in phenom
phenomenons of intelligence and if it
was called complex information
processing maybe I'd be doing something
wholly different now what do you think
of I've heard the term used cognitive
systems for example so using cognitive
yeah I mean cognitive has certain
associations with it and people like to
separate things like cognition and
perception which I don't actually think
are separate but often people talk about
cognition is being different from sort
of other aspects of intelligence it's
sort of higher level so to you cognition
is this broad beautiful mess of things
that's in calm
the whole thing memory yeah I I think
it's hard to draw lines like that when I
was coming out of grad school in the
night in 1990 which is when I graduated
that was during one of the AI winters
and I was advised to not put AI
artificial intelligence on my CV but
instead call it intelligent systems so
that was kind of a euphemism I guess
what about the stick briefly on on terms
and words the idea of artificial general
intelligence or or like beyond Laocoon
prefers human level intelligence sort of
starting to talk about ideas that that
achieve higher and higher levels of
intelligence and somehow artificial
intelligence seems to be a term used
more for the narrow very specific
applications of AI and sort of the
there's the what set of terms appeal to
you to describe the thing that perhaps
would strive to create people have been
struggling with this for the whole
history of the field and defining
exactly what it is that we're talking
about you know John Searle had this
distinction between strong AI and weak
AI and weak AI could be generally AI but
his idea was strong AI was the view that
a machine is actually thinking that as
opposed to simulating thinking or
carrying out intelligent processes that
we would call intelligent
high level if you look at the founding
of the field of McCarthy in sterlin and
so on are we closer to having a better
sense of that line between narrow weak
AI and strong AI yes I think we're
closer to having a better idea of what
that line is early on for example a lot
of people thought that playing chess
would be you couldn't play chess if you
didn't have sort of general human level
intelligence and of course once
computers were able to play chess better
than humans that revised that view and
people said ok well maybe now we have to
revise what we think of intelligence as
or and and so that's kind of been a
theme throughout the history of the
field is that once a machine can do some
task we then have to look back and say
oh well that changes my understanding of
what intelligence is because I don't
think that machine is intelligent at
least that's not what I want to call
intelligence do you think that line
moves forever or will we eventually
really feel as a civilization like we
cross the line if it's possible it's
hard to predict but I don't see any
reason why we couldn't in principle
create something that we would consider
intelligent I don't know how we will
know for sure maybe our own view of what
intelligence is will be refined more and
more until we finally figure out what we
mean when we talk about it but I I think
eventually we will create machines in a
sense that have intelligence they may
not be the kinds of machines we have now
and one of the things that that's going
to produce is is making us sort of
understand our own machine like
qualities that we in a sense are
mechanical in the sense that like an
eles cells are kind of mechanical they
part they have algorithms they process
information by and somehow out of this
mass of cells we get this emergent
property that we call intelligence but
underlying it is really just cellular
processing and and lots and lots and
lots of it do you think we'll be able to
do you think it's possible to create
intelligence without understanding our
own mind you said sort of in that
process we'll understand more and more
but do you think it's possible to sort
of create without really fully
understanding from a mechanistic
perspective sort of from a functional
perspective how our mysterious mind
works if I had to bet on it I would say
no we we we do have to understand our
own minds at least to some significant
extent but it I think that's a really
big open question I've been very
surprised at how far kind of brute force
approaches based on say big data and
huge networks can can take us I wouldn't
have expected that and they have nothing
to do with the way our minds work so
that's been surprising to me so it could
be wrong to explore the psychological
and the philosophical do you think we're
okay as a species with something that's
more intelligent than us do you think
perhaps the reason we're pushing that
line farther and farther is we're afraid
of acknowledging that there's something
stronger better smarter than us humans
well I'm not sure we can define
intelligence that way because you know
smarter then is with with respect to
what what you know computers are already
smarter than us in some areas they could
multiply much better than we can they
they can figure out driving routes to
take much faster and better than we can
they have a lot more information to draw
on they know about you know traffic
conditions and all that stuff so
for any given particular task sometimes
computers are much better than we are
and we're totally happy with that right
I'm totally happy with that I don't
doesn't bother me at all I guess the
question is you know what which things
about our intelligence would we feel
very sad or or upset that machine's had
been able to recreate so in the book I
talk about my former PhD advisor Douglas
Hofstadter who encountered a music
generation program and that was really
the line for him that if a machine could
create beautiful music that would be
terrifying for him because that is
something he feels is really at the core
of what it is to be human creating
beautiful music art literature I you
know I don't think he doesn't like the
fact that machines can recognize spoken
language really well like he doesn't he
personally doesn't like using speech
recognition I don't think it bothers him
to his core because it's like okay
that's not at the core of humanity but
it may be different for every person
what what really they feel would usurp
their humanity and I think maybe it's a
generational thing also maybe our
children or our children's children will
be adapted they'll adapt to these new
devices that can do all these tasks and
and say yes this thing is smarter than
me in all these areas but that's great
because it helps me looking at the broad
history of our species why do you think
so many humans have dreamed of creating
artificial life and artificial
intelligence throughout the history of
our civilization so not just this
century or the 20th century but really
many throughout many centuries that
preceded it that's a really good
question and I have wondered about that
because I'm I myself
you know was driven by curiosity about
my own thought processes and thought it
would be fantastic to be able to get a
computer to mimic some of my thought
process season I'm not sure why we're so
driven I think we want to understand
ourselves better and we also want
machines to do things for us but I don't
know there's something more to it
because it's so deep in in the kind of
Mythology or the dose of our species and
I don't think other species have this
drive so I don't know if you were to
sort of psychoanalyze yourself and
you're in your own interest in AI are
you what excites you about creating
intelligence you said understanding our
own selves
yeah I think that's what drives me
particularly I'm really interested in
human intelligence but I'm all I'm also
interested in the sort of the phenomenon
of intelligence more generally and I
don't think humans are the only thing
with intelligence
you know I or even animals that I think
intelligence is a concept that
encompasses a lot of complex systems and
if you think of things like insect
colonies or cellular processes or the
immune system or all kinds of different
biological or even societal processes
have as an emergent property some
aspects of what we would call
intelligence you know they have memory
they do in process information they have
goals they accomplish their goals etc
and to me that the question of what is
this thing we're talking about here was
really fascinating to me and and
exploring it using computers seem to be
a good way to approach the question so
do you think kind of
intelligence do you think of our
universes a kind of hierarchy of complex
systems and then intelligence is just
the property of any you can look at any
level and every level has some aspect of
intelligence so we're just like one
little speck in that giant hierarchy of
complex systems I don't know if I would
say any system like that has
intelligence but I guess what I want to
I don't have a good enough definition of
intelligence to say that so let me let
me do sort of multiple choice I guess
though
so you said ant colonies so our ant
colonies intelligent are the bacteria in
our body in intelligent and then look
going to the physics world molecules and
the behavior at the quantum level of of
electrons and so on is are those kinds
of systems do they possess intelligence
like words where's the line that feels
compelling to you I don't know I mean I
think intelligence is a continuum and I
think that the ability to in some sense
have intention have a goal have a some
kind of self-awareness is part of it
so I'm not sure if you know it's hard to
know where to draw that line I think
that's kind of a mystery but I wouldn't
say that say that you know this the
planets orbiting the Sun her is an
intelligent system I mean I would find
that maybe not the right term to
describe that and this is you know
there's all this debate in the field of
like what's what's the right way to
define intelligence what's the right way
to model intelligence should we think
about computation should we think about
dynamics and should we think about you
know free energy and all of that stuff
and I think that it's it's a fantastic
time to be in the field because there's
so many questions and so much we don't
understand there's so much work to do so
are we are we the most special kind of
intelligence
this kind of you said there's a bunch of
different elements and characteristics
of intelligent systems and colonies are
his human intelligence the thing in our
brain is that the most interesting kind
of intelligence in this continuum
well it's interesting to us because
because it is us I mean interesting to
me yes and because I'm part of the you
know human but to understanding the
fundamentals of intelligence what I'm
yeah yeah Jerry is studying the human is
sort of if everything we've talked about
will you talk about in your book what
just the AI field this notion yes it's
hard to define but it's usually talking
about something that's very akin to
human intelligence to me it is the most
interesting because it's the most
complex I think it's the most self-aware
it's the only system at least that I
know of that reflects on its own
intelligence and you talk about the
history of AI and us in terms of
creating artificial intelligence being
terrible at predicting the future or the
Iowa tech in general so why do you think
we're so bad at predicting the future
are we hopelessly bad so no matter what
well there's this decade or the next few
decades every time I make a prediction
there's just no way of doing it well or
as the field matures we'll be better and
better at it I believe as the field
matures we will be better and I think
the reason that we've had so much
trouble is that we have so little
understanding of our own intelligence so
there's the famous story about Marvin
Minsky assigning computer vision as a
summer project to his undergrad students
and I believe that's actually a true
story ya know there's a there's a
write-up on it everyone should read it's
like a I think it's like a proposal
this describes everything done in that
project is hilarious because that I mean
you can explain it but for my sort of
recollection it described
is basically all the fundamental
problems of computer vision many of
which they still haven't been solved
yeah and and I don't know how far they
really expected to get but I think that
and and they're really you know Marvin
Minsky is super smart guy and very
sophisticated thinker but I think that
no one really understands or understood
still doesn't understand how complicated
how complex the things that we do are
because they're so invisible to us you
know to us vision being able to look out
at the world and describe what we see
that's just immediate it feels like it's
no work at all so it didn't seem like it
would be that hard but there's so much
going on
unconsciously sort of invisible to us
that I think we overestimate how easy it
will be to get computers to do it and
sort of for me to ask an unfair question
you've done research you've thought
about many different branches of AI and
through this book widespread looking at
where AI has been where it is today what
if you were to make a prediction how
many years from now would we as a
society create something that you would
say achieved human level intelligence or
superhuman level intelligence that is an
unfair question a prediction that will
most likely be wrong so but it's just
your notion because okay I'll say I'll
say more than a hundred years more than
a hundred years and there I quoted
somebody in my book who said that human
level intelligence is a hundred Nobel
Prizes away which I like because it's a
it's a nice way to to sort of it's a
nice unit for prediction and it's like
that many fantastic discoveries have to
be made and of course there's no Nobel
Prize in
if we look at that hundred years your
senses
really the journey to intelligence has
to go through something something more
complicated as again to our own
cognitive systems understanding them
being able to create them in in the
artificial systems as opposed to sort of
taking the machine learning approaches
of today and really scaling them and
scaling them and scaling them
exponentially with both computing
hardware and and data that would be my
that would be my guess
you know I think that in in the the sort
of going along in the narrow AI that
these current the current approaches
will get better you know I think there's
some fundamental limits to how far
they're gonna get I might be wrong but
that's what I think but and there's some
fundamental weaknesses that they have
that I talked about in the book that
that just comes from this approach of
supervised learning we require requiring
sort of feed-forward networks and so on
it it's just I don't think it's a
sustainable approach to understanding
the world yeah I'm I'm personally torn
on it sort of I've everything read about
in the book and sort of we're talking
about now I agreed I agree with you but
I'm more and more depending on the day
first of all I'm deeply surprised by the
successful machine learning and deep
learning in general and from the very
beginning that when I was it's really
been many focus of work I'm just
surprised how far it gets
and I'm also think we're really early on
in these efforts of these narrow AI so I
think there will be a lot of surprise
off how far it gets
I think will be extremely impressed like
my senses everything I've seen so far
and we'll talk about autonomous driving
and so on I think we can get really far
but I also have a sense that we will
discover just like you said is that even
though we'll get really far in order to
create something like our own
intelligence is actually much farther
than we realized
right I think these methods are a lot
more powerful than people give them
credit for actually so that of course
there's the media hype but I think
there's a lot of researchers in the
community especially like not undergrads
right but like people who've been in AI
they're skeptical about how far deep
learning yet and I'm more and more
thinking that it can actually get
farther than I realize it's certainly
possible one thing that surprised me
when I was writing the book is how far
apart different people are in the field
are artisan their opinion of how how far
the field has come and what is
accomplished and what's what's gonna
happen next what's your sense of the
different who are the different people
groups mindsets thoughts in the
community about where AI is today yeah
they're all over the place so so there's
there's kind of the the singularity
transhumanism group I don't know exactly
how to characterize that approach which
is there as well yeah the sort of
exponential exponential progress we're
on the sort of almost at the the hugely
accelerating part of the exponential and
by in the next 30 years we're going to
see super intelligent AI and all that
and we'll be able to upload our brains
and that so there's that kind of extreme
view that most I think most people who
work in AI don't have they disagree with
that but there are people who who are
maybe don't aren't you know singularity
people but but they're they do think
that the current approach of deep
learning is going to scale and is going
to kind of go all the way basically and
take us to
ái or human-level AI or whatever you
want to call it and there's quite a few
of them and a lot of them like a lot of
the people I've met who work at big tech
companies in AI groups kind of have this
view that we're really not that far you
know just to linger on that point sort
of if I can take as an example like
Yannick kun I don't know if you know
about his work and so a few points
unless I do he believes that there's a
bunch of breakthroughs like fundamental
like Nobel Prizes there's yeah he did
still write but I think he thinks those
breakthroughs will be built on top of
deep learning right and then there's
some people who think we need to kind of
put deep learning to the side a little
bit as just one module that's helpful in
the bigger cognitive framework right so
so I think some what I understand yan
laocoön is rightly saying supervised
learning is not sustainable we have to
figure out how to do unsupervised
learning that that's going to be the key
and you know I think that's probably
true
I think unsupervised learning is going
to be harder than people think I mean
the way that we humans do it then
there's the opposing view you know that
there's a the the Gary Marcus kind of
hybrid view or where deep learning is
one part but we need to bring back kind
of these symbolic approaches and combine
them of course no one knows how to do
that very well which is the more
important part right to emphasize and
how do they how do they fit together
what's what's the foundation what's the
thing that's on top yeah the cake was
the icing right yeah then there's people
pushing different different things
there's the people the causality people
who say you know deep learning as its
formulated a completely lacks any notion
of causality and that's dooms it and
therefore we have to somehow give it
some kind of notion of cause
there's a lot of push from the more
cognitive science crowd saying we have
to look at developmental learning we
have to look at how babies learn we have
to look at intuitive physics all these
things we know about physics and it's
somebody kind of quipped we also have to
teach machines intuitive metaphysics
which means like objects exist causality
exists you know these things that maybe
were born with I don't know that that
they don't have the machines don't have
any of that you know they look at a
group of pixels and they maybe they get
10 million examples but they they can't
necessarily learn that there are objects
in the world so there's just a lot of
pieces of the puzzle that people are
promoting and with different opinions of
like how how how important they are and
how close we are to the you know we'll
put them all together to create general
intelligence looking at this broad field
what do you take away from it who is the
most impressive is that the cognitive
folks Gary Marcus camp the yawn camp son
supervising their self supervise there's
the supervisor and then there's the
engineers who are actually building
systems you have sort of the Andrey
Carpathia Tesla building actual you know
it's not philosophy it's real writing
systems that operate in the real world
what yeah what do you take away from all
all this beautiful yeah I don't know if
you know these these different views are
not necessarily mutually exclusive and I
think people like Jung McCune agrees
with the developmental psychology
causality intuitive physics etc but he
still thinks that it's learning like
end-to-end learning is the way to go
we'll take us perhaps all the way yeah
and that we don't need there's no sort
of innate
stuff that has to get built in this is
you know it's because no it's a hard
problem
I personally you know I'm very
sympathetic to the cognitive science
side because that's kind of where I came
in to the field I've become more and
more sort of an embodiment adherent
saying that you know without having a
body it's gonna be very hard to learn
what we need to learn about the world
that's definitely something like I'd
love to talk about in a little bit to
step into the cognitive world then if
you don't mind because you've done so
many interesting things if you look to
copycat taking a couple of decades step
back
you'd Douglas Hofstadter and others have
created and developed copycat more than
thirty years ago
ah that's painful here what is it what
is what is copycat it's a program that
makes analogies in an idealized domain
idealized world of letter strings so as
you say thirty years ago Wow
so I started working on it when I
started grad school in 1984 Wow and it's
based on Doug Hofstadter's ideas that
about that analogy is really a core
aspect of thinking I remember he has a
really nice quote in in in the book by
by himself and Emmanuel Sanders called
surfaces and essences I don't know if
you've seen that book but it's it's
about analogy he says without concepts
there can be no thought and without
analogies there can be no concepts so
the view is that analogy is not just
this kind of reasoning technique where
we go you know shoe is to foot as glove
as to what you know these kinds of
things that we have on IQ tests or
whatever that but that it's much deeper
much more pervasive in everything we do
in everything our language our thinking
our perception so we so he had a view
that was a very active perception idea
so the idea was that instead of having
kind of what a passive network in which
you have input that's being processed
through these feed-forward layers and
then there's an output at the end that
perception is really a dynamic process
you know we're like our eyes are moving
around and they're getting information
and that information is feeding back to
what we look at next influences what we
look at next and how we look at it and
so copycat was trying to do that kind of
simulate that kind of idea where you
have these agents it's kind of an agent
based system and you have these agents
that are picking things to look at and
deciding whether they were interesting
or not whether they should be looked at
more and and that would influence other
agents how do they interact so they
interacted through this global kind of
what we call the workspace so this
actually inspired by the old blackboard
systems where you'd have agents that
post information on a blackboard a
common blackboard this is like old very
old fashioned a set is that we're
talking about like in physical space is
a computer program computer programs
agents posting concepts on a blackboard
yeah we called it a workspace and it
it's the workspace is a data structure
the agents are little pieces of code
that you can think of them as detect
little detectors or little filters then
say I'm gonna pick this place to look
and I'm gonna look for a certain thing
and it's just the thing I I think is
important is it there so it's almost
like you know a convolution in way
except a little bit more general and
saying and then highlighting it on the
on the work in the workspace wasn't once
it's in the workspace how do the things
they're highlighted relate to each other
like what
so there's different kinds of agents
that can build connections between
different things so just to give you a
concrete example what copycat did was it
made analogies between strings of
letters so here's an example ABC changes
to a BD what does ijk change to and the
program had some prior knowledge about
the alphabet new the sequence of the
alphabet it you know had a concept of
letter successor of letter it had
concepts of sameness so it has some
innate things programmed in but then it
could do things like say discover that
ABC is a group of letters in succession
hmm and then it an agent can mark that
so the idea that there could be a
sequence of letters is that a new
concept that's formed or if that's a
concept that's a concept that's innate
sort of can you form new concepts or all
so in this program all the concepts of
the program were innate so cuz because
we weren't I mean obviously that limits
it quite up quite a bit but what we were
trying to do is say suppose you have
some innate concepts how do you flexibly
apply them to new situations right and
how do you make analogies let's step
back for a second so I really like that
quote that he said without concepts
there can be no thought and without
analogies that can be no concepts you
know in a Santa Fe presentation you said
that it should be one of the mantras of
AI yes and that you all see yourself
said how to form and fluidly use concept
is the most important open problem in AI
yes how to form and fluidly use concepts
is the most important open problem in AI
so let's what is the concept and what is
an analogy a concept is in some sense a
fundamental unit of thought so say we
have a concept
of a dog okay and a concept is embedded
in a whole space of concepts so that
there's certain concepts that are closer
to it or farther away from it are these
concepts are they really like
fundamental like we mention innate look
almost like XE o matic like very basic
and then there's other stuff built on
top of it or just include everything is
are they're complicated like you can
certainly have form new concepts right I
guess that's the question I'm asked yeah
can you form new concepts that our
company complex combinations of other
ago yes absolutely and that's kind of
what we we do you know learning and then
what's the role of analogies in that so
analogy is when you recognize that one
situation is essentially the same as
another situation and essentially is
kind of the key word there and because
it's not the same so if I say last week
I did a podcast interview in actually
like three days ago in Washington DC and
that situation was very similar to this
situation although it wasn't exactly the
same you know it was a different person
sitting across from me we had different
kinds of microphones the questions were
different
the building was different there's all
kinds of different things but really it
was analogous or I can say so by doing a
podcast interview that's kind of a
constant it's a new concept you know I
never had that concept before I mean and
I can make an analogy with it like being
interviewed for a news article in a
newspaper and I can say well you kind of
play the same role that the the
newspaper the reporter played it's not
exactly the same because maybe they
actually emailed me some written
questions rather than
and the writing the written questions
play the you know are analogous to your
spoken questions you know there's just
all kinds of this somehow probably
connects to conversations you have over
Thanksgiving dinner just general
conversations you could there's like a
thread you can probably take that just
stretches out in all aspects of life
that connect to this podcast I mean sure
conversations between humans sure and
and if I go and tell a friend of mine
about this podcast interview my friend
might say oh the same thing happened to
me you know let's say you know you ask
me some really hard question and I have
trouble answering it my friend could say
the same thing happened to me but it was
like it wasn't a podcast interview it
wasn't it was a completely different
situation and yet my friend is seen
essentially this the same thing you know
we say that very fluidly the same thing
happened to me essentially the same
thing we don't even say that right
things they imply it yes yeah and the
view that kind of what went into say
coffee cat that that whole thing is that
that that that act of saying the same
thing happened to me is making an
analogy and in some sense that's what's
underlies all of our concepts why do you
think analogy making that you're
describing is so fundamental to
cognition like it seems like it's the
main element action of what we think of
us cognition yeah so it can be argued
that all of this generalization we do
concepts and recognizing concepts in
different situations is done by analogy
that that's every time I'm recognizing
that say you're a person that's by
analogy because I have this concept of
what person is and I'm applying it to
you and every
time I recognize a new situation like
one of the things I talked about it in
the book was the the concept of walking
a dog that that's actually making an
analogy because all that you know the
details are very different so it's so
now--so reasoning could be reduced on to
sense your analogy making so all the
things we think of as like yeah like you
said perception so what's perception is
taking raw sensory input and it's
somehow integrating into our our
understanding of the world updating the
understanding and all of that has just
this giant mess of analogies that are
being made I think so yeah if you just
linger on it a little bit like what what
do you think it takes to engineer a
process like that for us in our
artificial systems we need to understand
better I think how how we do it how
humans do it and it comes down to
internal models I think you know people
talk a lot about mental models that
concepts are mental models that I can in
my head I can do a simulation of a
situation like walking a dog and that
there there's some work in psychology
that promotes this idea that all of
concepts are really mental simulations
that whenever you encounter a concept or
situation in the world or you read about
it or whatever you do some kind of
mental simulation that allows you to
predict what's going to happen to
develop expectations of what's going to
happen mm-hm so that's the kind of
structure I think we need is that kind
of mental model that and the in our
brain somehow these mental models are
very much inter connected again so a lot
of stuff we're talking about it they're
essentially open problems right so if I
ask a question I don't mean that you
would know the answer already just
hypothesizing but how big do you think
is the the network graph data structure
of concepts that's in our head like if
we're trying to build that ourselves
like it's we take it and that's one of
the things we take for granted we think
I mean that's why we take common sense
for granted within common sense is
trivial but how big of a thing of
concepts is on that underlies what we
think of as common sense for example
yeah I don't know and I'm not I don't
even know what units to measure it in
beautifully put right but but you know
we have you know it's really hard to
know we have what a hundred billion
neurons or something I don't know and
they're connected via trillions of
synapses and there's all this chemical
processing going on there's just a lot
of capacity for the stuff and their
informations encoded in different ways
in the brain it's encoded in chemical
interactions it's encoded and electric
like firing and firing rates and and
nobody really knows how it's encoded but
it just seems like there's a huge amount
of capacity so I think it's it's huge
it's just enormous and it's amazing how
much stuff we know yeah and but we know
and not just know like facts but it's
all integrated into this thing that we
can make analogies with yes there's a
dream of semantic web and there's
there's a lot of Dreams from expert
systems of building giant knowledge
bases or do you see a hope for these
kinds of approaches of building of
converting Wikipedia into something that
could be used in analogy making sure and
I think people have have made some
progress along those lines I mean people
have been working on this for a long
time but the problem is and this I think
was is is the problem of common sense
like people have been trying to get
these common sense networks here at MIT
there's this concept net project right
but the problem is that as I said most
of the knowledge that we have is
invisible to us it's not in Wikipedia
it's very basic things about you know
intuitive physics intuitive psychology
to ative metaphysics all that stuff if
you were to create a website that
described intuitive physics intuitive
psychology would it be bigger or smaller
than Wikipedia what do you think
I guess describe to whom no that's very
really good right yeah that's a hard
question because you know how do you
represent that knowledge is the question
right I can certainly write down F
equals MA and Newton's laws and a lot of
physics can be deduced from that but
that's probably not the best
representation of that knowledge for for
doing the kinds of reasoning we want a
machine to do so so I don't know it's
it's it's impossible to say and you know
the projects like there's a famous the
famous psych project right that Doug
Douglass Lynott did that was trying
still going I think it's still going and
if the the idea was to try and encode
all of common-sense knowledge including
all this invisible knowledge in some
kind of logical representation and it
just never I think could do any of the
things that he was hoping it could do
because that's just the wrong approach
of course that's what they always say
you know and then the history books will
say well the psych project finally found
a breakthrough in 2058 or something and
it did you know we're so much progress
has been made in just a few decades that
yeah okay knows what the next
breakthroughs will be it could be a
certainly a compelling notion what the
psych project stands for
I think Lenin was one of the early
people do say common sense is what we
need and that's what we need all this
like expert system stuff that is not
going to get you to AI you need common
sense and he basically gave up his whole
academic career to to go pursue that I
told my er that but I think that the
approach itself will not what do you
think is wrong with approach what kind
of approach would might be successful
well again he knows the answer right I
knew that you know one of my talks one
of the people in the audience's a
published lecture one of the people in
the audience said what AI companies are
you investing in advice I'm a college
professor extra funds to invest but also
like no one knows what's gonna work in
AI right that's the problem let me ask
another impossible question in case you
have a sense in terms of data structures
that will store this kind of information
do you think they've been invented yet
both in hardware and software or is
something else needs to be are we
totally you know I think something else
has to be invented I that's my guess is
the breakthroughs that's most promising
would that be in hardware and software
do you think we can get far with the
current computers or do we need to do
something you're saying I don't know if
Turing computation is gonna be
sufficient probably I would guess it
will I don't I don't see any reason why
we need anything else but so so in that
sense we have invented the hardware we
need but we just need to make it faster
and bigger and we need to figure out the
right algorithms and and the right sort
of architecture touring that's a very
mathematical notion when we try to have
to build intelligence it's not an
engineering notion where you throw all
that stuff
I guess I guess it is a it is a question
that their people have brought up this
question you know and when you asked
about like is our current Hardware will
our current Hardware work well turing
computation says that like our current
hardware is in principle a Turing
machine right so all we have to do is
make it faster and bigger but there have
been people like Roger Penrose if you
might remember that he said Turing
machines cannot produce intelligence
because intelligence requires continuous
valued numbers I mean that was sort of
my reading of his argument and quantum
mechanics and what else whatever you
know but I don't see any evidence for
that that we need new computation
paradigms but I don't know if we're you
know I don't think we're going to be
able to scale up our current approaches
to programming these computers what is
your hope for approaches like copycat or
other cognitive architectures I've
talked to the creator of sore for
example I've used that arm myself I
don't know if you're familiar with yeah
woody what do you think is what's your
hope of approaches like that in helping
develop systems of greater and greater
intelligence in the coming decades well
that's what I'm working on now is trying
to take some of those ideas and
extending it so I think there are some
really promising approaches that are
going on now that have to do with more
active generative models so this is the
idea of this simulation in your head a
concept when you if you want to when
you're perceiving a new a new situation
you have some simulations in your head
those are generative models they're
generating your expectations they're
generating predictions that's part of a
perception you haven't met the model
that generates a prediction then you
come
parrot with ya and then the difference
and you also that that generative model
is telling you where to look and what to
look at and what to pay attention to and
it I think it affects your perception
it's not that just you compare it with
your perception it it becomes your
perception in a way it is kind of a
mixture of that bottom-up information
coming from the world and your top-down
model being opposed in the world is what
becomes your perception so your hope is
something like that can improve
perception systems and that they can
understand things better yes understand
things yes what's the what's the step
was the analogy making step there well
there the the the idea is that you have
this pretty complicated conceptual space
you know you can talk about a semantic
network or something like that
with these different kinds of concept
models in your brain that are connected
so so let's let's take the example of
walking a dog we were talking about that
okay let's see I say see someone out on
the street walking a cat some people
walk their cats I guess this seems like
a bad idea but yeah so my model of my
you know there's connections between my
model of a dog and model of a cat and I
can immediately see the analogy of that
those are analogous situations but I can
also see the differences and that tells
me what to expect so also you know I
have a new situation so another example
with the walking the dog thing is
sometimes people I see people riding
their bikes with Elise holding a leash
and the dogs running alongside okay so I
know that the I recognize that as kind
of a dog walking situation even though
the person's not walking right and the
dogs not walking because I I have the
these these models that say okay
riding a bike
is sort of similar to walking or it's
connected it's a means of transportation
but I because they have their dog there
I assume they're not going to work but
they're going out for exercise and you
know these analogies help me to figure
out kind of what's going on what's
likely but sort of these analogies are
very human interpreter Bowl mm-hmm so
that's that kind of space and then you
look at something like the current deep
learning approaches they kind of help
you to take raw sensory information and
just to automatically build up
hierarchies of role you can even call
them concepts they're just not human
interpretive or concepts
what's your what's the link here do you
hope it's sort of the hybrid system
question how do you think that two can
start to meet each other what's the
value of learning in this systems of
forming of analogy making the the goal
of I you know the original goal of deep
learning in at least visual perception
was that you would get the system to
learn to extract features that at these
different levels of complexities may be
edge detection and that would lead into
learning you know simple combinations of
edges and then more complex shapes and
then whole objects or faces and this was
based on that the ideas of the
neuroscientists Hubel and Wiesel who had
seen laid out this kind of structure and
brain and I think that is that's right
to some extent of course people have
come found that the whole story is a
little more complex than that and the
brain of course always is and there's a
lot of feedback and so I see that
as absolutely a good brain inspired
approach to some aspects of perception
but one thing that it's lacking for
example is all of that feedback which is
extremely important the interactive
element do you mentioned the expectation
the sexual level go back and forth with
the the expectation the perception and
yes going back and forth so right so
that is extremely important and you know
one thing about deep neural networks is
that in a given situation like you know
they they're trained right they get
these weights everything but then now I
give them a new a new image let's say
yes they treat every part of the image
in the same way you know they apply the
same filters at each layer to all parts
of the image mm-hmm there's no feedback
to say like oh this part of the image is
irrelevant right I shouldn't care about
this part of the image or this part of
the image is the most important part and
that's kind of what we humans are able
to do because we have these conceptual
expectations there's a little bit work
in that there's certainly a lot more in
a tent what's under the called attention
in natural language processing knowledge
ease it's a that's exceptionally
powerful and it's a very just as you say
it's really powerful idea but again in
sort of machine learning it all kind of
operates in an automated way that's not
human it's not it's not also okay so
that yeah right it's not dynamic I mean
in the sense that as a perception of a
new example is being processed those
attentions weights don't change right so
I mean there's a this
kind of notion that there's not a memory
so you're not aggregating the idea of
the this mental model yes yeah he that
seems to be a fundamental idea there's
not a really powerful I mean there's
some stuff with memory but there's not a
powerful way to represent the world in
some sort of way that's deeper than and
it's it's so difficult because uh you
know neural networks do represent the
world they do have a mental model right
but it just seems to be shallow I like
it it's it's hard to it's it's hard to
criticize them at the fundamental level
to me at least it's easy to it's it's
easy to criticize and we'll look like
exactly you're saying mental models sort
of almost from a sec I'll put a
psychology head on say look these
networks are clearly not able to achieve
what we humans do with forming mental
models but analogy making so on but that
doesn't mean that they fundamentally
cannot do that like you can it's very
difficult to say that I mean I used to
me do you have a notion that the
learning approaches really I mean
they're going to not not only are they
limited today but they will forever be
limited in being able to construct such
mental models I think the idea of the
dynamic perception is key here the idea
that moving your eyes around and getting
feedback and that's something that you
know there's been some models like that
there's certainly recurrent neural
networks that operate over several time
steps and but the problem is that it
that the actual the recurrence is you
know basically the the feedback is to
the next time step is the entire hidden
state yes the network which which is it
that it that's that doesn't work very
well does he hit the the thing I'm
saying is mathematically speaking it has
the information in that recurrence to
capture everything it just doesn't seem
to work yeah so like my you know it's
like it's the same touring machine
question right
yeah maybe theoretically it computers
and anything that's throwing a universal
Turing machine can can be intelligent
but practically the architecture might
be very specific kind of architecture to
be able to create it so just I guess
it's sort of ask almost the same
question again is how big of a role do
you think deep learning needs will play
or needs to play in this in perception I
think deep learning as it's currently as
it currently exists you know will place
that kind of thing will play some role
and but I think that there's a lot more
going on in perception but who knows you
know that the definition of deep
learning I mean it this it's pretty
broad it's kind of an umbrella so what I
mean is purely sort of neural networks
yeah and a feed-forward neural networks
essentially or there could be recurrence
but yeah sometimes it feels like for us
I'll talk to Gary Marcus it feels like
the criticism of deep learning is kind
of like us birds criticizing airplanes
for not flying well or that they're not
really flying do you think deep learning
do you think it could go all the way
like you're looking things do you think
that yeah the brute force learning
approach can go all the way I don't
think so no I mean I think it's an open
question but I I tend to be on the
innate Ness side that there has that
there's some things that we've been
evolved to be able to learn and
that learning just can't happen without
them so so one example here's an example
I had in the book that that I think is
useful to me at least in thinking about
this so this has to do with the
deepmind's atari game playing program
okay and learned to play these Atari
video games just by getting input from
the pixels of the screen and it learned
to play the game break out thousand
percent better than humans okay that was
one of the results and it was great and
and it learned this thing where it
tunneled through the side of the the
bricks in the breakout game and the ball
could bounce off the ceiling and then
just wipe out bricks okay so there was a
group who did an experiment where they
took the paddle you know that you move
with the joystick and moved it up to
pixels or something like that and then
they they looked at a deep Q learning
system that had been trained on breakout
and said could it now transfer its
learning to this new version of the game
of course a human could but and it
couldn't maybe that's not surprising but
I guess the point is it hadn't learned
the concept of a paddle it hadn't
learned that it hadn't learned the
concept of a ball or the concept of
tunneling it was learning something you
know we caught we looking at it kind of
anthropomorphised it and said oh it
here's what it's doing and the way we
describe it but it actually didn't learn
those concepts and so because it didn't
learn those concepts it couldn't make
this transfer yes so that's a beautiful
statement but at the same time by moving
the paddle we also anthropomorphize
flaws to inject into the system that
will then flip out how impressed we are
by it what I mean by that is to me the
Atari games were to me deeply impressive
that that was possible at all so that
guy first pause on that and people
should look at that just like the game
of Go
which is fundamentally different to me
then then what deep blue did even though
there's still mighty calls distillate
research it's just everything in deep
mind is done in terms of learning
however limited it is still deeply
surprising to me yeah i i'm not i'm not
trying to say that what they did wasn't
impressive i think it was incredibly
impressive to me is interesting is
moving the path aboard just another love
another thing that needs to be learned
so like we've been able to maybe maybe
been able to through the current neural
networks learn very basic concepts that
are not enough to do this general
reasoning and it may be with more data i
mean the data that you know the
interesting thing about the examples
that you talk about and beautifully is
they it's often flaws of the data well
that's the question i mean i i think
that is the key question it whether it's
a flaw of the data or not or the mexico
the reason I brought up this example was
because you were asking do I think that
you know learning from data could go all
the way yes and that this was why I
brought up the example because I think
and this was is not at all to to take
away from the impressive work that they
did but it's to say that when we look at
what these systems learn do they learn
the human the things that we humans
consider to be the relevant concepts and
in that example
it didn't sure if you train it on a
movie you know the pat paddle being in
different places maybe it could deal
with maybe it would learn that concept
I'm not totally sure but the question is
you know scaling that up to more
complicated worlds to what extent could
a machine that only gets this very raw
data learn to divide up the world into
relevant concepts and I don't know the
answer but I would bet that that
without some innate notion that it can't
do it
yeah ten years ago a hundred percent
agree with you as the deal most experts
in a system but now I have a one but
like I have a glimmer of hope okay
have you no that's very nice and I think
I think that's what deep learning did in
the community is no no I still if I had
to bet all my money it's a hundred
percent deep learning will not takes all
the way but there's still other it still
I was so personally sort of surprised
mm-hmm why the Thar games by go by by
the power of self play of just yeah I'm
playing against you that I was like many
other times just humbled of how little I
know about what's possible you know yeah
I think fair enough self play is
amazingly powerful and you know that's
that goes way back to Arthur Samuel
Wright with his checker playing program
and that which was brilliant and
surprising that it did so well so just
for fun let me ask you a topic of
autonomous vehicles it's the area that
that I work at least these days most
closely on and it's also area that I
think is a good example that you use a
sort of an example of things we as
humans don't always realize how hard it
is to do it's like the the constant
trend AI but the different problems that
we think are easy when we first try them
and then realize how hard it is okay so
why you've talked about this autonomous
driving being a difficult problem more
difficult than we realize you must give
it credit for why is it so difficult one
of the most difficult parts in your view
I think it's difficult because of the
world is so open-ended as to what what
kinds of things can happen so you have
sort of what normally happens which is
just you drive along and nothing nothing
surprising happens and autonomous
vehicles can do the ones we have now
evidently can do really well on most
normal situations as long
as long as you know the weather is
reasonably good and everything but if
some we have this notion of edge cases
or or you know things in the tail of the
distribution you call it the long tail
problem which says that there's so many
possible things that can happen that was
not in the training data of the machine
that it won't be able to handle it
because it doesn't have common sense
right it's the old the paddle moved yeah
it's the paddle moved problem right and
so my understanding and you probably are
more of an expert than I am on this is
that current self driving car vision
systems have problems with obstacles
meaning that they don't know which
obstacles which quote unquote obstacles
they should stop for and which ones they
shouldn't stop for and so a lot of times
I read that they tend to slam on the
brakes quite a bit and the most common
accidents with self-driving cars are
people rear-ending them because they
were surprised they've warned expecting
the machine the car to stop yeah so
there's there's a lot of interesting
questions there whether because because
you mentioned kind of two things so one
is the the problem of perception of
understanding of interpreting the
objects that are detected right
correctly and the other one is more like
the policy the action that you take how
you respond to it so a lot of the cars
braking is a kind of notion of to
clarify there's a lot of different kind
of things that are people calling
autonomous vehicles but a lot the L for
vehicles with a safety driver are the
ones like way moe and cruise and those
companies they tend to be very
conservative and cautious so they tend
to be very very afraid of hurting
anything or anyone and getting in any
kind of accidents so their policy is
very kind of that it that results in
being exceptionally responsive to
anything that could possibly be an
obstacle right
right which which which the human
drivers around it it's unpredictably
yeah that's not a very human thing to do
caution that's not the thing we're good
at specially in driving we're in a hurry
often angry and etc especially in Boston
so and then there's of another and a lot
of times that's machine learning is not
a huge part of that it's becoming more
and more unclear to me how much you you
know sort of speaking to public
information because a lot of companies
say they're doing deep learning and
machine learning just attract good
candidates the reality is in many cases
it's still not a huge part of the the
perception this is this lidar there's
other sensors that are much more
reliable for obstacle detection and then
there's Tesla approach which is vision
only and there's I think a few companies
doing that protest the most sort of
famously pushing that forward and that's
because the lidar is too expensive right
well I mean yes but I would say if you
were to for free give to every test
vehicle I mean Elon Musk fundamentally
believes that lidar is a crutch right
fantasy said that that if you want to
solve the problem of machine learning
lidar is not should not be the primary
sensor is the belief okay the camera
contains a lot more information mm-hmm
so if you want to learn you want that
information but if you want to not to
hit obstacles you want like are it's
sort of it's this weird trade-off
because yeah it's sort of what Tesla
vehicles have a lot of which is really
the thing the price of the fallback the
primary fallback sensor is radar which
is a very crude version of lighter it's
a good detector of obstacles except when
those things are standing right the
stopped vehicle right that's why it had
problems with crashing into stop fire
trucks stop fire trucks right so the
hope there is that the vision sensor
would somehow catch that and infer
there's a lot of problems of perception
I they are doing actually some
incredible stuff in the almost like an
active learning space where it's
constantly taking edge cases and pulling
back in there's a state data pipeline
another aspect that is really important
that people are studying now is called
multitask learning which is sort of
breaking apart this problem whatever the
problem is in this case driving into
dozens or hundreds of little problems
that you can turn into learning problems
so this giant pipeline the you know it's
kind of interesting I've been skeptical
from the very beginning we've become
less and less skeptical over time how
much of driving can be learned I'm still
think it's much farther than then the
CEO of that particular company thinks it
will be but it it is costly surprising
that through good engineering and data
collection and active selection of data
how you can attack that long tail and
it's an interesting open question that
you're absolutely right there's a much
longer tail and all these edge cases
that we don't think about but it's this
it's a fascinating question that applies
to natural language in all spaces how
big how how big is that long tail right
and I mean not to linger on the point
but what's your sense in driving in
these practical problems of the human
experience can it be learned so the
current what are your thoughts are sort
of Elon Musk thought let's forget the
thing that he says it'd be solved in a
year but can it be solved in in a
reasonable timeline or do fundamentally
other methods need to be invented so I I
don't I think that ultimately driving so
so it's a trade-off in a way I you know
being able to drive and deal with any
situation that comes up does require
kind of full human
telogen sand even in humans aren't
intelligent enough to do it because
humans I mean most human accidents are
because the human wasn't paying
attention or the humans drunk or
whatever and not because they weren't
intelligent but not because they weren't
intelligent enough right whereas the
accidents with autonomous vehicles is
because they weren't intelligent enough
they're always paying attention so it's
a it's a trade off you know and I think
that it's a very fair thing to say that
autonomous vehicles will be ultimately
safer than humans because humans are
very unsafe it's kind of a low bar but
just like you said
the III I think he was get a bad rap
right cuz we're really good at the
common-sense thing yeah we're great at
the common-sense thing we're bad at the
paying atten thing being attached a
thing especially moral you know driving
is kind of boring and we have these
phones to play with and everything but I
think what what's gonna happen is that
for many reasons not just AI reasons but
also like legal and other reasons that
the the definition of self-driving is
going to change or autonomous is going
to change it's not going to be just I'm
gonna go to sleep in the back and you
just drive me anywhere
it's gonna be more certain areas are
going to be instrumented to have the
sensors and the mapping and all the
stuff you need for that that the
autonomous cars won't have to have full
common sense and they'll do just fine in
those areas as long as pedestrians don't
mess with them too much that's another
question I don't think we will have
fully autonomous self-driving in the way
that like most the average person thinks
of it for a very long time and just to
reiterate this is the interesting open
question that I think I agree with you
on is to solve fully
Thomas driving you have to be able to
engineer in common sense yes I think
it's an important thing to hear and
think about I hope that's wrong but I
currently I could agree with you that
unfortunately you do have to have to be
more specific sort of these deep
understandings of physics and yeah of
the way this world works and also the
human dynamics like you mentioned
pedestrians and cyclists actually that's
whatever that nonverbal communication is
some people call it there's that dynamic
that is also part of this common sense
right and we're pretty we humans are
pretty good at predicting what other
humans are gonna do and how are our
actions impacts the behaviors of yes
this is weird game theoretic dance that
we're good at somehow and work well the
funny thing is is because I've watched
countless hours of pedestrian video and
talked to people
we humans are also really bad at
articulating the knowledge we have right
which is a been a huge challenge yes so
you've mentioned embodied intelligence
what do you think it takes to build a
system of human level intelligence does
he need to have a body I'm not sure but
I I'm coming around to that more and
more and what does it mean to be I don't
mean to keep breaking on up yeah Laocoon
he looms very large yeah well he
certainly has a large personality yes he
thinks that the system needs to be
grounded meaning he needs to sort of be
able to interact with reality but it
doesn't think it necessarily need to
have a body
so when you think of what's the
difference I guess I want to ask when
you mean body do you mean you have to be
able to play with the world or do you
also mean like there's a body that you
that you have to preserve oh that's a
good question I haven't really thought
about that but I think both I would
guess because it's because I think you I
think intelligence it's so hard to
separate it from
self our desire for self-preservation
our emotions are all that non rational
stuff that kind of gets in the way of
logical thinking because we the way you
know if we're talking about human
intelligence or human level intelligence
whatever that means
a huge part of it is social that you
know we were evolved to be social and to
deal with other people and that's just
so ingrained in us that it's hard to
separate intelligence from that I I
think you know AI for the last 70 years
or however long has been around it it
has largely been separated there's this
idea that there's like it's kind of very
Cartesian there's this you know thinking
thing that we're trying to create but we
don't care about all this other stuff
and I think the other stuff is very
fundamental so there's idea that things
like emotion get in the way of
intelligence as opposed to being an
integral part and part of it so I mean
I'm Russian so romanticize the notions
of emotion and suffering and all that
kind of fear of mortality those kinds of
things so I I especially sort of by the
way did you see that there was this
recent thing going around the internet
of this so some I think he's a Russian
or some Slavic head had written this
thing a sort of anti the idea of super
intelligence mmm-hmm I forgot maybes
polish anyway so at all these arguments
and one one was the argument from Slavic
pessimism do you remember what the
argument is it's like nothing ever works
so what what do you think is the role
like that's such a fascinating idea that
the what we perceive as serve the limits
of human
of the human mind which is emotion and
fear and all those kinds of things are
integral to intelligence could could you
elaborate on that like what why is that
important
do you think for human level
intelligence at least the way the humans
work it's a big part of how it affects
how we perceive the world it affects how
we make decisions about the world it
affects how we interact with other
people it affects our understanding of
other people you know for me to
understand your what you're going what
you're likely to do I need to have kind
of a theory of mine and that's very much
a theory of emotions and motivations and
goals and and to understand that I you
know we have the this whole system of
you know mirror neurons you know I sort
of understand your motivations through
sort of simulating it myself so you know
it's not something that I can prove
that's necessary but it seems very
likely so ok you've written the op-ed in
New York Times titled we shouldn't be
scared by super intelligent AI and it
criticized a little bit just to rustle
in the boss room can you try to
summarize that articles key ideas so it
was spurred by a earlier New York Times
op-ed by Stewart Russell which was
summarizing his book called human
compatible and the article was saying
you know if we if we have super
intelligent AI we need to have its
values align with our values and it has
to learn about what we really want and
he gave this example what if we have a
super intelligent AI and we give it the
prob
of solving climate change and it decides
that the best way to lower the carbon in
the atmosphere is to kill all the humans
okay so to me that just made no sense at
all because a super intelligent AI first
of all thinking what trying to figure
out what what super intelligence means
and it doesn't it seems that something
that super intelligent can't just be
intelligent along this one dimension of
okay I'm gonna figure out all the steps
the best optimal path to solving climate
change and not be intelligent enough to
figure out that humans don't want to be
killed that you could get to one without
having the other and you know
boström in his book talks about the
orthogonality hypothesis where he says
he thinks that systems I can't remember
exactly what it is but it like a systems
goals and it's uh values don't have to
be aligned there's some orthogonal 'ti
there which didn't make any sense to me
so you're saying it in any system that's
sufficiently not even super intelligent
but is it approach greater greater
intelligence there's a holistic nature
that will sort of attention that will
naturally emerge
yes events it from sort of any one
dimension running away yeah yeah exactly
so so you know
boström had this example of the the
super intelligent AI that that makes
that turns the world into paperclips
because its job is to make paper clips
or something and that just as a thought
experiment didn't make any sense to me
well as a thought experiment or the
thing that could possibly be realized
either so so I think that you know what
my op ed was trying to do was say that
that intelligence is more complex than
these people are presenting it that it's
not like it's not so separable the
rationality the the values the emotions
all of that that it's the the view that
you could separate all these dimensions
and build the machine that has one of
these dimensions and it's super
intelligent in one dimension but it
doesn't have any of the other dimensions
that's what I was trying to criticize
that that that I don't believe that
so can I read a few sentences from
yoshua bengio who is always super
eloquent so he writes I have the same
impression as Melanie that our cognitive
biases are linked with our ability to
learn to solve many problems they may
also be a limiting factor for AI however
this is a may in quotes things may also
turn out differently and there's a lot
of uncertainty about the capabilities of
future machines but more importantly for
me the value alignment problem is a
problem well before we reached some
hypothetical super intelligence it is
already posing a problem in the form of
super powerful companies whose objective
function may not be sufficiently aligned
with humanity's general well-being
creating all kinds of harmful side
effects so he goes on to argue that at
you know the orthogonality and those
kinds of things the concerns of just
aligning values with the capabilities of
the system is something that might come
long before we reach anything like in
super intelligence so your criticism
it's kind of really nice as saying this
idea of super intelligence systems seem
to be dismissing fundamental parts of
what intelligence would take and then
you know kind of says yes but if we look
at systems that are much less
intelligent there might be these same
kinds of problems that emerge sure but I
guess the example that he gives there of
these corporations that's people right
those are people's values I mean we're
talking about people the corporations
are their value
are the values of the people who run
those corporations but the idea is the
algorithm that's right so does the
fundamental person that the fundamental
element of what does the bad thing as a
human being
yeah but the the algorithm kind of
controls the behavior this mass of human
beings which help whatever for a company
that's the outs of for example if it's
advertisement driving company that
recommends certain things and encourages
engagement so it gets money by
encouraging engagement and therefore the
company more and more it's like the
cycle that builds an algorithm that
enforces more engagement and made
perhaps more division in the culture and
so on so on again I guess the question
here is sort of who has the agency so
you might say for instance we don't want
our algorithms to be racist right and
facial recognition you know some people
have criticized some facial recognition
systems as being racist because they're
not as good on darker skin and lighter
skin okay but the agency there the the
the the actual algal recognition
algorithm isn't what has the agency it's
it's not the racist thing right it's
it's the that the I don't know the the
combination of the training data the
cameras being used I whatever but my
understanding of and I'll say I told
agree with Benjy oh there that he you
know I think there are these value
issues with our use of algorithms but my
understanding of what Russell's argument
was is more that the algorithm itself
has the agency now it's the thing that's
making the decisions and it's the thing
that has what we would call values yes
so whether that's just a matter of
degree you know it's hard it's hard to
say right because but I would say that's
sort of qualitatively different than a
face recognition neural network and to
broadly linger on that point if you look
at Elon Musk goes to a rustle or boström
people who are worried about existential
risks of AI however far into the future
the argument goes is it eventually
happens we don't know how far but it
eventually happens
do you share any of those concerns and
what kind of concerns in general do you
have a body I that approach anything
like existential threat to humanity so I
would say yes it's possible but I think
there's a lot more closer in existential
threats you had as you said like a
hundred years for so your times more
more than a hundred more than a hundred
years and so that maybe even more than
500 years I don't I don't know I mean
it's so the existential threats are so
far out that the future is the immune
there'll be a million different
technologies that we can't even predict
now that will fundamentally change the
nature of our behavior reality society
and so on before then I think so I think
so and you know we have so many other
pressing existential threats going on
new hangouts even their nuclear weapons
climate problems you know
poverty possible pandemics that you can
go on and on and I think though you know
worrying about existential threat from
AI is it's not the best priority for
what we should be worried about that
that's kind of my view because we're so
far away but I you know I I'm not I'm
not necessarily criticizing Russell or
boström or whoever for worrying about
that and I'm I think it's some some
people should be worried about it it's
it's certainly fine but I I was more
sort of getting at their their view of
intelligible intelligence is mmm-hmm so
I was more focusing on like their view
of the super intelligence then uh just
the fact of them worrying and the title
of the article was written by the the
New York Times editors I wouldn't have
called it that we shouldn't be scared by
super intelligent and no if you wrote it
be like we should redefine what you mean
by super in I actually said it said you
know something like super intelligence
is not is is not a sort of coherent idea
that's not like it's only New York Times
would put in and the follow-up argument
that Yoshio makes also not argument but
a statement and I've heard him say it
before and I think I agree he's kind of
has a very friendly way of phrasing it
is it's good for a lot of people to
believe different things yeah well no
but he's it's also practically speaking
like we shouldn't be like while your
article stands like Stuart Russell does
amazing work boström does amazing work
you do amazing work and even when you
disagree about the definition of super
intelligence or the usefulness of even
the term it's still useful to have
people that like use that term all right
and then argue it sir I
I absolutely agree with video there and
I think it's great that you know and
it's great that New York Times will
publish all this stuff that's right it's
an exciting time to be here what what do
you think is a good test of intelligence
IQ is is natural language ultimately a
test that you find the most compelling
like the the original or the what you
know the higher levels of the Turing
test kind of yeah yeah I still think the
original idea of the Turing test is a
good test for intelligence I mean I
can't think of anything better
you know the Turing tests the way that
it's been carried out so far has been
very impoverished if you will but I
think a real Turing test that really
goes into depth like the one that I
mentioned I talk about in the book I
talk about Ray Kurzweil and Mitchell
Kapoor have this bet right that that in
2029 I think is the date there a machine
will pass the Turing test and turn says
and they have a very specific like how
many hours many expert judges and all of
that and you know Kurzweil says yes
Kapoor says no we can't we only have
like nine more years to go to see I you
know if something a machine could pass
that I would be willing to call it
intelligent of course nobody will they
will say that's just a language model if
it does so you would be comfortable it's
a language a long conversation that well
yeah here I mean you're right because I
think probably to carry out that long
conversation you would literally need to
have deep common-sense understanding of
the world I think so and the
conversation is enough to reveal that so
another super fun topic of complexity
that you have worked on written about
let me ask the basic question what is
complexity so complexity is another one
of those terms like intelligence
it's perhaps overused but my book about
complexity was about this wide area of
complex systems studying different
systems in nature in technology in
society in which you have emergence kind
of like I was talking about with
intelligence you know we have the brain
which has billions of neurons and each
neuron individually could be said to be
not very complex compared to the system
as a whole but the system the the
interactions of those neurons and the
dynamics creates these phenomena that we
call we call intelligence or
consciousness you know that are we
consider to be very complex so the field
of complexity is trying to find general
principles that underlie all these
systems that have these kinds of
emergent properties and the the
emergence occurs from like underlying
the complex system is usually simple
fundamental interactions yes and the
emergence happens when there's just a
lot of these things interacting yes sort
of what and then most of science to date
can you talk about what what is
reductionism
well reductionism is when you try and
take a system and divide it up into its
elements whether those be cells or atoms
or subatomic particles whatever your
field is and then try and understand
those elements and then try and build up
an understanding of the whole system by
looking at sort of the sum of all the
elements so what's your sense whether
we're talking about intelligence or
these kinds of interesting complex
systems is it possible to understand
them in in a reductionist way it's just
probably the approach of most of science
today right
I don't think it's always possible to
understand the things we want to
understand the most so I don't think
it's possible to look at single neurons
and understand what we call intelligence
you know just look at sort of summing up
and the sort of the summing up is the
issue here that were you know that one
example is that the human genome alright
so there was a lot of work on excitement
about sequencing the human genome
because the idea would be that we'd be
able to find genes that underlies
diseases but it turns out that and I was
a very reductionist idea you know we
figure out what all the the parts are
and then we would be able to figure out
which parts cause which things but it
turns out that the parts don't cause the
things that we're interested in it's
like the interactions it's the networks
of these parts and so that kind of
reductionist approach didn't yield the
the explanation that we wanted would he
would use the most beautiful complex
system that you've encountered most
beautiful that you've been captivated by
is it sort of I mean for me that is the
simplest to be cellular automata oh yeah
so I was very captivated by cellular
automata and worked on cellular automata
for several years do you find it amazing
or is it surprising that such simple
systems such simple rules and cellular
Domino can create sort of seemingly
unlimited complexity yeah that was very
surprising to me I didn't make sense of
it how does that make you feel this is
just ultimately humbling or is there
hope to somehow leverage this into a
deeper understanding and even able to
engineer things like intelligence
it's definitely humbling how humbling in
that also kind of awe-inspiring that
it's that inspiring like part of
mathematics that these credible
simple rules can produce this very
beautiful complex hard to understand
behavior and that that's it's mysterious
you know and and surprising still but
exciting because it does give you kind
of the hope that you might be able to
engineer complexity just from from these
can you briefly say what is the Santa Fe
Institute its history its culture its
ideas its future stuff I've never
semester G I've never been but so has
been this in my - mystical place where
brilliant people study the edge of chaos
exactly so the Santa Fe Institute was
started in 1984 and it was created by a
group of scientists a lot of them from
Los Alamos National Lab which is about a
40-minute drive from the Santa Fe
Institute
they were mostly physicists and chemists
but they were frustrated in their field
because they felt so that their field
wasn't approaching kind of big
interdisciplinary questions like the
kinds we've been talking about and they
wanted to have a place where people from
different disciplines could work on
these big questions without sort of
being siloed into physics chemistry
biology whatever so they started this
Institute and this was people like
George Cowan who is a chemist in the
Manhattan Project and Nicholas
Metropolis who mathematician physicist
Murray gell-mann physicist nism so some
really big names here ken arrow an
economist Nobel prize-winning economist
and they started having these workshops
and this whole enterprise kind of grew
into this Research Institute that's
itself has been
kind of on the edge of chaos its whole
life because it doesn't have any it
doesn't have a significant endowment and
it's just been kind of living on
whatever funding it can raise through
donations and grants and however it can
you know business business associates
and so on but it's a great place it's a
really fun place to go think about ideas
from that you wouldn't normally
encounter I saw Sean Carroll so
physicists yeah yeah external faculty
and you mentioned that there's so
there's some external faculty and
there's people there's a very small
group of resident faculty maybe maybe
about ten who are there for five year
terms that can sometimes get renewed and
then they have some postdocs and then
they have this much larger on the order
of a hundred external faculty or people
come like me who come and visit for
various periods of time so what do you
think this is the future of the Santa Fe
Institute like what and if people are
interested like what what's there in
terms of the public interaction or
students or so on that's that could be a
possible interaction on the Santa Fe
Institute or its ideas yeah so there's a
there's a few different things they do
they have a complex system summer school
for graduate students and postdocs and
sometimes faculty attend to and that's a
four week very intensive residential
program where you go and you listen to
lectures and you do projects and people
people really like that I mean it's a
lot of fun they also have some specialty
summer schools there's one on
computational social science there's one
on
climate and sustainability I think it's
called there's a few and then they have
short courses where just a few days on
different topics they also have an
online education platform that offers a
lot of different courses and tutorials
from SFI faculty
including an introduction to complexity
course that I talk and there's a bunch
of talks to online from there's guest
speakers and so on they they host a lot
of yeah they have sort of technical
seminars and colloquia they all and they
have a community lecture series like
public lectures and they put everything
on their YouTube channel so you can see
it all watching douglas hofstadter
author of get olestra bach was your PhD
adviser he mentioned a couple times and
collaborator do you have any favorite
lessons or memories from your time
working with him that continues to this
day yes but just even looking back
through throughout your time working
with him so one of the things he taught
me was that when you're looking at a
complex problem to to idealize it as
much as possible to try and figure out
what are really what is the essence of
this problem and this is how like the
copycat program came into being was by
taking an analogy making and saying how
can we make this as idealized as
possible but still retain really the
important things we want to study and
that's really kept you know been a core
theme of my research I think and I
continue to try and do that and it's
really very much kind of physics
inspired Hofstadter was a PhD in physics
that was his background it's like first
principles kind of thinking like you
reduced to the the most fundamental
aspect of the problem yeah so there you
can focus on solving that fun than I
thought yeah and in AI you know that was
people used to work in these micro
worlds right like the blocks world was
very early important area in AI and then
that got criticized because they said oh
you know you can't scale that to the
real world and so people started working
on much like more real world like
problems but now there's been kind of a
return even to the blocks world itself
you know we've seen a lot of people who
are trying to work on
more of these very idealized problems or
things like natural language and common
sense so that's an interesting evolution
of those ideas so the perhaps the
block's world's represents the
fundamental challenges of the problem of
intelligence more than people realized
it might yeah is there sort of when you
look back at your body of work and your
life you've worked in so many different
fields is there something that you're
just really proud of in terms of ideas
that you've gotten chance to explore
create yourself so I am really proud of
my work on the copycat project I think
it's really different from what almost
everyone is done in AI I think there's a
lot of ideas there to be explored and I
guess one of the happiest days of my
life you know aside from like the births
of my children was the birth of copycat
when it actually started to be able to
make really interesting analogies and I
remember that very clearly you know it
was very exciting time well you kind of
gave life yes artificial so that's right
what in terms of what people can
interact I saw there's like a I think
it's called meta copy kinetic hat mad
cat and there's a Python three
implementation at if people actually
want to play around with it and actually
get into it and study it maybe integrate
into whether it's with deep learning or
any other kind of work they're doing
what what would you suggest they do to
learn more about it and to take it
forward in different kinds of directions
yeah so that there's a Douglas
Hofstadter's book called fluid concepts
and creative analogies talks in great
detail about copycat I have a book
called analogy making as perception
which is a version of my PhD thesis on
it
there's also code that's available that
you can get it to run I have some links
on my web page to where people can get
the code for it and I think that that
would really be the best way I get into
it yeah play with it well Melanie is a
honor talking to you I really enjoyed it
thank you so much for your time today
has been really great
thanks for listening to this
conversation with Melanie Mitchell and
thank you to our presenting sponsor cash
app downloaded use code Lex podcast
you'll get ten dollars and ten dollars
will go to first a stem education
nonprofit that inspires hundreds of
thousands of young minds to learn and to
dream of engineering our future if you
enjoyed this podcast subscribe on
youtube give it five stars an apple
podcast supported on patreon or connect
with me on Twitter and now let me leave
you some words of wisdom from Douglas
Hofstadter and Melanie Mitchell without
concepts there can be no thought and
without analogies there can be no
concepts and Melanie adds how to form
and fluidly use concepts is the most
important open problem in AI
thank you for listening and hope to see
you next time
you