Kind: captions
Language: en
the following is a conversation with
risto michelinen a computer scientist at
the university of texas at austin
and associate vice president of
evolutionary artificial intelligence at
cognizant
he specializes in evolutionary
computation
but also many other topics in artificial
intelligence
cognitive science and neuroscience quick
mention of our sponsors
jordan harbinger show grammarly belcampo
and indeed check them out in the
description to support this podcast
as a side note let me say that nature
inspired algorithms
from ant colony optimization to generic
algorithms to
cellular automata to neural networks
have always captivated my imagination
not only for their surprising power in
the face of long odds
but because they always opened up doors
to new ways of thinking about
computation
it does seem that in the long arc of
computing history
running toward biology not running away
from it
is what leads to long-term progress this
is
the lex friedman podcast and here is my
conversation
with risto mcelinen if we ran the
earth experiment this fun little
experiment we're on
over and over and over and over a
million times and watch the evolution of
life
as it uh pans out how much variation in
the outcomes of that evolution do you
think we would see
now we should say that you are a
computer scientist
that's actually not such a bad question
for computer scientists because
we are building simulations of these
things and we are simulating evolution
and that's a difficult question to
answer in biology but we can build a
computational model
and run it million times and actually
answer that question how much variation
do we see when we when we simulate it
uh and um you know that's a little bit
beyond what we can do today but
but i think that we will see some
regularities and it took evolution also
a really long time to get started and
then things
accelerated really fast uh towards the
end
but there are things that need to be
discovered and they probably will be
over and over again like manipulation
uh of objects uh opposable thumbs and
um and also some way to communicate
uh maybe orally like whether you have
speech it might be some other kind of
sound
and and decision making but also vision
uh
i has evolved many times various vision
systems have evolved
so we would see those kinds of solutions
i believe
emerge over and over again they may look
a little different but they
they get the job done the really
interesting question is would we have
primates would we have
humans or something that resembles
humans
uh and and would that be an apex of
evolution after a while
uh we don't know where we're going from
here but we certainly see a lot of tool
use and
and building our constructing our
environment so
i think that we will get that we get
some
evolution producing some agents that can
do that manipulate the environment and
build
what do you think is special about
humans like if you were running the
simulation
and you observe humans emerge like these
like tool makers they start a fire and
all stuff start running around building
buildings and then running for president
all those kinds of things
uh what would be how would you detect
that
because you're like really busy as the
creator of this evolutionary system
so you don't have much time to observe
like detect if
any cool stuff came up right how would
you detect humans
well you are running the simulation so
you also
put in visualization and measurement
techniques there so if you are looking
for
certain things like communication you'll
have detectors to find out whether
that's happening even if it's a lot
simulation
and i think that that's that's what what
we would do
we know roughly what we want intelligent
agents that communicate cooperate
manipulate
and we would build detections and
visualizations of those processes
yeah it and there's a lot of we have to
run it many times and
we have plenty of time to figure out how
we detect the interesting things
but also i think we do have to run it
many times because we don't quite know
what shape those will take and our
detectors may not be perfect for them
to begin with well it seems really
difficult to build the detector of
intelligent or intelligent conv
communication sort of uh if we take an
alien perspective observing earth are
you sure
that they would be able to detect humans
as the special thing wouldn't they be
already curious about other things
there's way more insects by body mass i
think than humans
by far and colonies
obviously dolphins is the most
intelligent uh creature on earth
we all know this so it could be the
dolphins that they detect
it could be the rockets that we seem to
be launching that could be the
intelligent creature they detect
uh it could be some other uh trees
trees have been here a long time i just
learned that sharks have been here
400 million years and that's longer than
trees have been here
so maybe it's the sharks they go by age
like there's a persistent thing
like if you survive long enough
especially through the mass extinctions
that could be the
the thing your detector is uh detecting
humans have been here for a short time
and we're just creating a lot of
pollution but so is the other creatures
i don't know you do you think you'd be
able to detect humans like how would you
go about detecting
in the computational sense maybe we can
leave humans behind
in the computational sense detect
interesting things
do you basically have to have a strict
objective function by which you measure
the performance of a system or can you
find curiosities and
interesting things yeah well i think
that the first
measurement would be to detect how much
of an effect you can have in your
environment so if you look at
look around we have cities and that is
constructed environments and that's
where a lot of people live most people
live
so that would be a good sign of
intelligence that you
don't just live in an environment but
you construct it to your liking yeah
and that's something pretty unique i
mean certainly birds build nests
but they don't build quite cities
termites build mounds and ice and things
like that
but the complexity of the human
construction cities i think would stand
out
even to an external observer of course
that's what a human would say
yeah and you know you can certainly say
that sharks are really smart because
they've been around so long and they
haven't destroyed the environment
which humans are about to do which is
not a very smart thing
uh but we'll get over it i believe
uh and and we can get over it by doing
some construction that actually is
benign
uh and maybe even enhances uh the um
resilience of nature so you mentioned
that this simulation that we run over
and over might
start so it's a slow start so do you
think uh
how unlikely first of all i don't know
if you think about this kind of stuff
but
how unlikely is step number zero
which is the springing up like the
origin of life on earth
and second how unlikely
is the anything interesting happening
beyond that
sort of like the start that that creates
all the rich complexity that we see on
earth today yeah
there are people who are working on
exactly that problem uh
from primordial soup how do you actually
get self-replicating yeah molecules
and they are very close uh with a little
bit of help you can make that happen
so we of course we know what we want so
they can set up the conditions and try
out conditions
that are conducive to that for evolution
to discover that
that took a long time for us to recreate
it probably won't take that long
and the next steps from there um
i think also with some hand-holding i
think we can make that happen
um but if with evolution what was really
fascinating was eventually the
runaway evolution of the brain that
created
humans and created well also other
higher animals
that that was something that happened
really fast and
that's a big question is that something
replicable is that something that
can happen and if it happens does it go
in the same direction
that is a big question to ask even in
computational
terms i think that it's relatively
possible to
come up here create an experiment where
we look at the primordial soup and the
first couple of steps
of multicellular organisms even but to
get something as complex as the brain
we don't quite know the conditions for
that and how to even get started and
whether we can get this kind of runaway
evolution happening
from a detector perspective
if we're observing this evolution what
do you think is the brain what do you
think is the
let's say what is intelligence so in
terms of the thing that makes humans
special we seem to be able to reason
we seem to be able to communicate but
the core of that is this
something in the broad category we might
call intelligence
so it's uh if you put your computer
scientists add on
uh is their favorite ways you like to
think about
that question of what is intelligence
well my goal is to create
agents that are that are intelligent
not to define what and and that that is
a way of defining it
and that means that it's some kind of an
um
object or or a program um
that has limited sensory and uh
effective capabilities interacting with
the world
and then also a mechanism for making
decisions
so with limited abilities like that can
it
survive um survival is the simplest goal
but it could
you could also give it other goals can
it multiply can it solve problems that
you give it
uh and that is quite a bit less than
human intelligence there are
animals would be intelligent of course
with that definition and you might have
even even some other forms of of life
even so what
so intelligence in that sense is a
survival um
skill uh given resources that you have
and using using your resources so that
you will stay around
do you think death mortality is
fundamental
to an agent so like there's a i don't
know if you're familiar there's a
philosopher named ernest becker who
wrote the
denial of death and his whole idea and
there's folks
psychologists cognitive scientists that
work on terror management theory
and they think that one of the special
things about humans is that we're able
to sort of
foresee our death right we can we can
realize not just as animals do sort of
constantly fear
in an instinctual sense respond to all
the dangers that are out there
but like understand that this ride ends
eventually yeah
and that in itself is the most is a
is the force behind all of the creative
efforts of human nature yeah that's
that's the philosophy
i think that makes sense a lot of sense
i mean animals probably don't think of
death the same way
but humans know that your time is
limited and you want to make it count
and you can make account in many
different ways but i think that has a
lot to do with creativity and the need
for humans
to do something beyond just surviving
and now going from that simple
definition to something that's
the next level i think that that could
be a second decision a second level of
definition that
um intelligence means something and you
do something that
stays behind you that's more than uh
your
existence um something you create
something that
um is useful for others is useful in the
future not just for yourself
and i think that that's a nice
definition of intelligence in a
next level uh and it's also nice because
it doesn't require
that they are humans or biological they
could be artificial
agents that intelligence they could they
could achieve those kind of goals
so particular agent the uh the ripple
effects of
of their existence on the entirety of
the system
is significant so like they leave a
trace where there's like uh
yeah like ripple effects it's the but
see then you go back to the
the butterfly with the flap of a wing
and then you can uh trace a lot of uh
like nuclear wars and all the conflicts
of human history
somehow connected to that one butterfly
that created all the
the chaos so maybe that's not
maybe that's a very poetic way to think
uh that's something we humans in a
human-centric way want to
hope we have this impact
like that is the the the secondary
effect of our intelligence we've
had the long-lasting impact on the world
but maybe the entirety
of physics in the universe
has a very long lasting effect sure but
you can also think of it what if um like
the wonderful life what if you're not
here will somebody else do this is it
is it something that you actually
contributed because you had something
unique to compute
that contribute that's a pretty high bar
though uniqueness
yeah yeah so you know you have to be
mozart or something to actually reach
that level that nobody would have
developed that but other people might
have solved this equation
um if you didn't do it
but but also within limited scope i mean
during your lifetime or next year
you could contribute something that
unique that other people did not see
and um and then that could change
the way things move forward for a while
uh so i don't think we have to be mozart
to be
called intelligence but we have this
local effect that is
changing if you weren't there that would
not have happened and it's a positive
effect of course you want it to be a
positive effect do you think it's
possible to engineer in
to uh computational agents a fear of
mortality
like uh does that make any
any sense so there's a very trivial
thing whereas like
you could just code in a parameter which
is how long the life ends but
more of a fear of mortality
like awareness of the the way that
things end
and somehow encoding a complex
representation
of that fear which is like maybe as it
gets closer
you become more terrified i mean there
seems to be something really profound
about this fear that's not
currently encodable in a trivial way
into our programs
well i think you're you're referring to
the emotion of fear something
because we are cognitively we know that
we have limited lifespan
and most of us cope with it by just hey
that's what the world is like and i make
the most of it but
sometimes you can have a like a a fear
that's not healthy
that paralyzes you you can't do anything
uh and and uh somewhere in between
they're
not caring at all and and getting
paralyzed because of fear
is a normal response which is a little
bit more than just logic
and and it's emotion so now the question
is what good are emotions i mean they
are quite
uh complex and they are multiple
dimensions of emotions
and they probably do serve as survival
function heightened focus for instance
and fear of death might be a really good
emotion when you are in danger that you
recognize it
even even if it's not logically
necessarily easy to derive and you don't
have time for
that logical detection a deduction you
may be able to recognize the situation
is dangerous and this fear
kicks in and you all of a sudden
perceive the facts that are important
for that and i think that's generally is
the role of emotions is it allows you to
focus
what's relevant uh for your situation
and maybe if fear of death plays the
same kind of role
uh but if it consumes you and it's
something that you think
in normal life when you don't have to
then it's not healthy and then it's not
productive
yeah but it's fascinating to think how
to uh
incorporate emotion into a computational
agent
it almost seems like a silly statement
to make
but it perhaps seems silly because we
have such a poor understanding of the
mechanism of
emotion of fear of uh
i think at the core of it is another
word that we know
nothing about but say a lot which is
consciousness
do you ever in your work or like maybe
on a coffee break think about what the
heck is this thing
consciousness and is it at all useful in
our thinking about ai systems
yes it is an important question
you can build representations and
functions
i think into these agents that act like
emotions
and consciousness perhaps so i mentioned
emotions being something that allow you
to focus and pay attention
filter out what's important yeah you can
have that kind of a filter mechanism
and you can it puts you in a different
state your computation is in a different
state certain things don't really get
through and others
are heightened now you label that box
emotion i don't know if that means it's
an emotion but it acts
very much like we understand what
emotions are
and we actually did some work like that
um modeling
hyenas who were trying to steal a kill
from lions
which happens in africa i mean hyenas
are
quite intelligent but not really
intelligent
and they they have this behavior that's
more complex than anything else they do
they can band together if there's
about 30 of them or so uh they can uh
coordinate their effort so that they
push the lions away from a kill
even though the lions are so strong that
they could kill a lion
kill a hyena by by striking with a paw
but when they work together and
precisely time this attack the lions
will leave and they get the kill
and probably there are some
states like emotions that the hyenas go
through the first they
they call for reinforcements they really
want that kill but there's not enough of
them so they vocalize and
there's more peop more people more
hyenas that come around
and then they have two emotions they're
very afraid of the lion
so they want to stay away but they also
have a strong affiliation
between each other and then this is the
balance of the two emotions and
and also yes they also want the kill so
it's both rebelled and attractive and
then but then this
affiliation eventually is so strong that
when they move they move together they
act as a unit
and they they can perform that function
so there's an
interesting behavior that seems to
depend on these emotions
strongly and makes it possible um
important reactions
and i think a cr a critical
aspect of that the way you're describing
is emotion there is
a mechanism of social communication
of a social interaction maybe that
maybe humans won't even be that
intelligent or most
things we think of as intelligent
wouldn't be that intelligent without
the social component of interaction
maybe most
much of our intelligence is essentially
in our growth of social interaction
and maybe for the creation of
intelligent agents we have to be
creating
yes fundamentally social systems yes i i
strongly believe that's true
and uh yes the uh communication is
multifaceted i mean
they they vocalize and call for friends
but they also
rub against each other and they push and
they do all kinds of
gestures and so on so they known act
alone and i don't think people act alone
uh very much either at least normal most
of the time
and social systems are so strong for
humans
that i think we build everything on top
of these kind of structures and
one interesting theory around that
bigger this theory for instance for
language but language origins is
that where did language come from and
and it's a plausible theory that first
came social systems that
you have different roles in a society
and then those roles are exchangeable
that you know i scratch
your back you scratch my back you can
exchange roles
and once you have the brain structures
that allow you to understand actions in
terms of roles that can be changed
that's the basis for language for
grammar and now you can start
using symbols to refer to uh objects in
the world
and you have this flexible structure so
there's a social structure
that's fundament fundamental for
language to develop
now again then you have language you can
you can refer to things that are not
here
right now and that allows you to then
build all the
all the good stuff about uh planning for
instance and
building things and so on so yeah i
think that very strongly
uh humans are social and that gives us
ability to structure the world
but also as a society we can do so much
more because we don't
one person does not have to do
everything you can have different roles
and together achieve a lot more and
that's also something we see in
computational simulations today i mean
we have multi-agent systems that
can perform tasks this fascinating uh
demonstration marco dorico i think it
was
um these robots little robots that had
to navigate through an environment and
there were
there were things that are dangerous
like maybe a
a big chasm or some kind of groove a
hole
and they could not get across it but if
they grab each other
with their gripper they formed a robot
that was much longer on
the team and this way they could get
across that yeah
so this is a great example of how
together we can achieve things we
couldn't otherwise like the hyenas
you know alone they couldn't but as a
team they could uh and i think humans do
that all the time we're really good at
that
yeah and the way you describe the the
system of hyenas
it almost sounds algorithmic like the
the problem with humans is they're so
complex
it's hard to think of them as algorithms
but with hyenas
there's a it's simple enough to where it
feels like
um at least hopeful that it's possible
to create
computational systems that mimic that
yeah that's exactly why why we looked at
that
as opposed to humans um like i said they
are intelligent but they are not quite
as intelligent
intelligent as say baboons which would
learn a lot and would be much more
flexible that hyenas
are relatively rigid in what they can do
and therefore you could look at this
behavior like this is a breakthrough in
evolution about to happen
yes that they've discovered something
about social structures communication
about cooperation and and it might then
spill over to other things too
yeah in thousands of years in the future
yeah
i think the problem with baboons and
humans is probably too much is going on
inside the head
we won't be able to measure it if we're
observing the system with hyenas is
probably
easier to observe the actual decision
making and the various
motivations that are involved yeah they
are visible
and we can even um quantify possibly
their
emotional state because they leave
droppings behind
and there are chemicals there that can
be associated with uh
with neurotransmitters and we can
separate what emotions they might have
experienced in the last 24 hours yeah
what to use the most beautiful speaking
of hyenas
uh what do you use the most beautiful uh
nature inspired
algorithm in your work that you've come
across something
maybe early on in your work or maybe
today
i i think that evolution computation is
the most amazing method so what
fascinates me
most is that with computers is that you
can
you can get more out than you put in i
mean you can write a piece of code
and your machine does what you told it i
mean
this happened to me in my freshman year
i it did something very simple and i was
just amazed i was blown away that it
would
it would get the number and it would
compute the result and i didn't have to
do it myself
very simple but if you push that a
little further
you can have machines that learn and
they might learn patterns
and already say deep learning neural
networks they can learn to recognize
objects sounds um patterns that humans
have trouble
with and sometimes they do it better
than humans and that's so fascinating
and now if you take that one more step
you get something like evolution
algorithms
that discover things they create things
they come up with solutions that you did
not think of
and that just blows me away it's so
great that we can build
systems algorithms that can be in some
sense
smarter than we are that they can
discover solutions that we might miss
a lot of times it is because we have as
humans we have certain biases we expect
the solutions to be a certain way
and you don't put those biases into the
algorithm so they are more free to
explore
and evolution is just absolutely
fantastic explorer
and that's what what really is
fascinating yeah i think uh
i get made fun of a bit because i
currently don't have any kids
but you mentioned programs i mean
um do you have kids yeah so maybe you
could speak to this but there's a magic
to the creation
creative process like i uh with spot
the boston dynamic spot but really any
robot that i've ever worked on
it just feels like the similar kind of
joy i imagine i would have as a father
not the same perhaps level but like the
same kind of wonderment like
exactly this which is like you know what
you had to do
initially uh to get this thing going
let's speak on the computer science side
like what the program looks like
but something about it uh doing
more than what the program was written
on paper
is like that somehow connects to the
magic
of this entire universe like that's
that's like i i feel like i found god
every time i like it's like uh because
you're you've really
created something that's living yeah
even if it's
it has a life of its own has the
intelligence of its own it's beyond what
you actually thought
yeah and that is i think it's exactly
spot on that's exactly what it's about
uh you created something and has a
ability to
uh live its life and and do good things
and um
you just gave it a starting point so in
that sense i think it's that may be part
of the joy actually
uh you see but you mentioned creativity
in this context uh
especially in the context of
evolutionary computation
so you know we don't often think of
algorithms as creative
so how do you think about creativity
yeah
algorithms absolutely can be creative um
they can
come up with solutions that you don't
think about i mean creativity can be
defined
a couple of requirements has to be new
it has to be useful and it has to be
surprising
and those certainly are true with say
evolution computation discovering
solutions so maybe
an example for instance we did this
collaboration with mit media lab kelp
harvest
lab where they had a
hydroponic food computer they called it
environment that was completely computer
controlled nutrients water
light temperature everything is
controlled now
um what do you do if you can control
everything
farmers know a lot about how to do how
to make plants grow in their own batch
of land
but if you can control everything it's
too much and it turns out that we don't
actually know very much about it
so we built a system evolution
optimization system
together with a surrogate model of how
plants grow
and let this system explore recipes
on its own and initially now we were
focusing on light
uh how strong what wavelengths how long
the light was on
um and we put some boundaries which we
thought were reasonable
for instance that there was um at least
six hours of darkness like night because
that's what we have in the world
and very quickly um the system evolution
pushed all the recipes to that limit uh
we were trying to grow basil
um and we had initially have some 200
300 recipes
exploration as well as known recipes but
but now we are going beyond that and
everything was like pushed at that limit
so
we look at it and say well you know we
can easily just change it let's have it
your way
and it turns out uh the system
discovered that bazel does not need to
sleep
uh 24 hours lights on and it will thrive
it will be bigger it will be tastier and
this was a big surprise
not just to us but also the biologists
in the team
that anticipated that this is some
constraints that
that are in the world for a reason it
turns out that evolution did not have
the same bias
and therefore it discovered something
that was creative it was surprising it
was useful and it was new
that's fascinating to think about like
the things we think that are fundamental
to
living systems on earth today whether
they're actually fundamental or they
somehow shape
uh fit the constraints of the system and
all we'll have to do is just remove the
constraints
do you ever think about um
i don't know how much you know about
bringing computer interfaces in your
link
the the idea there is you know our
brains are very limited
and if we just allow we plug in
we provide a mechanism for a computer to
speak with the brain
so you're thereby expanding the
computational power of the brain
the possibilities there sort of from a
very high level philosophical
perspective is limitless but i wonder
how limitless it is are the constraints
we have like features that are
fundamental to our intelligence
or is this just like this weird
constraint in terms of our brain size
and skull
and uh lifespan and the
senses it's just the weird little like
quirk of evolution and if we just open
that up like add much more senses
add much more computational power the uh
intelligence will be will expand
exponentially
do you have a do you have a sense about
constraints the relationship of
evolution computation to the constraints
of the environment
um well at first i'd like to comment on
on that like
changing the inputs uh to human brain uh
yes and
flexibility of of the brain i think
there's a lot
of that uh there are experiments that
are done in animals like megan kasir
um the mit is switching the um auditory
and visual
information and going going to the wrong
part of the cortex and the animal
was still able to hear and perceive the
visual environment
and there are kids that are born with
severe disorders and sometimes they have
to remove
half of the brain like one half and they
still grow up they have the functions
migrate to the other parts
there's a lot of flexibility like that
so i think it's quite possible to
hook up the brain with different kinds
of sensors for instance
and something that we don't even quite
understand or have today
on different kind of wavelengths or or
whatever they are um
and then the brain can learn to make
sense of it and that i think
is um this good hope that these
prosthetic devices for instance work
not because we make them so good and so
easy to use but the brain adapts to them
and can
learn to take advantage of them um and
so in that sense if there's a trouble a
problem i think that brain
can be used to correct it now going
beyond what we have today can you get
smarter
that's really much harder to do uh
giving
the brain more more input probably might
overwhelm it it would
have to learn to filter it and focus um
and in order to use the information
effectively
and augmenting intelligence with some
kind of external devices like that
might be difficult uh i think but
replacing what's lost
i think is quite possible right so our
intuition
allows us to sort of imagine that we can
replace what's been lost
but expansion beyond what we have i mean
we are already one of the most
if not the most intelligent things on
this earth right so it's hard to imagine
um if the brain can hold up with an
order of magnitude greater set of
information
thrown at it if it can do if you can
reason through that
part of me this is the russian thing i
think is uh i tend to think that the
limitations is where the
the superpower is that
you know immortality and uh
huge increase in bandwidth of uh
information by connecting computers with
the brain is not going to produce
greater intelligence
it might produce lesser intelligence so
i don't know there's something about
the scarcity being essential
to uh um fitness or performance
but that could be just because we're so
uh
limited no exactly you make do with what
you have but you can
uh you don't have to pipe it directly to
the brain i mean we already have
devices like phones where we can look up
information at any point
yeah and that can make us more
productive you don't have to argue about
i don't know what happened in that
baseball game or whatever it is because
you can look it up right away and i
think in that sense
we can learn to utilize tools and that's
what we
we have been doing for a long long time
um
so and we are already the brain is
already drinking from the water
fire hose like vision there's way more
information
in the vision that we actually process
so brain is already good at
identifying what matters yeah and
that we can switch that from vision to
some other wavelength or some other kind
of modality but i think that the same
processing principles probably still
apply uh but
but also indeed this uh ability to
uh have information more accessible and
more relevant i think
can enhance what we do i mean kids today
at school
they learn about dna i mean things that
we discovered just a couple of years ago
and it's already common knowledge and we
are building on it and we don't see
a problem where um where
there's too much information that we can
absorb and learn maybe people become a
little bit more
narrow in what they know they are in one
field
but this information that we have
accumulated it is passed on and people
are picking up on it
and they are building on it so it's not
like we have reached the point of
saturation
um we have still this process that
allows us to be selective
and decide what's interesting um i think
still works
even even with the more information we
have today yeah it's fascinating to
think about
like wikipedia becoming a sensor like uh
so the fire hose of information from
wikipedia so it's like you
integrate it directly into the brain to
where you're thinking like you're
observing the world with all of
wikipedia directly piping into your
brain
so like when i see a light i immediately
have like the history of
who invented electricity like integrated
very quickly into so just the way you
think about the world might be very
interesting
if you can integrate that kind of
information what are your thoughts if i
could ask
uh on uh early steps on that on the
neurolink side i don't know if you got a
chance to see but
uh there's a monkey playing pong
yeah through the brain computer
interface and uh
the dream there is sort of you're
already replacing the thumbs essentially
that you would use to play video game
the dream is to be able to
increase further the the interface by
which you interact with the computer
are you impressed by this are you
worried about this what are your
thoughts as a human
i think it's wonderful i think it's
great that we could we could do
something like that i mean you can
there are devices that read your eeg for
instance
and and you and humans can learn um
to control things using using just their
thoughts
in that sense and i i don't think it's
that different i mean those signals
would go to limbs they would go to
thumbs
uh now the same signals go through a
sensor to some computing system
it still probably has to be built on
human terms
uh not to overwhelm them but but utilize
what's there and sense the right kind of
um
patterns that are easy to generate but
oh that
i think is really quite possible and and
wonderful and could be very much more
efficient
is there so you mentioned surprising
being a characteristic of
uh creativity is there something you
already mentioned a few examples but
is there something that jumps out at you
as was particularly surprising
from the various evolutionary
computation systems you've worked on
the solutions that were
come up along the way not necessarily
the final solutions but
maybe things have even discarded is
there something that just jumps to mind
it it happens all the time i mean
evolution is so
creative uh so good at discovering
uh solutions you don't anticipate a lot
of times they are
taking advantage of something that you
didn't think was there like a bug in the
software for instance
a lot of there's a great paper uh the
community put it together
about uh surprising anecdotes about
evolution computation
a lot of them are indeed in some
software environment there was an
a loophole or a bug and the system uh
utilizes that by the way for people who
want to read it's kind of fun to read
it's called the surprising creativity of
digital evolution a collection of
anecdotes from the evolutionary
computation and
artificial life research communities and
there's just a bunch of stories from all
the seminal figures in this community
uh you have a story in there uh that
released to you at least
on the tic-tac-toe memory bomb so can
you can you uh
i guess uh describe that situation if
you think that's yeah
that was that's a quite a bit smaller
scale than our
um basil doesn't need to sleep surprised
but
but it was actually done by students in
my class um
in a neural net evolution computation
class uh there was an assignment
uh it was perhaps a final project where
people built game playing uh ai
it was an ai class uh and this one and
and it was for tic-tac-toe or
five in a row in a large board uh and uh
this one team
evolved a neural network to make these
moves
uh and um they set it up the evolution
they didn't really know what would come
out
but it turned out that they did really
well evolution actually won the
tournament
and most of the time when it won it went
because the other teams
crashed and then when we look at it like
what was going on
was that evolution discovered that if it
makes a move that's really really far
away
like millions of squares away
the other teams the other programs just
expanded memory
in order to take that into account until
they ran out of memory and crashed
and then you win a tournament by
crushing all your opponents
i think that's quite a profound example
which it probably applies to most
games from even a game theoretic
perspective
that sometimes to win you don't have to
be
better within the rules of the game you
have to come up with ways to
break your opponent's uh
brain as a human like not through
violence but through
some hack where the brain just is not um
you're basically uh how would you put it
you're
the you're going outside the constraints
of where the brain is able to
to function expectations of your
opponent i mean yeah this was even
kasparov pointed that out that when the
blue was playing against kasparov that
it was not playing
the same way as kasparov expected uh and
this has to do with
you know being not having the same
biases uh
and that's that's really one of the
strengths of of the
ai approach yeah can you at a high level
say
what are the basic mechanisms of
evolutionary computation
algorithms that use something that
could be called an evolutionary approach
like how does it work
uh what are the connections to the it's
what are the echoes of the connection to
is biological
a lot of these algorithms really do take
motivation from biology but they are
carry catches you try to essentialize it
and take the elements that
you believe matter so in evolution
computation
it is the creation of variation and then
the selection
upon that so the creation of variation
you have to have some mechanism that
allow you to create new individuals that
are very different from
what you already have that's the
creativity part and then you have to
have some way of measuring how well they
are doing
uh and using the uh that measure to
select
uh who goes to the next generation and
you continue so first you
also you have to have some kind of
digital representation
of an individual that can be then
modified so i guess humans
i mean biological systems have dna and
all those kinds of things and so you
have to have similar kind of encodings
in a computer program yes and that is a
big question how do you encode
these individuals so there's a genotype
which is that encoding and then
a decoding mechanism just gives you the
phenotype which is the actual individual
that then
performs the task and in an environment
can be evaluated how good it is so even
that mapping is a big question and how
do you do it
but typically the representations are
either they are strings of numbers or
they are some kind of trees those are
something we know very well in computer
science and we try to do that but they
and you know dna in some sense is also a
sequence
um and it's a string um
so it's not that far from it but dna
also has many other aspects
that we don't take into account
necessarily like this folding and
and interactions that are other than
just the sequence itself
and lots of that is not yet captured and
we don't know whether they are
really crucial um evolution biological
evolution has produced
wonderful things but if you look at them
it's not necessarily the case that every
piece is irreplaceable
and essential there's a lot of baggage
because you have to construct it and it
has to go through various stages and we
still have
appendiciti appendix and we have
tailbones and things like that that are
not really that useful
if you try to explain them now it would
make no sense
very hard but if you think of us as
productive evolution you can see where
they came from they were useful
at one point perhaps and and no longer
are but they're still there
so um that process is complex
uh and your representation should
support it
uh and that is quite difficult
if if we are limited with strings or
trees
and then we are pretty much limited what
can be constructed
and one thing that we are still missing
in evolution computation in particular
is
what we saw in biology major transitions
so that you go from for instance single
cell to multi-cell organisms
and eventually societies there are
transitions of level of
selection and level of what a unit is
and that's something we haven't captured
in evolution computation yet
does that require a dramatic expansion
of the representation
is that what that that is most likely it
does but it's
quite we don't even understand it in
biology very well where it's coming from
so it would be really good to look at
major transitions in biology try to
characterize them a little bit more in
detail what the processes are
how how does a so like a unit a cell
is no longer evaluated alone it's
evaluated as part of a community
organism right even though it could
reproduce now it can't
alone and it has has to have its
environment so there's a
there's a push to another level at least
the selection
and how do you make that jump to this
yes how do you make the jump that's part
of the algorithm
yeah yeah so we haven't really seen that
in computation
um yet and there are certainly attempts
to have
open-ended evolution things that could
add more complexity and start
selecting at a higher level but it is
still not
um quite the same as going from single
to multi to
society for instance in in biology so so
there essentially would be as opposed to
having one agent
those agent all of a sudden
spontaneously decide to
then be together and then your entire
system
would then be treating them as one agent
something like that
some kind of weird merger building but
also so you mentioned i think you
mentioned selection so basically there's
an
agent and they don't get to live on
if they don't do well so there's some
kind of measure of what doing well is
and isn't
and uh does the mutation come into play
at all
in the process and what the world does
it serve yeah so
in again back to what the computational
mechanisms of evolution computation are
so
um the way to create variation uh you
can take multiple individuals too
usually but
but you could do more and you exchanged
the part of the representation you do
some kind of recombination
it could be crossover for instance um
in biology you do have dna strings that
that are
cut and put together again we could do
something like that
um and it seems to be that in biology
crossover is really the workhorse in in
biological evolution in computation
we tend to rely more on mutation
and that is making random changes into
parts of the chromosome you try to be
intelligent and target certain areas of
it
and make the mutations also
follow some principle like you collect
statistics of performance and
correlations
and try to make mutations you believe
are going to be helpful
that's where evolution computation has
moved in the last
20 years i mean evolution competition
has been around for 50 years but a lot
of the recent
um success comes from mutation comes
from comes from
using statistics it's like the rest of
machine learning based on statistics we
use similar tools
to guide evolutionary computation and in
that sense it has diverged
a bit from biological evolution and
that's one of the things i think
we could look at again
having a weaker selection more crossover
large populations
more time and maybe a different kind of
creativity would come out of it
we are very impatient in evolution
competition today we want answers right
now right quickly
and every if no somebody doesn't perform
kill it yeah
uh and biological evolution doesn't work
quite that way
uh and and it's more patient yes much
more patient
so i guess we need to add some kind of
mating some kind of like dating
mechanisms like marriage may be in there
so to uh in into our algorithms to
improve the the
the the combination mechanism as opposed
to all mutation doing all of the work
yeah and many ways of being successful
you know
usually in every competition we have one
goal you know play
this game really well uh and compared to
others but
in biology there are many ways of being
successful you can build niches you can
be stronger faster larger or smarter
or you know eat this or eat that or you
know so
so there are many ways to solve the same
problem of survival
and that then breeds creativity um
and um it allows more exploration and
eventually you get
solutions that are perhaps more creative
rather than trying to go from
initial population directly or more or
less directly to your
maximum fitness which you measure that's
just one metric
so in a broad sense
before we talk about newer evolution
do you see evolutionary computation as
more effective than deep learning in
certain contexts
machine learning broadly speaking maybe
even
supervised machine learning i don't know
if you want to draw any kind of lines
and distinctions and
borders where they rub up against each
other kind of thing or one is more
effective than the other in the current
state of things
yes of course they are very different
and they address different kinds of
problems and
the deep learning has been really
successful in domains where we have a
lot of data
and that means not just data about
situations but also what the right
answers were
so labeled examples or they might be
predictions may be weather prediction
where the data itself
becomes labels what happened what the
weather was today and what will
be tomorrow so they are very effective
deep learning methods on that kind of
tasks
but there are other kinds of tasks where
we don't really know what the right
answer is
uh game playing for instance but many
robotics tasks and
actions in the world decision making um
and actual practical applications like
treatments and healthcare
or investment in stock market many tasks
are like that
we will we don't know and we'll never
know what the optimal answers were
and there you need different kinds of
approaches reinforcement learning is one
of those
uh reinforcement learning comes from
biology as well
agents learn during their lifetime they
eat berries and sometimes they get sick
and then they don't
and get stronger and then that's how you
learn
and evolution is also a mechanism like
that
at a different time scale because you
have a population not an individual
during its lifetime
but an entire population as a whole can
discover um
what works and there you can afford
individuals that don't work out
they will you know everybody dies and
you have a next generation and it will
be better than the previous one
so that's that's the big difference
between these methods they apply
to different kinds of problems um and
um in particular there's often a
comparison that's kind of interesting
and important between reinforcement
learning and evolution and computation
and initially um reinforcement learning
was about
individual learning during the lifetime
and evolution is more
engineering you don't care about the
lifetime you don't care about all the
individuals that are tested you only
care about the final result
the last one the best candidate that
evolution produced
in that sense they also apply to
different kinds of problems
and now that boundary is starting to
blur
a bit you can use evolution as an online
method and reinforcement learning to
create engineering solutions but that's
still
roughly the distinction and
from the point of view what algorithm
you want to use
if you have something where there is a
cost for every trial reinforcement
learning might be your choice
now if you have a domain where you can
use a surrogate perhaps
so you don't have much of a cost for
trial
and you want to have surprises you want
to explore more broadly
then this population-based method is
perhaps a better choice because you you
can try things out that you wouldn't
afford when you're doing reinforcement
there's very few things as entertaining
as watching either evolution competition
or reinforcement learning teaching a
simulated robot to walk
i maybe there's a
higher level question that could be
asked here but
do you find this whole space in of
applications in the robotics
interesting for evolution computation
yeah yeah very much
um and indeed that's the fascinating
videos of that
and that's actually one of the examples
where you can contrast the difference
so between reinforcement learning
evolution yes so
if you have a reinforcement learning
agent it tries to be
conservative because it wants to walk as
long as possible and be stable
but if you have evolutionary computation
it can afford
these agents that go haywire they
fall flat on their face and they could
take a step and then they jump and then
again fall flat yeah and eventually what
comes out of that
is something like a falling that's
controlled yeah
and you take another step another step
and you no longer fall
instead you run you go fast so that's a
way of discovering
something that's hard to discover step
by step incrementally
because you can afford these
evolutionists
dead ends although they are not entirely
dead ends in the sense that they can
serve as stepping stones
when you take two of those put them
together you get something that works
even better
and that is a great example of of this
kind of discovery yeah learning to walk
is a
is fascinating i talked quite a bit to
russ tedron because mit
there's a there's a community of folks
who who just roboticists who love
the elegance an