Kind: captions
Language: en
the following is a conversation with
jeff hawkins a neuroscientist
seeking to understand the structure
function
and origin of intelligence in the human
brain he previously wrote the seminal
book on the subject
titled on intelligence and recently
a new book called a thousand brains
which presents a new theory of
intelligence
that richard dawkins for example has
been
raving about calling the book quote
brilliant and exhilarating
i can't read those two words and not
think of him saying it in his british
accent
quick mention of our sponsors codecademy
bio optimizers
expressvpn a-sleep and blinkist check
them out in the description to support
this podcast
as a side note let me say that one small
but powerful idea that jeff hawkins
mentions in his new book
is that if human civilization were to
destroy itself
all of knowledge all our creations will
go with
us he proposes that we should think
about how to save that knowledge
in a way that long outlives us whether
that's on earth
in orbit around earth or in deep space
and then to send messages that advertise
this backup of human knowledge
to other intelligent alien civilizations
the main message of this advertisement
is not
that we are here but that we were
once here this little difference
somehow was deeply humbling to me that
we may
with some non-zero likelihood destroy
ourselves and that an alien civilization
thousands or millions of years from now
may come across this knowledge store
and they would only with some low
probability
even notice it not to mention be able to
interpret it
and the deeper question here for me is
what information in all of human
knowledge is even essential
does wikipedia capture it or not at all
this thought experiment
forces me to wonder what are the things
we've accomplished
and are hoping to still accomplish that
will outlive us
is it things like complex buildings
bridges cars
rockets is it ideas like science physics
and mathematics
is it music and art is it
computers computational systems or even
artificial intelligence systems
i personally can't imagine that aliens
wouldn't already have
all of these things in fact much more
and much better to me the only unique
thing we may have
is consciousness itself and the actual
subjective experience of suffering
of happiness of hatred of love
if we can record these experiences in
the highest resolution directly from the
human brain
such that aliens will be able to replay
them that
is what we should store and send as a
message
not wikipedia but the extremes of
conscious experiences
the most important of which of course is
love
this is the lex friedman podcast and
here is my conversation
with jeff hawkins we previously
talked over two years ago do you think
there's still neurons in your brain
that uh remember that conversation that
uh remember me
and got excited like there's a lex
neuron in your brain that just like
finally has a purpose
i do remember our conversation or i have
some memories of it
and i formed additional memories of you
in the meantime
um i wouldn't say there's a neuron or a
neurons in my brain that know you
there are synapses in my brain that have
formed
that reflect my knowledge of you and the
model i have of you in the world and
whether the exact same synapses were
formed two years ago it's hard to say
because these things come and go all the
time but
we know from one thing to know about
brains is that when you think of things
you often erase the memory and rewrite
it again so
yes but i have a memory of you and i
have that's instantiated in synapses
there's a simpler way to think about it
like so you have we have a model
of the world in your head and that model
is continually being updated
i updated this morning you offered me
this water you said it was from the
refrigerator
i remember these things and so we and so
the model includes where we live the
places we know the words the
objects in the world it's a monstrous
model and it's constantly being updated
and people are just part of that model
so we're animals or other physical
objects
so our events we've done so um
it's there's no special in my mind
special place for the memories of humans
i mean
obviously i know you know i know a lot
about my wife
um but and friends uh and so on but
it's not like a special place for humans
over here but we model everything and we
model other people's behaviors too so if
i said you're a
copy of your mind in my mind it's just
because i know how humans
i've learned how humans behave and um
and i've learned some things about you
and that's part of my world model well i
just also mean
the collective intelligence of the human
species
i wonder if there's something
fundamental to the brain that enables
that so modeling other humans with their
ideas
you're actually jumping into a lot of
big topics like collective intelligence
is a separate topic that a lot of people
like to talk about we can talk about
that
uh but um and so that's interesting like
you know we're not just individuals we
live in society and so on
but from our research point of view and
so again let's just talk
we study the neocortex it's a sheet of
neural tissue it's about 75
of your brain it runs on this very
repetitive algorithm
it's a very repetitive circuit and so
you can apply that algorithm to lots of
different problems but it's all
underneath it's the same thing we're
just building this model
so from our point of view we wouldn't
look for these special circuit someplace
buried in your brain that might be
related to
other you know understanding of the
humans it's more like
you know how do we build a model of
anything how do we understand anything
in the world and humans are just
another part of the things we understand
so there's nothing uh there's nothing to
the brain that
knows the emergent phenomena of
collecting the intelligence well i
certainly know about that i've heard the
terms i've read
no but that's right right well okay
right as an idea well i think we have
language which is is
sort of built into our brains and that's
a key part of collective intelligence so
there are some you know prior
assumptions about the world we're going
to live in when we're born we're not
just
a blank slate um and so you know did we
evolve
to take advantage of those situations
yes but again
we study only part of the brain the
neocortex there's other parts of the
brain are very much
involved in societal interactions and
human emotions and
um and how we interact and even societal
um
issues about you know how we are how we
interact with other people when we
support them when we're greedy and
things like that i mean certainly the
brain
is a great place where to study
intelligence i wonder if it's the
fundamental
uh atom of intelligence well i would say
it's
it's it's absolutely an essential
component even if you believe in
collective intelligence as
um hey that's where it's all happening
that's what we need to study which i
don't believe that by the way i think
it's really important but i don't think
that is the thing
um but even if you do believe that then
you have to understand how the brain
works in doing that
um it's you know it's more like we are
intelligent and
we are intelligent individuals and
together we are much more magnified our
intelligence we can do things that we
couldn't do individually
but even as individuals we're pretty
damn smart and
we can model things and understand the
world and interact with it
so um to me if you're going to start
some place you need to start
with the brain then you could say well
how do brains interact with each other
and what is the nature of language and
how do we
share models that i've learned something
about the world how do i share it with
you which is really what you know
sort of communal intelligence is i know
something you know something
we've had different experiences in the
world i've learned something about
brains maybe i can impart that to you
you've learned something about
you know whatever physics and you can
part that to me
but it also comes down to even just the
epistemological question of
well what is knowledge and how do you
represent it in the brain right
and it's not that's where it's going to
reside right or in our writings
it's obvious that human collaboration
human interaction
is how we build societies right but some
of the things you
talk about and work on
some of those elements of what makes up
an intelligent
entity is there with a single person oh
absolutely i mean
it'd be we can't deny that the brain is
the core element here in
in uh at least i can't i think it's
obvious the brain is the core element in
all theories of intelligence
uh it's where knowledge is represented
it's where knowledge is created
we interact we share we build upon each
other's work
but uh without a brain you'd have
nothing you know there would be no
intelligence without brains
and so um so that's where we start
i got into this field because i just was
curious as to who i am
you know how you know how do i think
what's going on in my head when i'm
what i'm thinking what does it mean to
know something you know i can ask what
it means for me to know something
independent of how i learned it from
you or from someone else or from society
so what does it mean for me to know that
i have a model of you in my head what
does it mean to know i know what this
microphone does and how it works
physically even though i can't see it
right now
how do i know that what does it mean how
the neurons do that at the
fundamental level of neurons and
synapses and so on those are
really fascinating questions and uh i'm
happy to
be just happy to understand those if i
could
so in your um in your new book
you talk about our brain our mind as
being made up of many brains
uh so the book is called the thousand
brains a thousand brain theory of
intelligence what is the key idea of
this book
uh the book has three sections
and it has sort of maybe three big ideas
so the first section is all about what
we've learned about the neurocortex and
that's the thousand brains theory
just did we complete the picture the
second section is all about ai and the
third section is about the future of
humanity
so the thousand brains theory
the the big idea there if i had to
summarize into one big idea
is that we think of the the brain the
neocortex is learning this model of the
world
but what we learned is actually there's
tens of thousands of independent
modeling systems going on and so each
what we call a column in the cortex is
about 150
000 of them is a complete modeling
system so
it's a collective intelligence in your
head in some sense so the thousand
brains theory says
well where do i have knowledge about you
know this coffee cup where is the model
of this
cell phone it's not in one place it's in
thousands of separate models that are
complementary and they communicate
with each other through voting so this
idea that we have we feel like we're one
person
you know that's our experience we can
explain that but reality there's
lots of these like almost like little
brands like but they're
they're sophisticated modeling systems
about 150 000 of them in each of
the human brain and that's a totally
different way of thinking about
how the neural cortex is structured than
we or anyone else thought of even just
five years ago
so you mentioned you started this
journey
and just looking in the mirror trying to
understand who you are
so if you have many brains who are you
then
so it's interesting we have a singular
perception right you know we think oh
i'm just
here i'm looking at you but it's it's
composed of all these things like
there's sounds and there's
and there's uh this vision and there's
touch and
all kinds of inputs yeah we have the
singular perception and what the
thousand brain theory says we have these
models that are visual models we have a
lot of models of auditory models models
of toxin models and so on
but they vote and so um they send in the
cortex you can think about these columns
as that like little grains of
rice 150 000 stacked next to each other
and each one is its own little modeling
system
but they have these long-range
connections that go between them
and we call those voting connections or
voting neurons
um and so the different columns
try to reach the consensus like what am
i looking at okay you know
each one has some ambiguity but they
come to a consensus oh there's a water
bottle i'm looking at
um we are only consciously able to
perceive the voting
we're not able to perceive anything that
goes on under the hood
so the voting is what we're we're aware
of
the results of the vote yeah the
velocity well it's it's you can imagine
it this way
we were just talking about eye movements
a moment ago so as i'm looking at
something my eyes are moving about three
times a second
and with each movement a completely new
input is coming into the brain it's not
repetitive it's not shifting it around
it's completely new
i'm totally unaware of it i can't
perceive it but yet if i looked at the
neurons in your brain they're going on
and off i don't know
but the voting neurons are not the
voting neurons are saying you know
we all agree even though i'm looking at
different parts of this is a water
bottle right now
and that's not changing and it's in some
position and
and pose relative to me so i have this
perception of the water bottle about two
feet away from me at a certain pose to
me
um that is not changing that's the only
part i'm aware of i can't be aware of
the fact that the inputs
from the eyes are moving and changing
and all this others happening
so these long range connections are the
part we can be conscious of
the individual activity in each column
is doesn't go anywhere else it doesn't
get shared anywhere else it doesn't
there's no way to extract it
and talk about it or extract it and even
remember it to say oh
yes i can recall that um so but these
long-range connections are the things
that are accessible to language
and to our you know it's like the
hippocampus or our memories you know
our short-term memory systems and so on
so we're not aware of
95 or maybe it's even 98 of what's going
on in your brain
we're only aware of this sort of stable
somewhat stable
voting outcome of all these things that
are going on underneath the hood
so what would you say is the basic
element in the thousand
brains theory of intelligence of
intelligence
like what's the atom of intelligence
when you think about it
is it the individual brains and then
what is a brain
well let's let's can we just talk about
what intelligence is first
and then and then we can talk about the
elements are so in my
in my book intelligence is the ability
to learn
a model of the world so to build
internal to your head
a model that represents the structure of
everything you know
to know what this is a table and that's
a coffee cup and this is a gooseneck
lamp and all this
to know these things i have to have a
model in my head i just don't look at
them and go what is that
i already have internal representations
of these things in my head
and i had to learn them i wasn't born of
any of that knowledge
you were you know we have some lights in
the room here i you know that's not part
of my evolutionary heritage right it's
not in my genes
so um we have this incredible model and
the model includes not only what things
look like and feel like but where they
are relative to each other and how they
behave
i've never picked up this water bottle
before but i know that if i took my hand
on that blue thing and i turn it it'll
probably make a funny little sound as
the little plastic things detach
and then it'll rotate and it'll look a
certain way it'll come off how do i know
that right because i have this model in
my head
so the essence of intelligence as our
ability to learn a model and the more
sophisticated our model is
the smarter we are not that there is a
single intelligence because
you can know about you know a lot about
things that i don't know and i know
about things you don't know
and we can both be very smart but we
both learn the model of the world
through interacting with it
so that is the essence of intelligence
then we can ask ourselves what are the
mechanisms in the brain
that allow us to do that and what are
the mechanisms of learning not just the
neural mechanisms what is the general
process but how we learn a model
so that was a big insight for us it's
like what are the what is the actual
things that how do you learn this stuff
it turns out you have to learn it
through movement
um you can't learn it just by that's how
we learn we learn through movement we
learn
um so you build up this model by
observing things and touching them and
moving them and
walking around the world and so on so
either you move or the thing moves
somehow yeah you obviously can learn
things just by reading a book something
like that but
think about if i were to say oh here's a
new house yeah i want you to learn
you know what do you do you have to walk
you have to walk from room to the room
you have to open the doors
look around see what's on the left
what's on the right as you do this
you're building a model in your head
it's just that's what you're doing you
can't just sit there and say i'm going
gonna grock the house
no you know or you could you don't even
want to sit there and read some
description of it right
yeah you literally physically
interactive the same with like a
smartphone if i want to
learn a new app i touch it and i move
things around i see what happens when i
when i do things with it so that's the
basic way we learn in the world and by
the way when you say model
you mean something that can be used for
prediction in the future
it's it's used for prediction and for
behavior
and planning right um and does a pretty
good job in doing so
yeah here's the way to think about the
model a lot of people get hung up on
this so
um you can imagine an architect making a
model of a house
right so there's a physical model that's
small and why do they do that
well we do that because you can imagine
what it would look like from different
angles you could say okay
look at them here look in there and you
can also say well how how far to get
from
from the garage to the to the swimming
pool or something like that right you
can imagine looking at this you can say
what would be the view from this
location so we built these physical
models to let you
imagine the future and imagine that
behaviors
now we can take that same model and put
it in a computer so we now
today they'll build models of houses and
a computer and they
and they do that using a set of um
we'll come back to this term in a moment
reference frames but eventually you
assign a reference frame for the house
and you assign different things for the
house in different locations
and then the computer can generate an
image and say okay this is what it looks
like in this direction
the brain is doing something remarkably
similar to this surprising
um it's using reference frames it's
building these it's similar to a model
in a computer
which has the same benefits of building
a physical model it allows me to say
what would this thing
look like if it was in this orientation
what would likely happen if i push this
button
i've never pushed this button before or
how would i accomplish something i want
to
i want to um convey a new idea i've
learned
how would i do that i can imagine in my
head well i could talk about it
i could write a book i could do some
podcasts
i could um you know maybe tell my
neighbor
you know and i can imagine the outcomes
of all these things before i do any of
them
that's what the model lets you do it
let's just plan the future and imagine
the
consequences of our actions prediction
you asked about prediction
prediction is not the goal of the model
prediction is an inherent property of it
and it's how the model corrects itself
so prediction is fundamental to
intelligence
it's fundamental to building a model and
the model's intelligent
and let me go back and be very precise
about this prediction you can think of
prediction two ways one is like
hey what would happen if i did this
that's the type of prediction um
that's a key part of intelligence but
using predictions like oh what's this
this is this water bottle gonna feel
like when i pick it up
you know and that doesn't seem very
intelligent but the way to think one way
to think about intelligence prediction
is
it's a way for us to learn where our
model is wrong
so if i picked up this water bottle and
it felt hot i'd be very surprised
or if i picked up was very light it
would be very i'd be surprised or
if i turned this top and it didn't i had
to turn the other way i'd be surprised
and so almost might have a prediction
like okay i'm gonna do it i'll drink
some water
i'm okay okay do this there it is i feel
opening right what if i had to turn it
the other way or what if it it split in
two
then i say oh my gosh i i misunderstood
this i didn't have the right model of
this thing
my attention would be drawn to i'll be
looking at it going well how the hell
did that happen
you know why did it open up that way and
i would update my model
by doing it just by looking at it and
playing around with that update and say
this is a new type of water bottle
but you so you're talking about sort of
uh
complicated things like a water bottle
but this also applies for just
basic vision just like seeing things
it's almost like a precondition of just
perceiving the world is predicting
it's just everything that you see is
first passed through your prediction
everything you
see and feel in fact this this is the
insight i had
uh back in the late 80s uh and excuse me
early 80s
and um another people reach the same
idea is that
every sensory input you get not just
vision but touch
and hearing you have an expectation
about it
and um a prediction sometimes you can
pick very accurately sometimes you can't
i can't predict what next word is going
to come out of your mouth but as you
start talking about
better and better predictions and if you
talk about some topics i'd be very
surprised
so i have this sort of background
prediction that's going on all the time
for all my senses again the way i think
about that
is this is how we learn it's it's more
about how we learn
it's the test of our understanding our
predictions are our test
did is this really a water bottle if it
is i shouldn't see
you know a little finger sticking out
the side and if i saw a little finger
stick and i was like what the hell is
going on
you know that's not normal um i mean
that's
fascinating that just let me linger on
this
for a second i it really honestly feels
that prediction is fundamental
to everything uh to the way our mind
operates
to intelligence so like it's just a
different way to see
intelligence which is like everything
starts at prediction
and prediction requires a model you
can't predict something unless you have
a model of it right but the action is
prediction it's like the
the thing the model does is prediction
and but it also yeah and you but
you can then extend it to things like uh
what would happen
if i took this today i went and did this
what would be like that
or how you can extend predictions like
oh i want to get a promotion at work
um what action should i take and you can
say if i did this i predict what might
happen if i
spoke to someone i predict what might
happen so it's not just low level
predictions yeah it's all prediction
it's all predictions like this
black box so you can ask basically any
question low level or highlight so we
start off with that observation it's all
it's like this non-stop prediction and i
write about this in the book about
and then we ask how do neurons actually
make predictions
physically like what does the neuron do
when it makes a prediction and
um what the neural tissue does when it
makes predictions and then we ask what
are the mechanisms by how we build a
model that allows you to make prediction
so we started with prediction as sort of
the fundamental
research agenda if in some sense like
and say well we understand how the brain
makes predictions
we'll understand how it builds these
models and how it learns and that's core
of intelligence so it was like it was
the key that got us in the door
to say that is our research agenda
understand predictions
so in this whole process where does
intelligence
originate would you say so
it if we look at things that are
much less intelligent to humans and you
start to build up a human the process of
evolution
where is this magic thing that uh
has a prediction model or a model that's
able to predict
that starts to look a lot more like
intelligence is there a place where
richard dawkins wrote an introduction to
your uh to your book an excellent
introduction
i mean it puts a lot of things into
context
and it's funny just looking at parallels
for your book and
darwin's origin of species so darwin
wrote about the origin
of species so
what is the origin of intelligence well
we have a theory about it and it's just
that it's a theory
theory goes as follows as soon as living
things
started to move they're not just
floating in sea they're not just
a plant you know grounded some place as
soon as they started the move
there was an advantage to moving
intelligently to moving in certain ways
and there's some very simple things you
can do you know bacteria or
single cell organisms can move towards a
source of gradient of food or something
like that
but an animal that might know where it
is and know where it's been and how to
get back to that place or an animal that
might
say oh there was a source of food
someplace how do i get to it or there
was a
danger how do i get to there was a mate
how do i get to them
um there was a big evolution advantage
to that so early on there was a pressure
to start
understanding your environment like
where am i
and where have i been and what happened
in those different places
so we still have this neural mechanism
in our brains um it's in in the in the
mammals it's in the
hippocampus and internal cortex these
are older parts of the brain
um and these are very well studied um
we build a map of the of our environment
so
these neurons in these parts of the
brain know where i am in this room and
where
the door was and things like that so a
lot of other
mammals have this all mammals have this
right and almost
any any animal that knows where it is
and get around must have some mapping
system must have some way of saying
i've learned a map of my environment i
have hummingbirds in my backyard and
they
and they go the same places all the time
they have to they must know where they
are they just know where they are when
they're
they're not just randomly flying around
they know they know particular flowers
they come back to
so we all have this and it turns out
it's
very tricky to get neurons to do this to
build a map of an environment it's just
and so we now know there's this these
famous studies that's still very active
about
place cells and grid cells and these
other types of cells in the older parts
of the brain
and how they build these maps of the
world it's really clever it's obviously
been under a lot of evolutionary
pressure over a long period of time to
get good at this
so animals not know where they are what
we think has happened
uh and there's a lot of evidence to
digest this is that that mechanism we
learn to map
like a space is
was repackaged the same type of neurons
was repackaged into a more compact form
and that became the cortical column and
it was
it was in some sense genericized if
that's a word it was turned into a very
specific thing about learning
maps of environments to learning maps of
anything
learning a model of anything not just
your space but coffee cups and so on
and it got sort of repackaged
into a more compact version a more
universal version
and then replicate it so the reason
we're so flexible is we have a very
generic version of this
mapping algorithm and we have 150 000
copies of it
sounds a lot like the progress of deep
learning
how so uh so take neural networks that
seem to work well for a specific task
compress them and multiply it
by a lot and then you just stack them on
top of it it's like the story of
transformers and uh yeah
but interesting networks they end up
you're replicating an element but you
still need the entire network to do
anything
right here what what's going on each
individual element is a complete
learning system
this is why i can take a human brain cut
it in half and it still works
it's it's pretty amazing it's
fundamentally distributed it's
fundamentally distributed complete
modeling systems
so but that's that's our story we like
to tell
i i i would guess it's it's likely
largely right um but you know it's
there's a lot of evidence supporting
that story this evolutionary story
the thing which brought me to this idea
is that the human brain
got big very quickly so that that
led to the proposal a long time ago that
well there's this common element just
instead of
creating new things it just replicated
something we also are extremely flexible
we can learn things that we had no
history about right and so that tells it
that the
learning algorithm is very generic it's
very kind of universal
because it's it doesn't assume any prior
knowledge about what it's learning
and so you combine those things together
and you say okay well how did that come
about where did that universal algorithm
come from it had to come from something
that wasn't universal it came from
something that was more specific
and so anyway this led to our hypothesis
that you would find grid cells and place
cell equivalents in the neocortex
and when we first published our first
papers on this theory
we didn't know of evidence for that it
turns out there was some but we didn't
know about it
uh and since then um so then we became
aware of evidence for grid cells in
parts of the neural cortex
and then now there's been new evidence
coming out there's some
interesting papers that came out just
january of this year so our one of our
predictions was
if this evolutionary hypothesis is
correct we would see grid cell place
cell equivalents cells that work like
them
through every column in the near cortex
and that's starting to be seen
what does it mean that uh why is it
important that they're present
because it tells us well we're asking
about the evolutionary origin of
intelligence right
so our theory is that these columns in
the cortex
are working on the same principles
they're modeling systems and it's hard
to imagine how neurons do this and so we
said
hey it's really hard to imagine how
neurons could learn these models of
things
we can talk about the details of that if
you want but
let's um but there's this other part of
the brain we know that learns models of
environments so could that mechanism to
learn to model this room be used to
learn a model the water bottle
is it the same mechanism so we said it's
much more likely the brain is using the
same mechanism
which case it would have these
equivalent cell types
so it's basically the whole theory is
built on the idea that
um these columns have reference frames
and they're learning these models
and these these grid cells create these
reference frames so it's it's basically
the major in some sense the major
predictive
part of this theory is that we will find
these equivalent mechanisms in
each column in the near cortex which
tells us that's that
that that's what they're doing they're
learning these sensory motor models
of the world so just we're pretty
confident
that would happen but now we're seeing
the evidence so the evolutionary process
nature does a lot of copy and paste and
see what happens yeah
yeah there's no direction to it but but
um it just found out like hey if i
took this these elements and and made
more of them what happens and let's hook
them up to the eyes and let's look up
the ears and
and um and that seems to work pretty
well yeah like for us
again just to take a quick step back to
our
conversation of collective intelligence
do you sometimes
see that as just another copy and paste
aspect is copying pasting these uh
brains and
humans and making a lot of them and then
creating
uh social structures that then almost
operates as a single brain
uh i wouldn't have said it but you said
it sounded pretty good
so to you the brain is fundamental is uh
is like uh
is its own thing right i mean our goal
is to understand how the neural cortex
works
we can argue how essential that is to
understand a human brain because it's
not the entire human brain
you can argue how essential that is to
understanding human intelligence
you can argue how essential it is to um
to uh you know a sort of communal
intelligence
um i i'm not i didn't our goal was to
understand the neocortex yeah so what is
the neural cortex and where does it fit
in um the various aspects of what the
brain does
like how important is it to you well
obviously
again we i mentioned again in the
beginning it's it's it's
about 70 to 75 of the volume of a human
brain
so it's you know it dominates our brain
in terms of size not in terms of number
of neurons but
in terms of size size isn't everything
jeff
i know but it's it's nothing it's
nothing
it's not that we know that all
high-level vision
hearing and touch happens in the air
context we know that all language
occurs and is understood in the
neurocortex whether that's spoken
language written language sign language
with
language of mathematics language of
physics music
math you know we know that all
high-level planning and thinking occurs
in the new york cortex
if i were to say you know what part of
your brain designed a computer
and understands programming and and
creates music it's all the neural cortex
so then that's kind of undeniable fact
uh if but then there's other parts of
our brain are important too
right our emotional states uh our body
regulating our body
um so the way i like to look at it is
you know could you can you understand
the neocortex about the rest of the
brain
and some people say you can't and i
think absolutely you can
it's not that they're not interacting
but you can understand them can you
understand the neocortex without
understanding the emotions of fear yes
you can you can understand how the
system works it's just a modeling system
i make the analogy in the book that it's
it's like a map of the world
and how that map is used depends on
who's using it
so how our map of our world in our
neocortex
how we how we manifest as a human
depends on the rest of our brain what
are our motivations you know what are my
desires am i a nice guy or not a nice
guy
am i a cheater or a you know or not a
cheater um
uh you know how important different
things are in my life
so um so but the
new projects can be understood on its
own um and and i say that
as a neuroscientist i know there's all
these interactions and i want to
say i don't know them and we don't think
about them but from a layperson's point
of view you can say
it's a modeling system i don't tend to
think too much about the communal aspect
of intelligence which you brought a
number of times already
um so that's not really been my concern
i just wonder if there's a continuum
from the origin of the universe like
this com pockets of complexities that
form
yeah living organisms i wonder if if
we're just
if you look at humans we feel like we're
at the top
but i wonder if there's like just where
everybody probably every living type
pocket of complexity
is probably thinks they're the uh pardon
the french
they're the yeah they're they're
they're at the top of the parent well
if they're thinking um well then then
what is thinking what the all right
in this sense the whole point is in
their
sense of the world they their sense
is that they're at the top of it i think
what is it turtle
but you're you're you're bringing up you
know the the problems of complexity and
complexity theory
are you know it's a huge interesting
problem in science
um and you know i think we've made
surprisingly little progress in
understanding complex systems
right in general um and so you know the
santa fe institute was founded to to
study this and and even the scientists
there will say it's really hard we
haven't really been able to figure out
exactly you know that science isn't
really congealed yet we're still trying
to figure out the basic elements of that
science
uh what you know where does complexity
come from and what is it and how you
define it whether it's
dna creating bodies or phenotypes or if
it's
individuals creating societies or ants
and you know
markets and so on it's it's a very
complex thing i'm not a complexity
theorist
person right um and i i think
they ask well the brain itself is a
complex system so
can we understand that um i think we've
made a lot of progress understanding how
the brain works
so but i haven't brought it out to like
oh well where are we on the complexity
spectrum
you know it's like um that's a great
question
i'd prefer for that answer to be we're
not special
it seems like if we're honest most
likely we're not special so if there is
a spectrum
we're probably not in some kind of
significant place there's one thing we
could say that we are special
and and again only here on earth i'm not
saying i'm bad
is that if we think about knowledge
what we know um we clearly
human brains have um the only brains
that have a certain types of knowledge
we're the only brains on
on this earth to understand uh what the
earth is how old it is
that the universe is a picture as a
whole the only organisms understand dna
and the origins of you know of species
uh no other species on on this planet
has that
knowledge so if we think about i like to
think about
you know one of the endeavors of
humanity is to
understand the universe as much as we
can um
i think our species is further along in
that undeniably
um whether our theories are right or
wrong we can debate but at least we have
theories you know we
we know that what the sun is and how
it's fusion is and how
what black holes are and you know we
know
general theory relativity and no other
animal has any of this knowledge
so in that sense that we're special uh
are we special in terms of
the the hierarchy of complexity in in
the universe probably not
can we look at a neuron yeah you say
that prediction
happens in the neuron what does that
mean so neuron traditionally seen as the
basic element
of the the brain so we i mentioned this
earlier
that prediction was our research agenda
yeah we said okay
um how does the brain make a prediction
like i i'm about to grab this water
bottle
and my brain is predicting what i'm
going to feel um on all my parts of my
fingers if i felt something really odd
on any part here i notice it
so my brain is predicting what it's
going to feel as i grab this thing
so what is that how does that manifest
itself in neural tissue right we got
brains made of neurons and there's
chemicals and there's neurons and
there's spikes and the connect you know
where where is the prediction going on
and one argument could be that well when
i'm predicting something
um a neuron must be firing in advance
it's like okay this neuron represents
what you're going to feel and it's
firing it's sending a spike
and certainly that happens to some
extent but our predictions are so
ubiquitous
that we're making so many of them which
we're totally unaware of just the vast
majority we have no idea that you're
doing this
um that it wasn't really
we were trying to figure how could this
be where where is these where are these
happening
right and i won't walk you through the
whole story unless you
insist upon it but we came to the
realization
that most of your predictions are
occurring
inside individual neurons especially
these the most common are in the
pyramidal cells
and there are there's a property of
neurons
we everyone knows or most people know
that a neuron is a cell and it has this
spike
called an action potential and it sends
information
but we now know that there's these
spikes internal to the neuron
they're called dendritic spikes they
travel along the branches of the neuron
and they don't leave the neuron they're
just internal only
there's far more dendritic spikes than
there are action potentials
far more they're happening all the time
and
what we came to understand that those
dendritic spikes the ones that are
occurring are actually a form of
prediction
they're telling the neuron the neuron is
saying i expect
that i might become active shortly and
that internal
so the internal spike is a way of saying
you're going to you might be generating
external spikes soon
i predicted you're going to become
active and and we we've we've
we wrote a paper in 2016 which explained
and how this
manifests itself in neural tissue and
how it is that this all works together
but the vast ma we think it's there's a
lot of evidence supporting it
um so we that's where we think that most
of these predictions are internal that's
why you can't
be per their internal neuron you can't
perceive them
from understanding the the prediction
mechanism of a single neuron
do you think there's deep insights to be
gained about the prediction
capabilities of the mini brains within
the bigger brain and the brain oh yeah
yeah yeah
so having a prediction side of the
individual neuron is not that useful
you know what so what um the way it
manifests itself
in neural tissue is that
when a neuron a neuron emits these
spikes or a very singular type event
if a neuron is predicting that it's
going to be active it makes it spike
very a little bit sooner just a few
milliseconds sooner than it would have
otherwise it's like
i give the analogy in the book there's
like a sprinter on a on a starting
blocks in
a race and if someone says get ready set
you get up and you're ready to go
and then when your race starts you get a
little bit earlier start so that it's
that
that ready set is like the prediction
and the neuron's like ready to go
quicker
and what happens is when you have a
whole bunch of neurons together
and they're all getting these inputs the
ones that are in the predictive state
the ones that are
anticipating to become active if they do
become active they they happen sooner
they disable everything else and it
leads to different representations in
the brain so
you have to it's not isolated just to
the neuron the prediction occurs within
the neuron
but the network behavior changes so what
happens under different predictions
different inputs have different
representations so how i
what i predict um it's going to be
different under different contexts
you know what my input will be is
different under different context so
this is this is a
key level theory how this works so the
theory of the thousand brains
if you were to count the number of
brains how would you do it
the thousand main theory says that
basically every cortical column
in the in your neurocortex is a complete
modeling system
and that when i ask where do i have a
model of something like a coffee cup
it's not in one of those models it's in
thousands of those models there's
thousands of models of coffee cups
that's what the thousand brains there's
a voting mechanism then there's a voting
mechanism which leads which
is the thing you're which you're
conscious of which leads to your
singular perception
um that's why you perceive something so
that's the thousand brains theory
the details how we got to that theory
um are complicated it wasn't you just
thought of it one day
and one of those details is we had to
ask how does a a model make predictions
and we've talked about just these
predictive neurons
that's part of this theory it's like
saying oh it's a detail but
it was like a crack in the doors like
how are we going to figure out how these
neurons build do this
you know what is going on here so we
just looked at prediction as like
well we know that's ubiquitous we know
that every part of the cortex is making
predictions
therefore whatever the predictive system
is it's going to be everywhere
we know there's a gazillion predictions
happening at once so let's see if we can
start teasing apart
you know ask questions about you know
how could neurons be making these
predictions and that
sort of built up to now what we have the
thousand brains theory
which is complex you know it's just some
i can state it simply but we just didn't
think of it
we had to get there step by step very it
took years
uh to get there and where does uh
reference frames fit in so yeah
okay so again a reference frame i
mentioned
um earlier about the you know a model of
a house and i said if you're going to
build a model of a house
in a computer they have a reference
frame and you can then reference them
like
cartesian coordinates like x y and z
axes
so i can say oh i'm going to design a
house i can say well the
the front door is at this location xyz
and the roof is at this location xyz and
so on
that's a type of reference frame so it
turns out
for you to make a prediction and then i
walk you through the thought experiment
in the book where i was
predicting what my finger was going to
feel when i touched the coffee cup
it was a ceramic coffee cup but this one
will do um
and what i realized is that to make a
prediction with my finger's going to
feel like it's just going to feel
different than this which would feel
different if i touch the hole or the
thing on the bottom
make that prediction the cortex needs to
know where the finger is the tip of the
finger
relative to the coffee cup and exactly
relative to the coffee cup
and to do that i have to have a
reference frame for the coffee up it has
to have a way of representing the
location of my finger
to the coffin up and then we realize of
course every part of your skin has to
have a reference frame relative things
to touch and then we
did the same thing with vision but so
the idea that a reference frame
is necessary to make a prediction when
you're touching something or when you're
seeing something
and you're moving your eyes you're
moving your fingers it's just a
requirement
to know what to predict if i have a if i
have a structure i'm going to make a
prediction i have to
i have to know where it is i'm looking
or touching it
so then we say well how do neurons make
reference frames it's not obvious
you know xyz coordinates don't exist in
the brain it's just not the way it works
so that's when we looked at the older
part of the brain the hippocampus and
the antorano cortex
where we knew that in that part of the
brain
there's a reference frame for a room or
reference name for environment remember
i talked earlier about how you could
know make a map of this room
so we said oh um that they are
implementing reference frames there so
we knew that reference frames needed to
exist in every cortical column
and so that was a deductive thing we
just deduced it
has to go so you take the old
mammalian ability to know where you are
in a particular space
and you start applying that to higher
and higher levels yeah you first you
apply it to physical like where your
finger is
so here's what i think about it the old
part of the brain says where's my body
in this room
yeah the new part of the brain says
where's my finger
relative to this this object yeah where
is
the a section of my retina relative to
this object like where where is
i'm looking at one little corner where
is that relative to this patch of my
retina yeah
um and then we take the same thing and
apply it to concepts
mathematics physics you know humanity
whatever you want to think eventually
you're pondering your own mortality
well whatever but the point is when we
think about the world when we have
knowledge about the world how is that
knowledge organized
lex where do you where is it in your
head the answer is it's in reference
frames
so the way i learn the structure of this
water bottle
where the features are relative to each
other when i think about history or
democracy or mathematics
the same basic underlying structures
happening there's reference frames for
where the knowledge
that you're assigning things to so in
the book i go through examples like
mathematics
and language and politics but
the evidence is very clear in the
neuroscience the same mechanism that we
use to model this coffee cup we're going
to use to model
high level thoughts your your your
demise of the humanity whatever you want
to think about
it's interesting to think about how
different are the representations of
those
higher dimensional concepts
higher level concepts how different the
representation there is in terms of
reference frames
versus spatial but interesting thing
it's it's
it's a different application but it's
the exact same mechanism
but isn't there some aspect to uh
higher level concepts that they seem to
be hierarchical
like they just seem to integrate a lot
of information into so
is our physical objects so take this
water bottle
uh i'm not particular to this brand but
this is a fiji water bottle
and it has um a logo and i use this
example in my book
our company's coffee cup has a logo on
it but
this object is hierarchical it is
it's got like a cylinder and a cap but
then has this logo on it and the logo
has a word the word has letters the
letters of different features
and so i don't have to remember i don't
think about this so i said oh there's a
fiji logo on this water bottle i don't
have to go through and say
oh what is the fiji logo it's the f and
i and the j and i and there's a hibiscus
flower and
and uh oh it has the pest you know the
stamen on it i don't have to do that i
just incorporate all of that
in some sort of hierarchical
representation i say um
you know put this logo on this water
bottle yeah and
and and then the logo has a word and the
word has letters
all hierarchical just all that stuff is
big it's amazing that the brain
instantly just does all that yeah the
idea that there's there's water it's
liquid and the idea that you can
uh drink it when you're thirst