Kind: captions
Language: en
the following is the conversation with
yoshi bach his second time on the
podcast yoshi is one of the most
fascinating minds in the world exploring
the nature of intelligence cognition
computation and consciousness
to support this podcast please check out
our sponsors coinbase codecademy linode
netsuite and expressvpn their links are
in the description
this is the lex friedman podcast and
here is my conversation with yosha bach
thank you for once again coming on to
this particular russian program and
sticking to the theme of a russian
program let's start with the darkest of
topics
so this is inspired by one of your
tweets
you wrote that
quote when life feels unbearable
i remind myself that i'm not a person
i am a piece of software running on the
brain of a random ape for a few decades
it's not the worst brain to run on
have you
experienced low points in your life have
you experienced depression
of course we all experience low points
in our life and
we get appalled by the things by the
ugliness of stuff around us we might get
desperate about our lack of
self-regulation and
sometimes
life is hard and i suspect you don't get
to your life nobody does to get through
their life without low points and
without moments where they're despairing
and i thought that
let's capture this state and
how to deal with that state and i found
that very often you realize that when
you stop taking things personally when
you realize that this notion of a person
is a fiction
similar as it is in westworld where the
robots realize that their memories and
desires are just stuff that keeps them
in the loop and they don't have to act
on those memories and desires that our
memories and expectations is what make
us unhappy and the present rarely does
the day in which we are for the most
part it's okay right when we are right
sitting here right here right now we can
choose how we feel
and the thing that affects us is the
expectation that something is going to
be different from what we wanted to be
or the memory that something was
different from what you wanted it to be
and
once we basically zoom out from all this
what's left is not a person what's left
is
this state of being conscious which is a
software state and software doesn't have
an identity it's a physical law
and it's a law that acts in all of us
and it's embedded in a suitable
substrate and we didn't pick that
substrate right we are mostly randomly
instantiated on it and there all these
individuals and
everybody has to be one of them
and uh eventually you're stuck on one of
them and um have to deal with that so
you're like a leaf floating down the
river
you just have to accept that there's a
river and you just
that you are an agent is a construct
right what part of that is actually
under your control
and i think that our consciousness is
largely a control model for our own
attention so we
notice where we are looking and we can
influence what we are looking how we are
disambiguating things how we put things
together in our mind
and the whole system that runs us is
this big cybernetic motivational system
so we're basically like a little monkey
sitting on top of an elephant and we can
put this elephant here and there to go
this way or that way and we might have
the illusion that we are the elephant or
that we are telling it what to do and
sometimes we notice that it walks into a
completely different direction and we
didn't set this thing up it just is the
situation that we find ourselves in
how much prodding can we actually do of
the elephant
a lot but
i think that our uh consciousness cannot
create the motive force
is the elephant consciousness in this
metaphor no the monkey
is the consciousness the monkey is the
attentional system that is observing
things there is a large perceptual
system combined with the motivational
system that is actually providing the
interface to everything and our own
consciousness i think is a tool that
directs the attention of that system
which means it singles out features
and performs conditional operations for
which it needs an index memory but this
index memory is what we perceive as our
stream of consciousness but the
consciousness is not in charge that's an
illusion so everything
outside of that consciousness
is the elephant so it's the physics of
the universe but it's also society
that's outside of europe i would say the
elephant is the agent so there is an
environment which the agent is stomping
and uh you are influencing a little part
of that agent
so uh can you is the agent a single
human being
what's what which object has agency
that's an interesting question i think a
way to think about an agent is that it's
a controller with a set point generator
the notion of a controller comes from
cybernetics and control theory control
system consists out of a system that is
regulating some value and the deviation
of that value from a set point and it
has a sensor that measures the system's
deviation from that set point
and an effector that can be parametrized
by the controller so the controller
tells the effector to do a certain thing
and the goal is to reduce the distance
between the set point and the current
value of the system and there's
environment which disturbs the regulated
system which brings it away from that
set point so simplest case is the
thermostat the thermostat is really
simple because it doesn't have a model
the thermostat is only trying to
minimize the set point deviation in the
next moment
and if you want to minimize the set
point deviation over a longer time span
you need to integrate it you need to
model what is going to happen so for
instance when you think about that your
set point is to be comfortable in life
maybe you need to make yourself
uncomfortable first
right so you need to make a model of
what's going to happen when and this is
task of the controller is to use its
sensors to measure the state of the
environment
and the system that is being regulated
and figure out what to do
and if the task is complex enough the
set points are complicated enough and if
the controller has enough capacity and
enough
sensor feedback then the task of the
controller is to make a model of the
entire universe that it's in the
conditions under which it exists and of
itself
and this is a very complex agent and we
are in that category
and
an agent is not necessarily a thing in
the universe it's a class of models that
we use to interpret aspects of the
universe
and
be when we notice the
around us a lot of things only make
sense at the level that you are
entangled with them is we interpret them
as control systems that make models of
the world and try to minimize their own
set points so but the models
are the agents
the agent is a class of model
and we notice that we are an agent
ourself we are the agent that is using
our own control model to perform actions
we notice we
uh produce a change in the model and
things in the world change and this is
how we discover
the idea that we have a body that we are
situated environment and that we have a
first person perspective
still don't understand
what's the best way to think of which
object has agency with with respect to
human beings
is is it the body
is it the brain
is it the contents of the brain that has
agency like what's the actuators that
you're referring to
what is the controller and where does it
reside or is it these impossible things
like because i keep trying to ground it
to space-time
the three-dimensional space
and the one dimension of time
what's the agent in that for humans
there is not just one it depends on the
way in which you're looking at the thing
in which you're framing it imagine that
you are
say angela merkel and you are acting on
behalf of germany
then you could say that germany is the
agent and in the mind of angela merkel
she is germany to some extent because in
the way in which she acts the destiny of
germany changes
there are things that she can change
that basically
affect the behavior of that nation state
okay so it's hierarchies of to go to
another one of your tweets
with uh i think your uh playfully
mocking jeff hawkins
with saying his brains all the way down
so it's like it's agents all the way
down
it's agents made up of agents made up of
agents like if phanja marco's germany
and germany's made up a bunch of people
and the people are themselves agents
in in some kind of context
and then people are made up of cells
each individual
so is it agents all the way down
i suspect that has to be like this in
a world where things are self-organizing
most of the complexity that we are
looking at
everything in life is about
self-organization yeah so i think up
from the level of
life you have
agents
and
below life you rarely have agents
because
sometimes you have control systems that
emerge randomly in nature and try to
achieve a set point but
they're not that interesting agents that
make models and because to make an
interesting model of the world you
typically need a system that is true and
complete can i ask you a personal
question
uh
what's the line between life and
non-life it's personal because you're
a life form
so what do you think in this
emerging complexity at which point does
the thing start being living and have
agency
personally i think that the simplest
answer is that life is sales because
life is what cells cells biological
cells so it's a particular kind of
principle that we have discovered to
exist in nature it's modular stuff that
consists out of
basically this dna tape is a read write
head on top of it
that is able to perform arbitrary
computations and state transitions
within the cell and it's combined with a
membrane that insulates the cell from
its environment
and there are
chemical reactions inside of the cell
that are in this equilibrium and the
cell is running in such a way that this
this equilibrium doesn't disappear and
the cell
goes if the cell goes into an
equilibrium state it dies
and it requires something like an neck
entropy extractor to maintain this this
equilibrium so it's able to harvest like
entropy from its environment and keep
itself running
yeah so there's information and there's
a wall to protect to to to maintain this
disequilibrium but isn't this very
earth-centric
like what you're referring to as
i'm not making a normative notion uh you
could say that there are probably other
things in the universe that are
cell-like and life-like and you could
also call them life but eventually it's
just a
willingness of to find an agreement of
how to use the terms
i like cells because it's completely
co-extensional with the way that we used
the word even before we knew about cells
so people were pointing at some stuff
and saying this is somehow animate and
this is very different from the
non-animated stuff and what's the
difference between the living and the
dead stuff and it's mostly whether the
cells are working or not
and uh also this boundary of life where
we say that for instance a virus is
basically an information packet that is
subverting the cell and not life by
itself
that makes sense to me and
it's somewhat arbitrary you could of
course say that systems that permanently
maintain a disequilibrium and can
self-replicate are always life
and
maybe that's a useful definition too but
this is eventually just how you want to
use the word is it uh so useful for
conversation but
is it uh
somehow fundamental to the universe do
you think there's a actual line to
eventually be drawn between life and
non-life or is it all a kind of
continuum i don't think it's a continuum
but there's nothing magical that is
happening um living systems are a
certain type of machine
what about non-living systems is it also
a machine there are non-living machines
but the question is at which point is
the system able to un uh perform
arbitrary state transitions in uh to
make representations
and living things can do this and of
course we can also build non-living
things that can do this but we don't
know anything in nature that is not a
cell
and is not created by the lola life that
is able to do that
not
not only do we not know
i don't think we have the tools to see
otherwise
i always worry that we we
look at the world too narrowly
like we have there could be life
of a very different kind right under our
noses
that we're just not
seeing because we're not
either limitations of our cognitive
capacity or
we're just not open-minded enough
either with the tools of science or just
the tools of our own mind
yeah that's possible i find the thought
very fascinating and i suspect that many
of us ask ourselves since childhood what
are the things that we are missing what
kind of systems and interconnections
exist
that are outside of our
gaze but the um
we are looking for it and physics
doesn't have much room at the moment
for uh
opening up something that would not
violate the conservation of information
as we know it
yeah but i i wonder about time time
scale and scale spatial scale whether we
just need to um
open up our idea of what
like how life presents itself it could
be operating in a much slower time scale
yeah a much faster time scale
and
it's almost sad to think that there's
all this life around us that we're not
seeing because we're just not
like
thinking in terms of the right of the
right scale
both time and space
what is your definition of life what do
you understand this life
[Music]
entities of sufficiently high complexity
that are full of surprises
i don't know i don't have a free will so
that just came out of my mouth i'm not
sure that even makes sense there are
certain characteristics so complexity
seems to be an unnecessary property of
life
and
i almost
want to say it has
ability to do something unexpected
it seems to me that life is the main
source of complexity on earth
yes and complexity is basically a
bridgehead that order builds into chaos
by modeling
by processing information in such a way
that you can perform reactions that
would not be possible for dump systems
and this means that you can harvest neck
entropy that dump systems cannot harvest
and this is what complexity is mostly
about
yeah in some sense the purpose of life
is to create complexity
yeah
increasing i mean there there's um
there seems to be some kind of universal
drive towards increasing pockets of
complexity
i don't know what that is that seems to
be like a fundamental
i don't know if it's a property of the
universe or it's just the consequence of
the way the universe works but there
seems to be this small pockets of
emerging complexity that builds on top
of each other and starts
having like greater and greater
complexity by having like a hierarchy of
complexity little organisms building up
a little society that then operates
almost as an individual organism itself
and all of a sudden you have uh germany
and merkel but that's not obvious to me
everything that goes up has to come down
at some point
right so every if you see this
big exponential curve somewhere it's
usually the beginning of an s-curve
where something eventually reaches
saturation and the s-curve is the
beginning of some kind of bump that goes
down again
and
there is
just the thing that when you are in
sight of an evolution of life you are on
top of a puddle of negentropy that is
being sucked dry
by life and during uh that happening you
see an increase in complexity
because life forms are competing with
each other to get more and more and a
finer and finer corner of that like
entropy extraction but that i feel like
that's a gradual beautiful process like
that's almost
you know follows a process akin to
evolution and the way it comes down
is not the same way it came up
the way it comes down is usually harshly
and quickly
so usually there's some kind of
catastrophic event well the roman empire
took a long time
uh
but that's would that be would you
classify this as a decrease in
complexity though yes i think that this
uh size of the cities that could be fed
has decreased dramatically and you could
see that the quality of the art
decreased and it did so gradually
and
maybe
future generations when they look at the
history of the united states in the 21st
century will also talk about the gradual
decline not something that suddenly
happens
do you have a sense of where we are
are we on the exponential rise are we at
the peak
or are we the downslope
of the the united states empire it's
very hard to say from a single human
perspective but
i it seems to me that we are probably
at the peak
i think that's probably the definition
of like optimism and cynicism
so my nature of optimism is i think
we're on the rise
but
uh i think it's just all a matter of
perspective nobody knows but i do think
that erroring on the side of optimism
like you need a sufficient number
you need a minimum number of optimists
in order to make that up thing actually
work
and so i tend to be on the side of the
optimists i think that we are basically
a species of grasshoppers that have
turned into locusts
and when you are in that locust mode you
see an amazing rise of population
numbers and of the
complexity of the interactions between
the individuals
but
it's ultimately the question is is it
sustainable see i think we're a bunch of
lions and tigers that have become
domesticated cats
to use a different metaphor as i'm not
exactly sure
we're so destructive or just softer
and nicer and lazier
but i think we have monkeys and not the
cats and if you look at the monkeys they
are very busy
are the ones that have a lot of sex
those monkeys not just the bonobos i
think that all the monkeys are basically
a discontent species that always needs
to meddle
well the gorillas seem to have a little
bit more of a structure but it's a
different different part of the tree
[Laughter]
okay uh you mentioned the elephant and
the the monkey riding the elephant
and uh consciousness is the monkey
and there's some prodding that the
monkey gets to do and sometimes the
elephant listens
i heard you got
into some content maybe you can correct
me but i heard you got into some
contentious free will discussions
uh is this with sam harris or something
like that not that i know of
some people on clubhouse told me you
made a a bunch of uh um
big debate points about free will well
let me just then ask you where where
in terms of the monkey and the elephant
uh do you think we land in terms of the
illusion of free will how much control
does the monkey have
we have to think about what the
free will is in the first place
we are not the machine we are not the
thing that is making the decisions we
are a model of that decision making
process yeah and there is a difference
between
making your own decisions and predicting
your own decisions yes and that
difference is the first person
perspective
and
what
basically makes decision-making um and
the conditions of free will distinct
from just automatically doing the best
thing is
that uh we often don't know what the
best thing is we make decisions under
uncertainty we make informed bets using
a betting algorithm that we don't yet
understand because we haven't reverse
engineered our own mind sufficiently we
don't know the expected rewards we don't
know the mechanism by which we estimate
the rewards and so on but there is
we observe ourselves performing where we
see that uh we evade facts and factors
and the future and then
some kind of possibility some motive
gets raised to an intention
and that's informed bad that the system
is making
and that making of the open bet the
representation of that is what we call
free will
and it seems to be paradoxical because
we think that's the crucial thing is
about it that it's somehow
indeterministic
and yet if it wasn't deterministic it
would be random
and of course it cannot be random
because it was if it was random if just
dice were being thrown in the universe
randomly forces you to do things it
would be meaningless so the important
part of the decisions is always the
deterministic stuff
but it appears to be indeterministic to
you because it's unpredictable because
if it was predictable you wouldn't
experience it as a free will decision
you would experience it as just doing
the necessary right thing
and you see this continuum between the
free will and the execution of automatic
behavior
when you're observing other people so
for instance when you are observing your
own children
if you don't understand them you will
use this agent model where you have a
agent with a set point generator and uh
the agent is doing the best it can to
minimize the difference to the set point
and it might be confused and uh
sometimes impulsive or whatever but it's
acting on its own free will
and when you understand what happens in
the mind of the child you see that is
automatic and you can outmodel the child
you can build things around the child
that will lead the child to making
exactly the decision that you are
predicting
and in under these circumstances like
when you were a stage magician or
somebody who is dealing uh with people
that this you sell a car to and you
completely understand the psychology and
the impulses and the space of thoughts
that this individual can have at that
moment under these circumstances it
makes no sense to attribute free will
because it's no longer decision making
under uncertainty you are already
certain for them there is uncertainty
but you already know what they are doing
but what about for you so
is this akin to like
systems like cellular automata
where it's
deterministic
but when you
squint your eyes a little bit
it starts look like there's agents
making decisions at the higher so
when you zoom out and look at the
entities that
are composed by the individual cells
even though the there's
underlying simple rules that
make the system evolve in deterministic
ways it looks like there's organisms
making decisions is that
where the illusion of free will emerges
that jump and scale
it's a particular type of model but this
jump in scale is crucial the jump in
scale happens whenever you have too many
parts to count and you cannot make a
model at that level and you try to find
some higher level regularity
and the higher level regularity is a
pattern that you project into the world
to make sense of it and agency is one of
these patterns right you have all these
cells that interact with each other and
the cells in our body are set up in such
a way that they benefit if their
behavior is coherent which means that
they act as if they were serving a
common goal
and which that means that they will
evolve regulation mechanisms that
act as if they were serving a common
goal and now you can make sense of these
all these cells by projecting the common
goal into them
right so for you then free will is an
illusion
no it's a model and it's a construct
it's basically a model that the system
is making of its own behavior and it's
the best model that it can come up with
under the circumstances and it can get
replaced by a different model which is
automatic behavior when you fully
understand the mechanism under which you
are acting yeah but the another word for
model is what story
so it's the story you're telling i mean
you actually have control is there such
a thing as a you
and is there such a thing as you having
control
it's like are you
manifesting
your
evolution as an entity in some sense the
u is the model of the system that is in
control it's a story that the system
tells itself about somebody who is in
control
yeah and the contents of that model are
being used to inform the behavior of the
system
okay so the system is completely
mechanical and the system creates that
story like a loom
and then it uses the contents of that
story to inform its actions and writes
the results of that actions into the
story so how's that not an illusion
the story is written then
or
or rather we're not the writers of the
story
yes but we always knew
that no we we don't know that when did
we know that i think that's mostly a
confusion about concepts the
conceptual illusion in our culture comes
from the idea that we live in physical
reality
and that we experience physical reality
and that you have ideas about it
and then you have this dual list
interpretation where you have
two substances res extensor the world
that you can touch and that is made of
extended things and res cognitions which
is the world of ideas
and in fact both of them are mental
representations
one is the representations of the world
as a game engine that your mind
generates to make sense of the
perceptual data and the other one
yes that's what we perceive as the
physical world but we already know that
the physical world is nothing like that
right quantum mechanics is very
different from what you and me perceive
as the world the world that you and me
perceive is a game engine yeah and there
are no colors and sounds in the physical
world they only exist in the game engine
generated by your brain and then you
have ideas that are not cannot be mapped
onto extended regions right so the
objects that have a spatial extension in
the game engine are res extensor and the
objects that don't have a physical
extension in the game
engine our ideas
and they both interact in our mind to
produce models of the world yep but
you know when you play video games
i understand that what's actually
happening is zeros and ones inside of uh
inside of a computer instead of a cpu
and a gpu
but you're still seeing
like uh the rendering of that
and you're still making decisions
whether to shoot to turn left or to turn
right if you're playing a shooter or
every time you start thinking about
skyrim and elder scrolls and walking
around in beautiful nature and swinging
a sword but
it feels like you're making decisions
inside that video game
so even though you don't have direct
access
uh in terms of perception to the
bits to the zeros and ones it still
feels like you're making decisions and
your decisions are actually feels like
they're being applied
all the way down to the zeros and ones
yes it feels like you have control even
though you don't direct
access to reality so there is basically
a special character in the video game
that is being created by the video game
engine yeah and this character is
serving the aesthetics of the video game
and that is you
yes but i feel like you have control
inside the video game
like the all those like 12 year olds
that kick my ass on the internet
so uh for when you play the video game
it doesn't really matter that they're
zeros and once right you don't care
about the vids of the bus you don't care
about the nature of the cpu that it runs
on what you care about are the
properties of the game that you're
playing and you hope that the cpu is
good enough
yes and a similar thing happens when we
interact with physics the world that you
and me are in is not the physical world
the world that you and me are in is a
dream world
how close is it to the real world though
we know that it's not very close but we
know that the dynamics of the dream
world match the dynamics of the physical
world to a certain degree of resolution
right the causal structure of the
dreamworld is different
so you see waves crashing on your feet
right but there are no waves in the
ocean there's only water molecules that
have tangents uh between the molecules
that are uh
ex
the result of electrons in the molecules
interacting with each other aren't they
like very consistent we're just seeing a
very uh crude approximation isn't
our dream world very consistent like to
the point of being mapped directly
one-to-one to the actual physical world
as opposed to
us being completely tricked is this is
like where you have like that it's not a
trick that's that's my point it's not an
illusion it's a form of data compression
yeah it's an attempt to deal with the
dynamics of too many parts to count at
the level at which we're entangled with
the best model that you can find yeah so
we can act in that dream world and our
actions have impact in the in the real
world in the physical world yes to which
we don't have access yes but it's
basically like
accepting the fact that the software
that we live in the dream that we live
in is generated by something outside of
this world that you and me are in so is
the software deterministic and do we not
have any control
do we have
so free will
is uh
having a conscious being
the free will is the monkey being able
to steer the elephant
no
it's slightly different
basically in the same way as you are
modeling the water molecules in the
ocean that engulf your feet when you are
walking on the beach as waves and there
are no waves uh but only the atoms on
more complicated stuff underneath the
atoms and so on and you know that right
you would accept yes there is a certain
abstraction that happens here it's a
simplification of what happens in
simplification that is designed in such
a way that your brain can deal with it
temporarily and spatially in terms of
resources and tuned for the predictive
value so you can predict with some
accuracy whether your feet are going to
get wet or not but it's a really good
approach it's a really good interface
and approximation yes it's like equals
mg squared is a good
equations are good approximations for
what they're much better approximation
so
to me waves is a really nice
approximation what's all the complexity
that's happening underneath basically
it's a machine learning model that is
constantly tuned to minimize surprises
so it basically tries to predict as well
as it can what you're going to perceive
next are we talking about
which is the machine learning our
perception system or the dream world
the machine world is a dream world is
the result of the machine learning
process of the perception system that's
doing the compression yes
and uh the model of you as an agent is
not a different type of model or it's a
different type but not uh not different
as in its model like nature from the
model of the ocean right some things are
oceans some things are agents and one of
these agents is using your own control
model the output of your model the
things that you perceive yourself as
doing
and that is you
what about
the fact that like when you're standing
um and with the water on your feet and
you're looking out
into the vest
like open water of the ocean and then
there's a beautiful sunset
and it well the fact that it's beautiful
and then maybe you have like friends or
a loved one with you and like you feel
love what is that as the dream world
what is that yes it's all uh happening
inside of the dream okay
but see the word dream makes it seem
like it's not real
yeah of course it's not real
the physical universe is real but the
physical universe is incomprehensible
and it doesn't have any feeling of
realness the feeling of realness that
you experience gets attached to certain
representations where your brain
assesses this is the best model of
reality that i have so the only thing
that's real to you is the thing that's
happening at the very
base of reality like
for something to be real it needs to be
implemented
so uh the model that you have of reality
is a real in as far as it is a model
right it's an appropriate description of
the world to say that there are models
that are being experienced
but
the world that you experience is not
necessarily implemented
there is a difference between a reality
a simulation and a simulacrum
the
reality that we are talking about is
something that fully emerges over a
causally closed lowest layer the idea of
physicalism is that we are in that layer
that basically our world emerges over
that every alternative to physicalism is
a simulation theory which basically says
that we are in some kind of simulation
universe and the real world needs to be
an apparent universe of that where the
actual causal structure is right
and when you look at the ocean and your
own mind you are looking at a simulation
that explains what you're going to see
next
and we are living in a simulation yes
but the simulation generated by our own
brains
yeah and this simulation is different
from the physical reality because the
causal structure that is being produced
what you are seeing is different from
the causal structure of physics but
consistent
hopefully if not then you are going to
end up in some kind of institution where
people will take care of you because
your behavior will be inconsistent right
your uh behavior needs to work in such a
way that it's interacting with an
accurately predictive model of reality
and if your brain is unable to make your
model of reality predictive
um you will need help so what uh what do
you think about donald hoffman's
argument
that it doesn't have to be consistent
the dream world
to the the what he calls like the
interface
uh to the actual physical reality where
there could be evolution i think he
makes an evolutionary argument
which is like it could be an
evolutionary advantage to have the dream
world drift away from physical reality i
think that only works if you have tenure
as long as you are still interacting
with the ground tools your model needs
to be somewhat predictive
well
in some sense humans have achieved a
kind of tenure in the animal kingdom
at some point we became too big to fail
so we became postmodernist
it all makes sense the version of
reality that we like
oh man
okay
yeah but
basically you can do magic you can
change your assessment of reality but
eventually uh reality is going to come
bite you in the s if it's not predictive
do you have a sense
of what is that base layer physical
reality
you have
like uh so you have these attempts at
the theories of everything
the very very small of like strength
theory
or what um stephen wolfram talks about
with a hyper grass these are these tiny
tiny tiny tiny objects
and then there is more like
quantum mechanics
that's talking about objects that are
much larger but still very very very
tiny do you have a sense of where the
tiniest thing is that is like
at the lowest level the turtle at the
very bottom do you have a sense
i don't think that you can talk about
where it is because space is emergent
over the activity of these things so
space
uh the coordinates only exist in
relation to the
things other things and so you could in
some sense abstract it into locations
that can hold information and
trajectories that the information can
take between the different locations and
this is how we construct our notion of
space
yeah and uh physicists uh usually have a
notion of space that is continuous
and this is a point where i
tend to agree with people like stephen
warfram who are very skeptical of the
geometric notions
i think that geometry is the dynamics of
too many parts to count and
when there are no infinities if there
were two infinities you would be running
into contradictions which is in some
sense what uh google and turing
discovered
in response to hilbert's call
so there are no infinities there are no
infinities fake there is unboundedness
but if you have a language that talks
about infinity at some point the
language is going to contradict itself
which means it's no longer valid
in order to deal with infinities and
mathematics you have to postulate the
existence in uh initially you cannot
construct the infinities and that's an
issue right you cannot build up an
infinity from zero but in practice you
never do this right when you perform
calculations you only look at the
dynamics of too many parts to count
and
usually these
numbers are not that large they're not
googles or something the big
the infinities that we are dealing with
in our universe are mathematically
speaking
relatively small integers
and um still
what we're looking at is dynamics where
um
a trillion things behave similar to 100
trillion things or
something that is very very large
because they're converging and these
convergent dynamics these operators this
is what we
deal with when we are doing the geometry
right geometry is stuff where we can
pretend that it's continuous because uh
as if we subdivide the space
sufficiently
fine grained
these things approach a certain dynamic
and this approached dynamic that is what
we mean by it but i don't think that
infinity would work so to speak that you
would know the last digit of pi and that
you have a physical process that rests
on knowing the last digit of pi
yeah that that could be just a peculiar
quark of human cognition that we like
discrete discrete makes sense to us
infinity doesn't
so in terms of our intuitions no the
issue is that uh everything that we
think about uh needs to be expressed in
some kind of mental language not not
necessarily a natural language but some
kind of mathematical language that your
neurons can speak that refers to
something in the world and what we have
discovered is that
uh we cannot construct a notion of
infinity without running into
contradictions which means that such a
language is no longer valid
and i suspect this is what made
photographers so unhappy when somebody
came up with the notion of irrational
numbers before it was time right there's
this miss that he had this person killed
when he blapped out the secret that not
everything can be expressed as a ratio
between two numbers but there are there
are numbers between the ratios the world
was not ready for this and i think he
was right that has confused
mathematicians uh
very seriously because these numbers are
not values they're functions
right so you can calculate these
functions to a certain degree of
approximation but you cannot pretend
that pi has actually a value
pi is a function that would generally
approach this value to some degree
but nothing in the world rests on
knowing pie
uh how much does how important is this
distinction between discreet and
continuous uh
for you to get to the because there's a
i mean
in discussion of your favorite flavor of
the theory of everything there's a few
on the table
so there's string theory there's a
particular
there's a
loop quantum gravity which focus on one
particular unification
uh there's
there's just a bunch of favorite flavors
of different people trying to
uh
propose a theory of everything uh eric
weinstein
and a bunch of people throughout history
and then of course stephen wolfram who i
think is one of the only people doing a
discrete
no no there's a bunch of physicists who
do this right now and okay like um
topholy and tomasello and um
the
digital physics is something that is i
think growing in popularity
but uh
the
main reason why this is interesting is
because it uh it's important sometimes
to settle disagreements i don't think
that you need infinities as or at all
and you never needed them
you can always deal with very large
numbers and you can do it with limits
right you're fine with doing that you
don't need any kind of infinity you can
build your computer algebra systems just
as well without believing in infinity in
the first place you're okay with limits
yeah so basically a limit means that
something is behaving pretty much the
same
if you make the number larger
right because it's converging to a
certain value and at some point the
difference becomes measurable and you
can no longer measure it
and uh in this sense you have things
that uh
yeah if every ngon which is has enough
corners then it's going to behave like a
circle at some point right and it's only
going to be in some kind of esoteric
thing that cannot exist in the physical
universe that you would be talking about
this perfect circle and now it turns out
that it also wouldn't work in
mathematics because you cannot construct
mathematics that has infinite resolution
without running into contradictions
so that is itself not that important
because we never did that right it's
just a thing that some people thought we
could
and this leads to confusion so for
instance roger penrose uses this as an
argument to say
that there are certain things that
mathematicians can do
dealing with infinities
and by extension our mind can do
that computers cannot do yeah he he
talks about that there's the human mind
can do certain mathematical things
that the computer as defined by
the universal touring machine cannot yes
what so that it has to do with infinity
yes it's one of the things so he is
basically pointing at the fact that
there are things that are possible
in
the mathematical mind and in pure
mathematics that are not possible in uh
machines that can be constructed in the
physical universe
and because he's an honest guy he thinks
this means that uh present physics
cannot explain operations that happen in
our mind do you think he's right and uh
so let's let's leave his discussion of
consciousness aside for the moment do
you think he's right about just
what he's basically referring to as
intelligence
so
are is the human mind fundamentally more
capable as a thinking machine than a
universal touring machine no
but so he's suggesting that right
so our mind is actually less than a
turing machine there can be no touring
machine because it's defined as having
an infinite tape
and we always only have a finite tape
but you can perform finally many
operations yes
it can do the kind of computation the
yes the touring machine cannot and
that's because he thinks that our minds
can do operations that have infinite
resolution in some sense
and i don't think that's the case our
minds are just able to discover these
limit operators over too many parts to
count
what about his idea that consciousness
is
more uh more than a computation so it's
more than something that uh a touring
machine can can do
so again
saying that there's something special
about our mind they cannot be replicated
in the machine
the issue is that i don't even know how
to construct a language to
express this statement correctly
well
the the the basic statement is
there's a there's a human experience
that includes intelligence that includes
self-awareness that includes
the hard problem of consciousness
and the question is can that be fully
simulated
in the computer
in the mathematical model of the
computer as we understand it today
rajapanos says no
so the the
uh
universal turing machine cannot simulate
the universe
so the interesting question is uh and
you have to ask him this is why not what
is this specific thing that cannot be
modeled and
when i looked at his writings and i
haven't read all of it but when i read
for instance um the
section that he writes in the
introduction to and wrote to infinity
the thing that he specifically refers to
is
the way in which human minds deal with
infinities
and
that itself can i think easily be
deconstructed
a lot of uh people feel that our
experience cannot be explained in a
mechanical way
and therefore it needs to be different
and i concur our experience is not
mechanical our experience is simulated
it exists only in a simulation the only
assimilation can be conscious physical
systems cannot be conscious because
they're only mechanical cells cannot be
conscious neurons cannot be conscious
brains cannot be conscious people cannot
be conscious as far as you if you
understand them as physical systems
what can be conscious is
the story of a system in the world where
you write all these things into the
story
you have experiences for the same reason
that a character novel has experiences
because it's written into the story
and now the system is acting on that
story and it's not a story that is
written in a natural language it's
written in a perceptual language in this
multimedia language of the game engine
and in there
you write in what kind of experience you
have and what this means for the
behavior of the system for your behavior
tendencies for your focus for your
attention for your experience of valence
and so on and this is being used to
inform the behavior of the system in the
next step and then the
story updates with the reactions of the
system and the changes in the world and
so on and you live inside of that model
you don't live inside of the physical
reality
and
i mean just just to linger on it like
you see okay yeah it's in the perceptual
language the multimodal perceptual
language
that's the experience that's what
consciousness is within that
within that model within that story
but do you do you have agency
when you play a video game you can turn
left and you can turn right
in that story
so in that dream world how much control
do you is there such a thing as you in
that story
like
is it right to say the main character
you know everybody's npcs and then
there's the main character and you're
controlling the main character
or is that an illusion is there a main
character that you're controlling i'm
getting to the point of like
the free will point
imagine that you are building a robot
that plays soccer yeah and you've been
to mit computer science you basically
know how to do that
right and so uh you would say the robot
is an agent that solves the control
problem
how to get the ball into the goal and it
needs to perceive the world and the
world is disturbing him and trying to do
this right so he has to control many
variables to make that happen and to
project itself and the ball into the
future and understand its position on
the field relative to the ball and so on
in the uh position of its limbs or in in
the space around it and so on so it
needs to have an adequate model that
abstracting reality in a useful way
and
you could say that this
robot does have agency over what it's
doing in some sense
and the model is
going to be a control model and inside
of that control model you can
possibly get to a point where this thing
is sufficiently abstract to discover its
own agency our current robots don't do
that they don't have a unified model of
the universe but
there is not a reason why we shouldn't
be getting there at some point in the
not too distant future and once that
happens you will notice that the uh
robot tells a story about the robot
playing soccer
so the robot will experience itself
playing soccer in a simulation of the
world that it uses to
construct a model of the locations of it
lacks on and limbs in space on the field
with relationship to the ball and it's
not going to be at the level of the
molecules
it will be an abstraction that is
exactly at the level that is most
suitable for past planning of the
movements of the robot
right it's going to be a high level
abstraction but a very useful one that
is as predictive as we can make it and
in that side of that story there is a
model of the agency of that system so
this model can accurately
predict that the contents of the model
are going to be driving the behavior of
the robot in the immediate future but
there's the hard problem of
consciousness
which i would also
there's a subjective experience of free
will as well
that i'm not sure where the robot gets
that where that little leap is because
for me right now everything i imagine
with that robot as it gets more and more
and more sophisticated
the agency comes from the programmer of
the robot still
of what was programmed in
you could probably do an end-to-end
learning system you maybe need to give
it a few prayers so you nudge the
architecture in the right direction that
it converges more quickly but ultimately
uh discovering the suitable hyper
parameters of the architecture is also
only a search process right and as the
search process was evolution that has
informed our brain architecture so we
can converge in a single lifetime on
useful interaction with the world and
if we define hyper parameters broadly so
it's not just this the uh the parameters
that control this end-to-end learning
system but the entirety of the design of
the robot like the
there's
you have to remove the human completely
from the picture and then in order to
build the robot you have to
create an entire universe because you
have to go you can't just shortcut
evolution you have to go from the very
beginning
in order for it to have because i feel
like there's always a human
pulling the strings
um and that
makes it seem like the robot is cheating
it's getting a shortcut to consciousness
when you are looking at the current
boston dynamics robots it doesn't look
as if there is somebody pulling the
strings it doesn't look like cheating
anymore okay so let's go there because i
gotta talk to you about this so
obviously with the case of boston
dynamics
as you may or may not know
it's
always
either hard coded or remote controlled
there's no intelligence i don't know how
the current generation of boston
dynamics robots works but
what i've been told about the previous
ones was that it's basically all
cybernetic control
which means you still have uh feedback
mechanisms and so on but it's not uh
deep learning for the most part as it's
currently done it's
for the most part just identifying a
control hierarchy that is congruent to
the limbs that exist and the parameters
that need to be optimized for the
movement of these limbs and then there
is a convergence progress so it's
basically just regression that you would
need to control this but again i don't
know whether that's true that's just
what i've been told about how they work
we have to separate several
levels of discussions here so the only
thing they do is pretty sophisticated
control no with no machine learning
in order to be
to maintain balance or to write itself
it's a control problem in terms of using
the actuators to when it's pushed or
when it steps on a thing that's uneven
how to always maintain balance yes and
there's a tricky like set of heuristics
around that but
uh that's the only goal
everything you see boston dynamics doing
in terms of that to us humans is
compelling
which is any kind of um higher order
movement like turning
uh wiggling its butt
uh like uh you know uh
jumping back on its two feet
dancing the dancing is even worse
because dancing is hard coded in it's um
it's choreographed by humans there's
choreography software so like there is
no of all that high level movement
there's no
anything that you can call certainly
can't call ai but there's no uh
even like basic heuristics it's all hard
coded in and yet
we humans immediately project
agency onto them which is which is
fascinating so the gap here is uh it
doesn't necessarily have agency well it
has a cybernetic control and the
cybernetic control means you have a
hierarchy of feedback loops that keep
the behavior in certain boundaries so
the robot doesn't fall over and it's
able to perform the movements and the
choreography cannot really happen with
motion capture because the robot would
fall over because the physics of the
robot the weight distribution and so on
is different from the weight
distribution in the human body so if you
were using the directly motion captured
movements of the human body to project
it into this robot it wouldn't work you
can do this with the computer animation
it will look a little bit off but who
cares but
if you want to correct for the physics
you need to basically tell the robot
where it should move its limbs and then
the control algorithm is going to
approximate a solution that makes it
possible within the physics of the robot
and you have to find um the basic
solution for making that happen and
there's probably going to be some
regression necessary to get the control
architecture to to make these movements
but those two layers are separate yes
the the the thing the higher level
instruction of what how you should move
and where you should move is that so i
expect that the control level of these
robots at some level is dumb this is
just the
physical control movement the motor
architecture
but uh it's a relatively smart motor
architecture it's just that there is no
high level deliberation about what
decisions to make necessarily right but
see it doesn't feel like um free will
no that was not where i was trying to
get to i think that in our own uh
body we have that too so we have a
certain thing that is basically just a
cybernetic control architecture that is
moving our limbs
and
deep learning can help in discovering
such an architecture if you don't have
it in the first place
if you already know your hardware you
can maybe handcraft it
but if you don't know your hardware you
can search for such an architecture and
this work already existed in the
80s and 90s people were starting to
search for control architectures by
motor babbling and so on and just use
reinforcement learning architectures to
discover such a thing
and
now imagine that you have the cybernetic
control architecture already inside of
you and you extend this a little bit so
you are seeking out food for instance or
rest or and so on and
you get to have a baby at some point
and now you add more and more control
layers to this and the system is reverse
engineering its own control architecture
and builds a high level model to
synchronize the pursuit of very
different conflicting goals
and this is how i think you get to
purposes purposes are models of your
goals the goals may be intrinsic as the
result of the different set point
violations that you have hunger and
thirst for very different things and
rest and pain avoidance and so on and
you put all these things together and
eventually you need to come up with a
strategy to synchronize them all
and you don't need uh just to do this
alone by yourself because we are state
building organisms we cannot function in
the isolation the way that homo sapiens
is set up so our own behavior only makes
sense when you zoom out very far into a
society or even into ecosystemic
intelligence on the planet and our place
in it so the individual behavior only
makes sense in these larger contexts and
we have a number of priors built into us
so we are behaving as if we are acting
on these high level goals pretty much
right from the start
and eventually in the course of our life
we can reverse engineer the goals that
we are acting on what actually are our
higher level purposes
and the more we understand that the more
our behavior makes sense but this is all
at this point complex stories within
stories that are driving our behavior
yeah i just
don't know how big of a leap it is to
start uh create a system that's able to
tell stories within stories
like how big of a leap that is from
where currently boston dynamics is
or any robot that's operating in the
physical space
that and that leap might be
big if it requires to solve the hard
problem of consciousness which is
telling a hell of a good story
i suspect that um consciousness itself
is relatively simple what's hard is
perception
and the interface between perception and
reasoning
that's for instance the idea of
the consciousness prior that would be
built into such a system by uh joshua
bangio
and uh
what he describes and i think that's
accurate is
that
our own
model of the world can be described
through something like an energy
function the energy function is modeling
the contradictions that exist within the
model at any given point and you try to
minimize these contradictions the
tangents in the model
and to do this you need to sometimes
test things you need to conditionally
disambiguate figure and ground you need
to just
distinguish whether this is true or that
is true and so on eventually you get to
an interpretation but you will need to
manually depress a few points in your
model to let it snap into a state that
makes sense and this function that tries
to get the biggest dip in the energy
function in your model according to
joshua bangio is related to
consciousness it's a low dimensional
discrete function that tries to
maximize this dip in the energy function
i yeah
i think i would need to dig into details
because i think the way he uses the word
consciousness is more akin to like
self-awareness like modeling yourself
within the world
as opposed to the subjective experience
the hard problem no it's not even the
self is in the world the self is the
agent and you don't need to be aware of
yourself in order to be conscious
the self is just a particular content
that you can have but you don't have to
have
right you can be conscious in uh for
instance a dream at night or during a
meditation state but you don't have a
self
right where you're just aware of the
fact that you are aware and what we mean
by consciousness and
the colloquial sense is largely this
reflexive self-awareness
that we become aware of the fact that
you're paying attention
that we are the thing that pays
attention we are the thing that pays
attention right
i don't see where uh
the uh awareness that we're aware
the the heart problem doesn't feel like
it's solved i mean they they're they're
it it's called a hard problem for a
reason
because
it seems like there needs to be a major
leap
yeah i think the major leap is to
understand how it is possible that a
machine can dream
that the physical system
is able to create a representation that
the physical system is acting on and
that is spun force and so on but once
you accept the fact that you are not in
physics but that you exist inside of the
story i think the mystery disappears
everything is possible in a statement
exists inside the story okay so your
consciousness is being written into the
story the fact that you experience
things is written to the story you ask
yourself is this real what i'm seeing
and your brain writes into the story yes
it's real
so what about the perception of
consciousness so
to me you look conscious
so
um the illusion of consciousness the
demonstration of consciousness i ask for
the
the legged robot how do we make this
legged robot conscious
so
there's two things and maybe you can
tell me if they're neighboring ideas
one is actually make it conscious
and the other is make it appear
conscious to others
are those related
uh let's ask from the other direction
what would it take to make you not
conscious
so
when you are thinking about how you
perceive the world
can you decide to
switch from looking at qualia
to looking at representational states
and it turns out you can yeah there is a
particular way in which you can
look at the world and recognize its
machine nature including your own
and in that state you don't have that
conscious experience in this way anymore
it becomes
apparent as a representation everything
becomes opaque
and i think this thing that you
recognize everything as a representation
this is typically what we mean with
enlightenment states
and yeah you can't have a motivational
level but it you can also do this on the
experiential level and the perceptual
level see but then i can come back to a
conscious state
okay i particularly
i'm referring to the
social aspect that the demonstration of
consciousness
is a really nice thing at a party when
you're trying to meet a new person
it's it's a nice thing to to to know
that they're conscious and they can um
how i don't know how fundamental
consciousness is in human interaction
but it seems like to be at least
uh an important part and i i asked that
in the same kind of way
for robots
you know in order to create a rich
compelling human robot interaction it
feels like there needs to be elements of
consciousness within that interaction
my cat is obviously conscious
and
so my cat can do this party trick she
also knows that i am conscious be able
to have feedback about the fact that we
are both acting on models of our own
awareness the question is how hard is it
for
uh the robot
artificially created robot to achieve
cat level and
party tricks yes so the issue for me is
currently not so much on how to build a
system that creates a story about a
robot that lives in the world but to
make an adequate representation of the
world
and the model model that you and me have
is a unified one it's verb one where you
basically make sense of everything that
you can perceive every feature in the
world that enters your perception can be
relationally mapped to a unified model
of everything
and we don't have an ai that is able to
construct such a unified model yet
so you need that unified model to do the
party trick yes i think that uh you it
doesn't make sense if this thing is
conscious but not in the same universe
as you because you could not relate to
each other
so what's the process would you say of
engineering consciousness in the machine
like what are the ideas here
so uh you probably want to have some
kind of perceptual system this
perceptual system is a processing agent
that is able to track sensory data and
predict the next frame and the sensory
data from the
previous frames of the sensory data in
the current state of the system so the
current state of the system is
perception instrumental to predicting
what happens next
and this means you build lots and lots
of functions that take all the blips
that you feel on your skin and that you
see on your retina or that you hear
and
puts them into a set of relationships
that allows you to predict what kind of
sensory data what kind of sensor of
blips your vector of blips you're going
to perceive in the next frame right this
is tuned and it's
constantly tuned until it gets as
accurate as it can
you build a very accurate prediction
mechanism
that is step one of the perception so
first you predict then you perceive and
see the error in your prediction and you
have to do two things to make that
happen one is you have to build a
network of relationships that are
constraints
that
take all the variants in the world to
put each of the variances into a
variable variable that is connected with
relationships to other variables and
these relationships are computable
functions that constrain each other so
when you see a nose that points a
certain direction in space you have a
constraint that says there should be a
face nearby that has the same direction
right and if that is not the case you
have some kind of contradiction that you
need to resolve because it's probably
not a nose what you're looking at it
just looks like one so you have to
reinterpret the data and until you get
to a point where your model converges
and this process of making the sensory
data fit into your model structure is
what prg calls
the assimilation and accommodation is
the change of the models where you
change your model such a way that you
can assimilate everything
so you're you're talking about building
a hell of an awesome perception system
that's able to do prediction and
perception and correct and improvement
wait just uh if you had to wait there's
more yes there's more so the first thing
that we want to do is we want to
minimize the contradictions in the model
yes and of course it's very easy to make
a model in which you minimize the
contradictions just by allowing that it
can be in many many possible states
right so if you increase degrees of
freedom you will have fewer
contradictions but you also want to
reduce the degrees of freedom because
degrees of freedom mean uncertainty
you want your model to reduce
uncertainty as much as possible
but reducing uncertainty is expensive so
you have to have a trade-off between
minimizing contradictions and reducing
uncertainty and you have only finite
amount of compute and experimental time
and effort available to reduce
uncertainty in the world so you need to
assign value to what you observe
so you need some kind of motivational
system that is estimating what you
should be looking at and what you should
be thinking about it how you should be
applying your resources to model what
that is
right so you need to have something like
uh convergence links that tell you how
to get from the present state of the
model to the next one you need to have
these compatibility links that tell you
which constraints exist and which
constraint violations exist and you need
to have some kind of motivational system
that tells you what to pay attention to
so now we have a second agent next to
the perceptual age we have a
motivational agent
this is a cybernetic system that is
modeling what the system needs what's
important for the system and that
interacts with the perceptual system to
maximize the expected reward
and you're saying a motivational system
is some kind of
like what is it a higher level narrative
over some lower level no it's just your
brainstem stuff the limbic system stuff
that tells you okay now you should get
something to eat because i've just
measured your dual blood sugar like
motivational system like the lower
levels yes like hungry yes but there's
basically a physiological needs and some
cognitive needs and some social needs
and they all interact and they're all
implemented at different parts in your
nervous system as the motivational
system but they're basically cybernetic
feedback loops it's not that complicated
it's just a lot of code
and
so you now have a motivational agent
that makes your robot go for the ball or
that makes your worm go to eat
food and so on and you have the
perceptual system that lets it predict
that environment so it's able to solve
that control problem to some degree
and now what we learned is that it's
very hard to build a machine learning
system that looks at all the data
simultaneously to see what kind of
relationships could exist between them
so you need to selectively model the
world you need to figure out where can i
make the biggest difference if i would
put the following things together
sometimes you find a gradient for that
right when you have a gradient you don't
need to remember where you came from you
just follow the gradient until it
doesn't get any better but if you have a
word where the problems are
discontinuous and the search spaces are
discontinuous you need to retain memory
of what you explored and you need to
construct a plan of what to explore next
and this thing that means that you have
next to this perceptual construction
system and the motivational cybernetics
an agent that is paying attention to
what it should select at any given
moment to maximize reward and this
scanning system this attention agent
is required for consciousness and
consciousness it is its control model
so it's the index memories that this
thing retains when it manipulates the
perceptual representations
to
maximize the value and minimize the
conflicts and it to increase coherence
so the purpose of consciousness is to
create coherence in your perception
representations remove conflicts predict
the future construct counterfactual
representations so you can coordinate
your actions and so on
and in order to do this it needs to form
memories these memories are partial
binding states of the working memory
contents that are being revisited later
on to backtrack to undo certain states
to look for alternatives and these index
memories that you can recall that is
what you perceive as your stream of
consciousness and being able to recall
these memories this is what makes you
conscious if you could not remember what
you paid attention to you wouldn't be
conscious
so consciousness is the index in the
memory database okay
uh but
let me sneak up to the questions of
consciousness a little further
so we usually
relate suffering to consciousness
so the capacity to suffer
i think to me that's a really strong
sign of consciousness
is a thing that can suffer
how how is that useful
suffering
and like in your model what you just
described which is indexing of memories
and
what is the coherence with the
perception
uh with this predictive thing that's
going on the perception
how how does suffering relate to any of
that
you know the higher level suffering that
humans do
basically pain is a reinforcement signal
it pain
is a signal that one part of your brain
sends to another part of your brain or
an abstract sense part of your mind
sends to another part of the mind to
regulate its behavior to tell it the
behavior that you're currently
exhibiting should be improved
and this is the signal that i tell you
uh
to move away from what you're currently
doing and push into a different
direction so pain gives you a
part of you an impulse to do something
differently
but sometimes this doesn't work because
for the training part of your brain is
talking to the wrong region or because
it has the wrong model of the
relationships in the world maybe you're
mismodeling yourself or you're
mismodding the relationship of yourself
to the world or you're mismodeling the
dynamics of the world so you're trying
to improve something that cannot be
improved by generating more pain but the
system doesn't have any alternative
so the uh it doesn't get better what do
you do if something doesn't get better
and you want it to get better you
increase the strength of the signal and
then the signal becomes chronic when it
becomes permanent without a change
inside this is what we call suffering
and the purpose of consciousness is to
deal with contradictions with things
that cannot be resolved
the purpose of consciousness i think is
similar to a conductor in an orchestra
when everything works well the orchestra
doesn't need much of a conductor as long
as it's coherent but when there is a
lack of coherence or something is
consistently producing disharmony and
mismatches then the conductor becomes
alert and interacts with it so suffering
attracts the activity of our
consciousness
and the purpose of that is ideally that
we bring new layers online new layers of
modeling that
are able to create a model of the
dysregulation so we can deal with it
and this means that we typically get
higher level consciousness so to speak
right we get some consciousness above
our pay grade maybe if we have some
suffering early in our life most of the
interesting people had trauma early on
in their childhood and trauma means that
you are suffering an injury for which
the system is not prepared which it
cannot deal with which it cannot
insulate itself from so something breaks
and this means that the behavior of the
system is permanently
disturbed in the way that
some mismatch exists now in the
regulation that just by following your
impulses by following the pain in the
direction which it hurts the situation
doesn't improve but get worse
and so what needs to happen is that you
grow up
yeah that and that's part that has grown
up is able to deal with the part that is
stuck in this earlier phase yeah so it
leads to growth adding extra layers to
okay to your cognition uh
let me ask you then because i got to
stick on suffering uh the ethics of the
whole thing
so not our consciousness but the
consciousness of others you've uh
uh tweeted one of my biggest fears
is that insects could be conscious the
amount of suffering on earth would be
unthinkable
so
when we think of other conscious beings
is
suffering a property of consciousness
that we're most concerned about
so
i'm still thinking about robots
how to make sense of other non-human
things
that appear to have
the depth of experience that humans have
and to me
that means consciousness and the darkest
side of that which is suffering
the capacity to suffer
and so i start thinking
how much responsibility do we have for
those other conscious beings
that's where the um
the definition of consciousness
becomes most urgent like having to come
up with a definition of consciousness
becomes most urgent
is
who should we and should we not be
torturing
there's no general answer to this was
genghis khan doing anything wrong
it depends right on how you look at it
well he
he drew he drew a line somewhere
where this is us and that's them it's
the circle of empathy
it's like these
we don't have to use the word
consciousness but
these are the things that matter to me
if they suffer or not and these are the
things that don't matter yeah but when
one of his commanders failed him he uh
broke his spine and let him die
in a horrible way and uh so
in some sense i think he was indifferent
to suffering
or he was not different in the sense
that he didn't
see it as useful if he inflicted
suffering
but he did not see it as something that
had to be avoided that was not the goal
the question was how can i use suffering
and the infliction of suffering to reach
my goals from his perspective
i i see so like different societies
throughout history put different value
on
different individuals different psyches
but also even the uh the objective of
avoiding suffering like some society is
probably
i mean this is where like religious
belief really helps
that that afterlife
that doesn't matter that you suffer or
die what matters is you suffer honorably
right
so that you enter the afterlife
it seems to be superstitious to me
basically beliefs that
assert things uh for which no evidence
exists
are incompatible with sound epistemology
and i don't think that religion has to
be superstitious otherwise it should be
condemned okay in all cases you're
somebody who's saying we live in a dream
world we have zero evidence for anything
so that's not the case
there are limits to what languages can
be constructed mathematics brings solid
evidence for its own structure and once
we
have some idea of what languages exist
and how a system can learn and what
learning itself is in the first place
and so on we can
begin to realize that our intuitions
that we are able to learn about the
regularities of the world and minimize
the appraisal and understand the nature
of our own agency to some degree of
abstraction that's not an illusion so
useful approximation just because we
live in a dream world doesn't mean
mathematics can't
uh give us a
consistent glimpse of
uh physical of objective reality we can
basically distinguish useful encodings
from useless encodings
and when we
apply our truth seeking to the world we
know we usually cannot find out whether
a certain thing is true
what we typically do is we take the
state vector of the universe separate it
into separate objects that interact with
each other so interfaces and this
distinction that we are making is not
completely arbitrary it's done to
optimize the compression that we can
apply to our models of the universe so
we can predict what's happening with our
limited resources in this sense is not
arbitrary but the separation of the
world into objects that are somehow
discrete and interacting with each other
is not the true reality right the
boundaries between the objects are
projected into the world not arbitrarily
projected but still it's only an
approximation of what's actually the
case
and we sometimes notice that we run into
contradictions when we try to understand
high-level things like economic aspects
of the world and so on or political
aspects or psychological aspects where
we make simplifications and the objects
that we are using to separate the world
are just one of many possible
projections of what's going on
and so it's not in this postmodernist
sense completely arbitrary and you're
free to pick what you want or dismiss
what you don't like because it's all
stories no that's not true you have to
show for every model of how well it
predicts the world so the confidence
that you should have in the entities of
your models should correspond to the
evidence that you have
can i ask you in a small tangent
uh to talk about your um
favorite
set of ideas and people which is post
modernism
what is what is post-modernism how would
you define it and
why
to you is it not a useful framework of
thought
uh post-modernism is something that i'm
really not an expert on
and uh post-modernism is a a set of
philosophical ideas that it's difficult
to lamp together
that
is characterized by some useful thinkers
some of them post structuralist and so
on and i'm mostly not interested in it
because i i think that it's not leading
me anywhere that i find particularly
useful it's mostly i think born out of
the inside that the ontologies that we
impose on the world are not literally
true and that we can often get to a
different interpretation by the world by
using a different ontology that is
different separation of the world into
interacting objects
but the idea that this makes the world a
set of stories
that are arbitrary i think is wrong
and the people that are engaging in this
type of philosophy are working in in an
area that i largely don't find
productive there's nothing useful coming
out of this so this idea that truth is
relative is not something that has in
some sense informed physics or theory of
relativity and there is no feedback
between those there is no meaningful
influence of this type of philosophy on
the sciences or in engineering or in
politics but there is a very strong
information on
of this on
ideology
because it basically has become an
ideology that is justifying itself by
the notion that truth is a relative
concept
and it's not being used in such a way
that the the philosophers that or
sociologists that take up these ideas
say oh uh i should doubt my own ideas
because maybe my separation of the world
into objects is not completely valid and
they should maybe use a different one
and be open to a pluralism of ideas
but it's mostly exists to dismiss the
ideas of other people
it becomes yeah it becomes a political
weapon of sorts
to achieve power basically this uh
there's nothing wrong i think with uh
developing a philosophy around this but
to develop norms around the idea that
truth is something that is completely
negotiable is incompatible with the
scientific project
and i think if the uh if the academia
has no defense against the ideological
parts of
the postmodernist movement it's doomed
right you have to acknowledge
the ideological part of any movement
actually
uh including post-modernism well the
question is what an ideology is and to
me an ideology is basically a viral
memplex that is
changing your mind in such a way that
reality gets warped
it gets warped in such a way that you're
being cut off from the rest of human
thought space and you cannot consider
things outside of the range of ideas of
your own ideology it was as possibly
true right so i mean there's certain
properties to an ideology that make it
harmful one of them is that like
dogmatism
of just certainty
dogged certainty in in that you're right
you have the truth and nobody else but
what is creating the certainty it's very
interesting to look at the type of model
that is being produced is it basically
just to draw a strong prior and you tell
people oh this idea that you consider to
be very true the evidence for this is
actually just much weaker than you
thought and look here at some studies no
this is not how it works it's usually
normative which means
some thoughts are unthinkable because
they would change your identity into
something that is no longer acceptable
and this cuts you off from considering
an alternative and many uh de facto
religions use this trick to lock people
into a certain mode of thought this
removes agency over your own thoughts
it's very ugly to me it's basically not
just a process of domestication but it's
actually an intellectual
castration that happens it's an
inability to think creatively and to
bring forth new thoughts
can i ask you about
substances chemical substances that
affect the video game
the dream world
so psychedelics
that increasingly have been getting a
lot of research done on them so in
general psychedelics
psilocybin mdma but also really
interesting one
the big one which is dmt
what and where are the places that these
substances take
the mind
that is operating in the dream world
do you have an interesting sense how
this
throws a wrinkle
into the prediction model
is it just some weird little quirk or is
there
is there some fundamental expansion of
the mind going on
i suspect that a way to look at
psychedelics is that they induce
particular types of lucid dreaming
states
so it's a state in which
certain connections are being
severed in your mind when no longer
active your mind basically gets free to
move in a certain direction because some
inhibition some particular inhibition
doesn't work anymore
and as a result you might stop having
yourself or you might
stop perceiving the world as
three-dimensional
and
you can explore that state
and i suppose that for every state that
can be induced with psychedelics there
are people that are naturally in that
state
so
sometimes psychedelics that shift you
through a range of possible mental
states and they can also shift you out
of the range of permissible mental
states that is where you can make
predictive models of reality
and what i observe in people that use
psychedelics
a lot is that they tend to be over
fitting
overfitting means that you are
using more bits for modeling the
dynamics of a function than you should
and so you can fit your curve to
extremely detailed things in the past
but this model is no longer predictive
for the future
what is it about psychedelics that
forces that
i thought it would be the opposite i
thought i thought
uh that it's a it's a good mechanism for
uh
uh for generalization for regularization
so it feels like
psychedelics expansion of the mind like
taking you outside of like forcing your
model to be
uh non-predictive is a good thing
meaning like
uh it's almost like okay what i would uh
say psychedelics are akin to is
traveling to a totally different
environment
like going if you've never been to like
india or something like that from the
united states
very different set of people different
culture different food different roads
and
values and all those kinds of things
yeah so psychedelics can for instance
teleport people into
a universe that is uh hyperbolic which
means that if you imagine a room that
you are in you can turn around 360
degrees and you didn't go full circle
you need to go 720 degrees to go full
circle exactly so the things that people
learn in that state cannot be easily
transferred in this universe that we are
in
it could be that if they're able to
abstract and understand what happened to
them that they understand that some part
of their spatial cognition
has been desynchronized and has found a
different synchronization and this
different synchronization happens to be
a hyperbolic one right so you learn
something interesting about your brain
it's difficult to understand what
exactly happened but we get a pretty
good idea once we understand how the
brain is representing geometry yeah but
doesn't give you a fresh perspective on
the physical reality
who's making that sound
is inside my head or is it external
well there is no sound outside of your
mind but uh it's making sense
often on my nine physics
uh yeah in the physical reality there's
uh there's sound waves
traveling through air okay
that's our model of what happened
tomorrow what happened right
uh
that doesn't uh don't psychedelics give
you a fresh perspective on this physical
reality like uh not this physical
reality but this this more um
[Applause]
what do you call the dream world
that's mapped directly to purpose of
dreaming at night i think is yeah
theater augmentation well exactly so
that's very different that's a very
similar just change parameters uh about
the things that you have learned
and uh for instance when you are young
you have seen things from certain
perspectives but not from others so your
brain is generating new perspectives of
objects that you already know which
means they can learn to recognize them
later from different perspectives and i
suspect that's the reason why many of us
remember to have flying dreams as
children because it's just different
perspectives of the world that we
already know and that it it starts to
generate these
different perspective changes and then
it fluidly turns this into a flying
dream to make sense of what's happening
right so you fill in the gaps and
suddenly you see yourself flying
and
similar things can happen with semantic
relationships so it's not just spatial
relationships but it can also be the
relationships between ideas that are
being changed
and it seems that the mechanisms that
make that happen during dreaming
um are interacting with these same
receptors
that are being stimulated by
psychedelics
so uh i suspect that there is a thing
that i haven't read really about
the way in which dreams are induced in
the brain it's not just that the
activity of the brain gets tuned down
because you are somehow your eyes are
closed and you no longer get enough data
from your eyes but there is a particular
type of neurotransmitter that is
saturating your brain during these
phases during the rm phases and you
produce
controlled hallucinations
and psychedelics are linking into these
mechanisms
i suspect
so there's an
another trickier form of data
augmentation yes
but uh it's also data augmentation that
can happen outside of the specification
that your brain is tuned to so basically
people are overclocking their brains and
that
that produces states that are
subjectively extremely interesting
yeah i just but from the outset very
suspicious so
i think i'm over applying the metaphor
of a neural network in my own mind
which
i just think that doesn't lead to
overfitting right but
but you were just sort of anecdotally
saying my experiences with people that
have no psychedelics or that that kind
of quality i think it typically happens
so if you look at people like uh timothy
leary
and he has written beautiful manifestos
about
the effect of lsd on people he genuinely
believed he writes in these manifestos
that in the future science and art will
only be done on psychedelics because
it's so much more efficient and so much
better and he gave
lsd to children in this community of a
few thousand people that he had near san
francisco and
basically he was losing touch
with reality he did not understand the
effects that the things that he was
doing would have on the reception of
psychedelics by society because he was
unable to think critically about what
happened what happened was that he got
in an euphoric state that euphoric state
happened because he was overfitting he
was taking this sense of euphoria and
translating it into a model of actual
success in the world right he was
feeling better
limitations had disappeared that he exp
appearance to be existing but he didn't
get superpowers i understand what you
mean by overfitting now
there's a lot of interpretation to the
term overfitting in this case but i i
got you so he was getting
he was getting uh positive rewards from
a lot of actions this year but not just
this so if you take for instance john
lilly who um was studying dolphin
languages and aliens and so on yeah a
lot of people that use psychedelics
became very loopy
and the typical thing that you notice
when people are in psychedelics is that
they are in a state where they feel that
everything can be explained now
everything is clear everything is
obvious yeah and uh sometimes they have
indeed discovered a useful connection
but not always very often these
connections are over interpretations i
wonder
you know there's a question of uh
correlation versus causation
and also i wonder if it's the
psychedelics or if it's more the social
like being the outsider
uh and having a strong community
of outside and being having a leadership
position in an outside occult-like
community that could have a much
stronger effect of overfitting than do
psychedelics themselves the actual
substances
because it's a counter culture thing so
it could be that as opposed to the
actual substance if you're a boring
person who wears a suit and tie and
works at a bank
and takes psychedelics that could be a
very different effect of psychedelics on
on your mind it i'm just sort of raising
the point that the people you referenced
are already weirdos i'm not sure exactly
oh no not necessarily a lot of the
people that uh tell me that they use
psychedelics in a useful way
started out as squares and were
liberating themselves because they were
stuck
they were basically stuck in local
optimum of their own self model of their
relationship to the world and suddenly
they had data augmentation they
basically saw as an experience the space
of possibilities they experienced what
it would be like to be another person
yeah and they took uh
important lessons from that experience
back home
yeah i mean uh i love the the the
metaphor of data augmentation because
that's uh
been the the primary driver of
self-supervised learning in the vision
computer vision domain is data
augmentation so it's funny to think of
data augment like like chemically
induced
data augmentation in the human mind
there's also a very interesting effect
that i uh
noticed i've i know uh several people
who are sphere to me that
lsd has cured their migraines so
severe cluster eight headaches or
migraines that didn't respond to
standard medication that disappeared
after a single dose and i don't
recommend anybody doing this especially
not in the us where it's illegal uh and
there are no studies on this for that
reason but uh it seems that uh
anecdotally that it basically can reset
the serotonergic system so it's
basically
pushing them outside of their normal
boundaries and as a result it needs to
find a new equilibrium and in some
people that equilibrium is better but it
also follows that in other people it
might be worse so if you have a brain
that is already teetering on the
boundary to psychosis
it can be permanently pushed over that
boundary well that's why you have to do
good science which they're starting to
do on all these different substances of
how well it actually works for the
different conditions like mdma seems to
help with ptsd
uh same with psilocybin
that you know you need to do good
science meaning large studies of large
and yeah so based on the existing
studies with mdma it seems that
if you look at rick doblin's work and
what he has published about this and
talks about
mdma seems to be a psychologically
relatively safe drug but it's
physiologically not very safe that is uh
there is a neurotoxicity if you would
use two large dose and if you uh combine
this with alcohol which a lot of kids do
in uh party settings during raves and so
on it's very hepatotoxic so basically
you can kill your liver and this means
that it's probably something that is
best and most productively used in
clinical setting
by people who really know what they're
doing and i suspect that's also true for
the other psychedelics that is
uh while the other second eggs are
probably not as toxic as say alcohol
the effects on niseki can be much more
profound and lasting
yeah well as far as i know psilocybin so
mushrooms magic mushrooms
as far as i know in terms of the studies
they're running i think have no over
like they're allowed to do what they're
calling heroic doses so that one does
not have a toxicity so they could do
like huge doses in a clinical setting
when they're doing study on psilocybin
which is kind of fun
yeah it seems that most of the
psychedelics work in extremely small
doses
which means that the effect on the rest
of the body is relatively low and mdma
is probably the exception maybe ketamine
can be dangerous and larger doses
because it can
depress breathing and so on but
the
lsd and psilocybin work in very very
small doses at least the active part of
them
of um lsd is only the active part
and the but the effect that you it can
have on your mental wiring can be very
dangerous i think
let's talk about ai a little bit
what are your thoughts
about gpt3 and
language models trained with
self-supervised learning
it came out quite a bit ago but i wanted
to get your thoughts on it yeah
in the 90s i was in new zealand and
i had an amazing professor ian witton
who realized there was ward in class and
put me in his lab and he gave me the
task to discover grammatical structure
in an unknown language
and the unknown language that i picked
was english because it was the easiest
one to find corpus four construct one
and he gave me the largest computer at
the whole university it had two
gigabytes of ram which was amazing and i
wrote everything in c with some
in-memory compression to do statistics
over the language
and
first would
create a dictionary of all the words
which basically tokenizes everything and
compresses things so they don't need to
store the whole world but just
a code for every word and then i was um
taking this all apart in sentences and i
was trying to
find all the relationships between all
the words in the sentences and do
statistics over them
and that proved to be impossible because
the complexity is just too large so if
you want to discover the relationship
between an article and a noun and there
are three adjectives in between you
cannot do engram statistics and look at
all the possibilities that can exist at
least not with the resources that we had
back then
so i realized i need to make some
statistics over what i need to make
statistics over so i wrote something
that was pretty much a hack
that did this for um at least first
order relationships and i came up with
some kind of mutual information graph
that was indeed discovering uh something
that looks exactly like the grammatical
structure of the sentence just by trying
to encode the sentence in such a way
that the words would be written in the
optimal order
inside of the model and
what i also found is that if we would be
able to increase the resolution of that
and
not just use this model to reproduce
grammatically correct sentences we would
also be able to correct stylistically
correct sentences by just having more
bits in these relationships and if we
wanted to have meaning we would have to
go much higher order
and i didn't know how to make higher
order models back then without spending
way more years in research on how to
make the statistics over what we need to
make statistics over
and
this thing that we cannot look at the
relationships between all the bits in
your input is being solved in different
domains in different ways so in computer
graphics the
computer vision standard methods for
many years now is convolutional neural
networks convolutional neural networks
are hierarchies of filters
that exploit the fact that neighboring
pixels in images are usually
semantically related and distance pixels
and images are usually not semantically
related
so you can just by grouping the pixels
that are next to each other
hierarchically together reconstruct the
shape of objects
and this is an important prior that we
built into these models so they can
converge quickly but this doesn't work
in language for the reason that adjacent
words are often but not always related
and distant words are sometimes related
while the words in between are not
right so how can you learn the topology
of language
and i think for for this reason that
this difficulty existed the transformer
was invented in
natural language processing not in
vision
and what the transformer is doing it's a
hierarchy of layers where every layer
learns what to pay attention to in the
given context in the previous layer
so what to make the statistics over
and the context is
significantly larger than the adjacent
word yes so the context that this um
that gpt3 has been using the transformer
itself is from 2017 and it's uh wasn't
using that large of a context
openai has basically scaled up this idea
as far as they could at the time and the
context is
about 2048
symbols tokens in the language these
symbols are not characters but they take
the words and project them into a vector
space
where
words that are statistically
co-occurring a lot are neighbors already
so it's already a simplification of the
problem a little bit
and so every word is basically a set of
coordinates in a high dimensional space
and then they use some kind of trick to
also encode the order of the words in a
sentence or in the not just sentence but
2048 tokens is about couple pages of
text or two and a half pages of text
and so they managed to do pretty
exhaustive statistics over the potential
relationships
between two pages of text which is
tremendous right i was just using a
single sentence back then
and
i was
only looking for first order
relationships and there were really
looking for much much higher level
relationships and what they discover
after they fed this with an enormous
amount of treatment in their pretty much
the written internet or a subset of it
that had some quality but
substantial portion of the common draw
that uh they're not only able to
reproduce style but they're also able to
reproduce some pretty detailed semantics
like being able to um add three digit
numbers and multiply two digit numbers
or to translate between programming
languages and things like that
so the results that gbt3 got i think
were amazing by the way um
i actually didn't check carefully it's
funny you just mentioned
how you coupled semantics to the
multiplication is it able to do some
basic math on two uh uh two digit
numbers yes
okay interesting i thought
i thought there's a lot of
failure cases yeah but basically it
fails if you take larger digit numbers
so four digit numbers and so on uh makes
carrying mistakes and so on and if you
take large larger numbers you don't get
useful results at all
and
this could be
an issue of the training set
down.net many examples of successful
long-form edition and standard uh human
written text and humans aren't very good
at doing three-digit numbers either yeah
they're not you're not writing a lot
about it yeah and
the other thing is that the loss
function that is being used is only
minimizing surprises so it's predicting
what comes next in the typical text it's
not trying to go for causal closure
first as we do yeah
and but the fact that that kind of
prediction works
to generate text that's semantically
rich and consistent
is interesting yeah so yeah so it's
amazing that it's able to uh
generate symmetrically consistent text
it's not consistent so the problem is
that it loses coherence at some point
but it's also i think not correct to say
that gptc3 is unable to deal with
semantics at all because you ask it to
perform certain transformations in text
and it performs this transformation and
text and the kind of additions that it's
able to perform are
transformations and text right and
there are proper semantics involved you
can also do more there was a paper that
was generating
lots and lots of mathematically correct
text
and was feeding this into a transformer
and as a result it was able to learn how
to do differentiation integration in
race that according to the authors
mathematica could not
to which some of the people in
mathematica responded that uh they were
not using the mathematica in the right
way and so on i have not really followed
the this resolution of this conflict
this this part as a small tangent i
really don't like in machine learning
papers which
they often
do um
anecdotal evidence they'll find like one
example in some kind of specific use of
mathematica and demonstrate look here's
they'll show successes and failures but
they won't have a very clear
representation of how many cases this
actually represents yes but i think as a
first paper this is a pretty good start
and yeah so uh the whole message is that
the authors could get better results
from this in their
experiments than they could get from the
vein which they were using computer
ultra systems which means uh that was
not nothing yeah and uh it's able to
perform substantially better uh than
gptsv can based on a much larger amount
of training data using the same
underlying algorithm well let me let me
ask
again so i'm using your tweets as if
this is like plato right
as if this is
well thought out novels that you've
written
uh you tweeted gpt4 is listening to us
now
um
this is one way of asking what are the
limitations of gpt-3
when it scales so what do you think will
be the capabilities of gpt-4 gpt-5 and
so on what are the limits of this
approach so uh obviously when we are
writing things right now uh everything
that we are writing now is going to be
training data for the next generation of
machine learning models so yes of course
tbt4 is listening to us and i think the
tweet is already a little bit older and
the we now have wu dao and we have a
number of other systems that basically
are placeholders for gpt4 don't know
what open ais plans are in the card i
read that tweet
in several ways so one is obviously
everything you put on the internet is
used as training data
but
in the second way i read it is
in a uh
we talked about agency
i read it as almost like gpt4 is
intelligent enough
to be choosing to listen
so not only like did a programmer tell
it to collect this data and use it for
training i almost
saw the humorous angle which is like it
has achieved agi
kind of thing well the thing is um could
we be already be living in gpt5
[Laughter]
so gpt4 is listening and dpt5 actually
constructed the entirety of the
reality we in some sense the what
everybody is trying to do right now in
ai is to extend the transformer to be
able to deal with video
and uh there are very promising
extensions where there is a work by
google that is called perceiver
and that is
overcoming some of the limitations of
the transformer by letting it learn the
topology of the different modalities
separately
and by
training it to find better input
features so these basically feature
abstractions that are being used by
this um successor or to gpt3 are chosen
such a way that it's able to deal with
video input
and there is more to be done so i one of
the limitations of gpt three is that
it's uh amnesiac so it forgets
everything beyond the two pages that it
currently reads also during generation
not just during learning
do you think that's fixable
within the space of deep learning
can you just make a bigger bigger bigger
input no uh i don't think that our own
uh working memory is infinitely large
it's probably also just a few thousand
bits
but uh what you can do is you can
structure this working memory so instead
of just
force feeding this thing a certain thing
that it has to focus on and it's not
allowed to focus on anything else with
its network
you allow it to construct its own
working memory as we do right when we
are reading a book
it's not that we are focusing our
attention in such a way that we can only
remember the current page
we will also try to remember other pages
and try to undo what we learned from
them or modify what we learned from them
we might get up and take another book
from the shelf we might go out and ask
somebody and we can
edit our working memory in any way that
is useful to put a context together
allows us to draw the right inferences
and to learn the right things
so this ability to perform experiments
on the world based on
an attempt to become fully coherent and
to achieve causal closure to achieve a
certain aesthetic of your modeling that
is something that eventually needs to be
done and at the moment we are skirting
this in some sense by building systems
that are larger and faster so they can
use dramatically larger resources and
human beings can do and much more
training data to get to models that in
some sense are already very superhuman
and in other ways are laughingly
incoherent
so do you think uh sort of
making um the systems like what would
you say multi-resolutional so like
some
uh some of the language models are
focused on
two pages some are focused on
uh two books some are focused on
two years of reading some are focused on
a lifetime like so it's like stacks of
it's the gpt3s all the way down you want
to have gaps in between them so it's not
necessarily two years there's no gaps it
stinks out of two years or out of twenty
years or two thousand years or two
billion years yeah where you are just
selecting those bits
that are predicted to be the most useful
ones to understand what you're currently
doing and this prediction itself
requires a very complicated model that's
the actual model that you need to be
making it's not just that you are trying
to understand the relationships between
things but what you need to make
relationships or discover relationships
over
i wonder what that
thing looks like with the architecture
for that for the thing that's able to
have that kind of model that i think it
needs more degrees of freedom than the
current models have so it starts out
with the fact that you
possibly don't just want to have a feed
forward model but you want it to be
fully recurrent
and to make it fully recurrent you
probably need to loop it back into
itself and allow it to skip connections
once you do this
right when you are predicting the next
frame and your internal next frame
in every moment
and you
are able to skip connection it means
that signals can travel from the output
of the uh network into the middle of the
network faster than the inputs do you
think it can still be differentiable do
you think it still could be a neural
network sometimes it can and sometimes
it cannot so it it can still be a neural
network but not a fully differentiable
one and when you want to deal with
non-differentiable ones you need to have
an attention system that is discrete and
dual dimensional and can perform
grammatical operations you need to be
able to perform program synthesis you
need to be able to backtrack in this
operations that you perform on this
thing this thing needs a model of what
it's currently doing and i think this is
exactly the purpose of our own
consciousness
yeah the program things that triculo
networks
so let me ask you it's not quite program
synthesis but uh the application of
these language models to generation
to program synthesis but generation of
programs so if you look at github open
pilot
which is based on openai's codex i don't
know if you got a chance to look at it
but it's
the system that's able to generate code
once you uh prompt it with
what is it like the header of a function
with some comments and it seems to do an
incredibly good job
or
not a perfect job is very important but
an incredibly good job of generating
functions
what do you make of that are you is this
exciting or is this just a party trick a
demo
or is this revolutionary
i haven't worked with it yet so it's
difficult for me to judge it but i would
not be surprised if it turns out to be
revolutionary that's because the
majority of programming tasks that are
being done in the industry right now are
not creative
yeah people are writing code that other
people have written or they're putting
things together from code fragments that
others have had and a lot of the work
that program has done practice is to
figure out how to overcome the gaps in
their current knowledge and
the things that people have already done
how to copy and paste from stuck over
that's right and so of course we can
automate that yeah
to uh make it much faster to copy and
paste from stack overflow yes but it's
not just copying and pasting it's also
basically learning which parts you need
to modify
to make them fit together
yeah
uh like literally sometimes as simple as
just changing the variable names
so it fits into the rest of your code
yes but this requires that you
understand the semantics of what you're
doing to some degree yeah
and you can automate some of those
things yes the the thing that makes
people nervous of course is that
a little bit wrong in a program
can have a dramatic effect on the actual
final operation of that program so it's
one little error
which in in the space of language it
doesn't really matter but in the space
of programs
can matter a lot yes but this is already
what is happening when humans program
code yeah this is so we have a
technology to deal with this
somehow it becomes scarier
when you know that a program generated
code that's running a nuclear power
plant
it becomes scarier you know humans have
errors too
exactly but
it's scarier when a program is doing it
because
why why i mean there's a
there's a fear that a program
like a program may not be
as good as humans to know when stuff is
important to not mess up
like
there's a misalignment of priorities
of values that's potential that maybe
that's the source of the worry
i mean okay if i give you code generated
by
uh
github open pilot and code generated by
a human and say here
use use one of these which which how do
you select today and in the next 10
years which code to use wouldn't you
still be comfortable with the human
at the moment when you go to stanford to
get an mri
they will write a bill to the insurance
over twenty thousand dollars
and of this maybe half of that gets paid
by the insurance and the quarter gets
paid by you yeah and the mri cost them
six hundred dollars to make maybe
probably less
and
what are the values of the person that
writes the software and deploys this
process
it's very difficult for me to say
whether i trust people
i think that what happens there is a
mixture of proper anglo-saxon protestant
values where somebody is trying to serve
an abstract greater whole and organized
crime well that's a very harsh you're
you're um
i think
that's a harsh view of humanity there's
a lot of bad people whether incompetent
or just malevolent in this world yes
but it feels like the more
malevolent you so the more damage you do
to the world
uh the more resistance you have in your
own
human
like
but cannot explain with malevolence or
stupidity what can be explained by just
people acting on their incentives
right so what happens in stanford is not
that somebody is evil
it's just that they do what they're
being paid for
no and this is that's not evil that's i
i tend to so no i see that as
malevolence i see uh
as i
uh even like being a good german as i
told you offline is some it's not
it's not
absolute malevolence but it's a small
amount it's cowardice
i mean when you see there's something
wrong with the world
um it's either incompetence that you're
not able to see it
or it's cowardice that you're not able
to stand up not it's not necessarily in
a big way but in a small way
so i
i do think that is in a bit of
malevolence i'm not sure the example
you're describing is the question is
what is it that you are aiming for and
if you don't believe in the future
if you for instance think that the
dollar is going to crash by what you try
to save dollars
if you uh don't think that humanity will
be around in 100 years from now because
uh global warming will wipe out
civilization why would you need to act
as if it were
right so the question is is there an
overarching aesthetics
that is projecting you and the world
into the future which i think is the
basic idea of religion that you
understand the interactions that we have
with each other as some kind of
civilization level agent that is
projecting itself into the future if you
if you don't have that shared purpose
uh right what is there to be ethical for
so i think when we talk about
essex and ai we need to go beyond the
insane bias discussions and so on where
people are just measuring the distance
between a statistic to their uh
preferred um
current world model but the optimism
i was a little confused by the previous
things just to clarify
it's
there is a kind of underlying
morality to
having an optimism that human
civilization will persist for longer
than a hundred years like that like uh i
think a lot of people believe
that it's it's a good thing for us to
keep living yeah of course and thriving
morality itself is not an end to itself
it's instrumental to people living in
100 years from now
right or 500 years from now right so uh
it's only justifiable if you actually
think that it will lead to people uh or
increase the probability of people being
around in that time frame and a lot of
people don't actually believe that at
least not actively
but i believe what exactly so
most people don't believe that they can
afford to act on such a model
basically what happens in in the u.s is
i think that the healthcare system is
for a lot of people no longer
sustainable which means that if they
need the help of the healthcare system
they're often not able to afford it yeah
and when they cannot help it they are
often going bankrupt it's i think the
leading course cause of personal
bankruptcy in the us is the healthcare
system
yeah and uh that would not be necessary
it's not because people are consuming
more and more uh medical services and
are achieving a much much longer life as
a result that's not actually the story
that is happening because you can
compare to other countries and life
expectancy in the u.s is currently not
increasing and it's not as high as in
all the other industrialized countries
so some industrialized countries are
doing better with a much cheaper
healthcare system
and
what you can see is for instance
administrative bloat the
healthcare system has maybe to some
degree deliberately set up as a job
placement program to allow people to
continue living in middle class
existence despite not having useful and
use case in productivity so they are
being paid to push paper around and the
number of administrators in the
healthcare system has been increasing
much faster than the number of
practitioners
and this is something that you have to
pay for right and also the
revenues that are being generated in the
healthcare system are relatively large
and somebody has to pay for them and
the result why they are so large is
because market mechanisms are not
working right the fda is largely not
protecting people from malpractice
of healthcare providers the fda is
protecting healthcare providers from
competition right okay so this is a
thing that is has to do with values and
this is not because people are malicious
on all levels it's because they are not
incentivized to act on a greater whole
on this idea that you treat somebody who
comes to you as a patient like you would
treat a family member yeah but yeah but
we're trying i mean
you're highlighting a lot of the flaws
of the different institutions the
systems we're operating under but i
think there's a continuum throughout
history
mechanism design
of trying to design incentives in such a
way that these systems behave better and
better and better i mean it's a very
difficult thing to operate
a society of hundreds of millions of
people effectively with yes
so do we live in a society that is ever
correcting is this uh right do we
observe that our models of what we are
doing are predictive of the future and
when they are not we improve them
our laws
adjudicated with clauses that you put
into every law what is meant to be
achieved by that law and the law will be
automatically repealed if it's not
achieving that right if you are
optimizing your own laws if you're
writing your own source code
you probably
make an estimate of what is the thing
that's currently wrong in my life what
is it that i should change about my own
policies what is the expected outcome
and if that outcome does manifest
i will change the policy back right or i
will change it to something different
are we doing this on a societal level i
think so i think i think it's easy to
sort of highlight the i think we're
doing it in the way that um
like i operate my current life i didn't
sleep much last night uh
you would say that lex the way you need
to operate your life is you need to
always get sleep the fact that you
didn't sleep last night is is totally
the wrong way to operate in your life
like you should have
gotten all your done in time and
gotten to sleep because sleep is very
important for health and you're
highlighting look this person is not
sleeping look the the medical the health
care system is operating poor but the
point is that we just
it seems like this is the way especially
in the capital society we operate we
keep running into trouble and last
minute we we try to get our our way out
through innovation and it seems to work
you have a lot of people that ultimately
are trying to build a better world and
and get uh urgency about them
when it's the problem becomes more and
more imminent
and that's the way this operates but if
you look at the
the
the history the the long arc of history
it seems like that
operating on deadlines
produces progress and builds better and
better systems you probably agree with
me that the us should have engaged in
mass production in january 2020
and that we should have shut down the
airports early on and that we should
have made it mandatory that the people
that work in nursery homes are
living on campus rather than living at
home and then uh coming in and infecting
uh people in the nursing homes that had
no immune response to covet
and uh
that is something that was i think
visible back then the correct decisions
haven't been made
we would have the same situation again
how do we know that these wrong
decisions are not being made again have
the people that made the decisions to
not protect the nursing homes been
punished
has have the people that made the wrong
decisions with respect to testing that
prevented the development of testing by
startup companies and the importing of
tests from countries that already had
them have these people been held
responsible well first of all so what do
you what do you want to like
put before the firing squad i think they
are no just make sure that this doesn't
happen again no but
you see but it's not that
yes they're being held responsible by
many voices by people being frustrated
there's new leaders being born now
they're going to see rise to the top in
10 years
this moves slower than
there's obviously a lot of uh
older incompetence and bureaucracy and
these systems move slowly
they move you know like science one
death at a time so like
yes i think the the pain has been felt
in the previous year
is reverberating throughout the world
maybe i'm getting old i suspect that
every generation in the us after the war
has lost the plot even more i don't see
this development the war world war ii
yeah so basically there was a time when
uh we were modernist and in this
modernist time
the u.s felt actively threatened by the
things that happened in the world the us
was worried about possibility of failure
and this imminence of possible failure
led to decisions where there was a time
when the government would listen to
physicists about how to do things yeah
and the physicists were actually
concerned about what the government
should be doing so they would be writing
letters to the government and so for
instance the decision for the manhattan
project was something that was driven in
the conversation between physicists and
the government i don't think such a
discussion would take place today i i
disagree i think if the virus was much
deadlier we would see a very different
response i think the virus was not
sufficiently deadly and instead because
it wasn't very deadly what happened is
uh the the current system started to
politicize it the mask this is what i
realized with masks early on they were
not
very quickly became not as a solution
but they became a thing
that politicians used to divide the
country so that that same thing is
happening with vaccines same thing so
like nobody's really people weren't
talking about solutions to this problem
because i don't think the problem was
bad enough when you talk about the world
the war i think what i think our lives
are too comfortable
i think in uh in the developed world
things are too good and we have not
faced severe dangers
one the dangers the severe dangers
existential threats are faced that's
when we step up on a small scale and a
large scale now i don't
i
that's sort of my argument here but i
did think the virus is
i was hoping that it was actually
sufficiently
uh dangerous
for us to step up because especially in
the early days it was unclear it still
is unclear because of mutations
how bad it might be
right and so
i thought we would step up and
even so the masks point is is uh it's a
tricky one because
to me the manufacturer of masks isn't
isn't even the problem i'm still to this
day and i was involved with with a bunch
of this work have not seen good signs
done on whether masks work or not
like there still has not been a
large-scale study to me that should be
there should be large-scale studies and
every possible solution like aggressive
in the same way that the vaccine
development was aggressive there should
be
masks which test what kind of tests work
really well uh what kind of uh like even
the question of how the virus spreads
there should be aggressive studies on
that to understand i'm still
as far as far as i know there's still a
lot of uncertainty about that nobody
wants to see this as an engineering
problem that needs to be solved it's
it's uh
that i was surprised about but i find
that our views are largely convergent
but not completely so i agree with the
thing that
uh because our society in some sense
perceives itself as too big to fail
right and uh the virus did not alert
people to the fact that we are facing
possible failure
uh that basically put us into the
post-modernist mode and i don't mean in
the philosophical sense but in a
societal sense the difference between a
post-modern society and the modern
society is that the modernist society
has to deal with the ground truth and
the postmodernist society has to deal
with appearances
politics becomes a performance and the
performance is done for an audience and
the organized audience is the media and
the media evaluates itself via other
media right so you have an audience of
critics that evaluate themselves and i
don't think it's so much the failure of
the politicians because to get in power
and to stay in power you need to be able
to deal with the published opinion well
i think it goes in cycles because the
what's going to happen
is all of the small business owners all
the people who truly are suffering and
will suffer more because the effects
of the closure of the economy and the
lack of solutions to the virus
they're going to uprise and hopefully
i mean this is this is where charismatic
leaders can get the the world in trouble
but hopefully we'll
elect great leaders
that will break through this
post-modernist idea of uh
of uh
the media and the perception and the
drama on twitter and all that kind of
stuff but you know this can go either
way yeah when the weimar republic was
unable to
deal with the economic crisis that
germany was facing
there was an option to go back
but there were people which thought
let's get back to a constitutional
monarchy and let's get this this to work
because democracy doesn't work
and eventually uh there was no way back
where people decided there was no way
back they needed to go forward and the
only options for going forward was to
become a stalinist communist basically
an option to completely expropriate
the factories and so on and nationalize
them and to reorganize
germany and communist terms and ally
itself with stalin
and fascism and both options were
obviously very bad
and the one that the germans picked led
to a catastrophe that was
that devastated europe
and i'm not sure if the us has an immune
response against that i think that the
far right is currently very weak in the
u.s but this can easily change
do you think uh
from a historical perspective hitler
could have been stopped
from within germany or or from outside
or this
well depends on who you want to focus
whether you want to focus on stalin or
hitler but he feels like hitler was the
one
as a political movement that could have
been stopped
i think that uh
the point was that a lot of people
wanted hitler so he got support from a
lot of quarters it was a number of
industrialists who supported him because
they thought that the democracy is
obviously not working and unstable and
you need a strong man
and he was willing to play that part
there were also people in the us who
thought that hitler would stop uh stalin
and would act as a balrack against
bolshevism
which he probably would have done right
but uh at which cost
and then many of the things that he was
going to do like the holocaust
was
something where people thought this is
rhetoric he's not actually going to do
this right especially many of the jews
themselves which were humanist and for
them this was outside of the scope that
was thinkable right
i mean i i i wonder
if hitler is uniquely
i i want to carefully use this term but
uniquely evil
so if hitler was never born
if somebody else would come in this
place
so like
just
thinking about the progress of history
how important are those singular figures
that
that lead to mass destruction and
cruelty
because
my senses
hitler was unique
the
it wasn't just about the environment and
the context that gave him
like
like another person would not come in
his place to do as destructive of the
things that he did
there was a combination of uh charisma
of madness of psychopathy of just uh ego
all those things which are very unlikely
to come together in one person in the
right time
it also depends on the context of the
country that you're operating in
if you uh tell the germans that they
have a historical destiny uh
in this romantic country the effect is
probably different than it is in other
countries
but uh the uh
stalin has killed a few more people than
hitler did and if you look at the
probability that you're survived under
stalin
uh right uh hitler killed people if uh
they if he thought they were not worth
living or if they were harmful to his uh
racist uh project right he basically
felt that the jews would be too
cosmopolitan and would not be willing to
uh participate in the racist
redefinition of society and the value of
society and an isno state in this way
that as he wanted it to have it so they
he saw them as
a
harmful danger especially since they
played such an important role in the
economy and culture of germany
and uh
it so we had basically had some radical
but rational reason to murder them
and stalin just killed everyone
basically the stalinist purchase were
such a random thing where he said that
there's a
certain possibility that this particular
part of the population has a number of
german collaborators or something and we
just kill them all yeah right well if
you look at what mao did the number of
uh people that were killed absolute in
absolute numbers were much higher under
mao that they were under stalin
so it's super hard to say the the the
other thing is that you look at jinga's
khan and so on how many people he killed
where do you see there are a number of
of things that happen in human history
that actually really put a substantial
dent in the existing population or
napoleon
and
it's it's very difficult to eventually
measure it because what's happening is
basically evolution
uh on a human scale
where uh one monkey figures out a way to
become viral
and uh is
using this viral technology to change
the patterns of society at the very very
large scale
and what we find so abhorrent about
these changes is the complexity that is
being destroyed by this that it's
basically like a big fire that burns out
a lot of the existing culture and
structure that existed before
yeah and it all just starts with one
monkey
one one cares mata gabe and there's a
bunch of them throughout history yeah
but it's in a given environment it's
basically similar to wildfires in
california right the temperature is
rising
there is less rain falling
and then certainly a single spark can
have an effect that in other times would
be contained
okay
speaking of which i i love when we went
to hitler and stalin from 20 30 minutes
ago uh gpt 3 generating
doing program synthesis the argument was
about morality of ai versus human
so
um
and specifically in the context of
writing programs specifically in the
context of programs that can be
destructive so running nuclear power
plants or autonomous weapons systems for
example
and i think
your inclination was to say that it's
not so obvious that ai would be less
moral than humans
or less effective at making a a world
that would make humans happy so i i'm
not talking about self-directed systems
that are
making their own goals at a global scale
if you just talk about the deployment of
technological systems that are able to
see order and patterns and use this as
control models to act on the goals that
we give them
then if we have the correct incentives
to set the correct incentives for these
systems i'm quite optimistic
so but
so humans versus ai let me give you an
example
autonomous weapon system
let's say there's a city somewhere in
the middle east
that has a number of terrorists
and the question is what's currently
done with with drone technologies you
have information about the location of a
particular terrorist and you have a
targeted attack you have a bombing of
that particular building
and that's all directed by humans at the
high level strategy and also at the
deployment of individual bombs and
missiles like that the actual
everything is done by human except the
the final targeting
and the the like the country so like
with spot similar thing like control the
flight
okay
what if you give ai control and saying
um write a program that says here's the
best information i have available about
the location of these five terrorists
here's the city make sure it's cons all
the bombing you do is constrained to the
city make sure it's precision based
but you take care of it so you do one
level of abstraction out and saying
take care of the terrorists in the city
which are you more comfortable with the
humans
or the javascript gpt-3 generated code
that's doing the
deployment i mean that's this is the the
kind of
question i'm asking is
the kind of bugs that we see in human
nature are they better or worse than the
kind of bugs we see in ai
they're different bugs there is an issue
that if people are
creating an in imperfect automation
of a process that normally requires a
moral judgment
and this mobile judgment is
the reason why it cannot be automated
often is not because the
computation is too expensive
but because the model that you give the
ai is not an adequate model of the
dynamics of the world because the ai
does not understand the context that
it's operating in in the right way and
this is something that already happens
with excel right you don't need to have
an ai system to do this if you have an
automated process in place where humans
decide using automated criteria whom to
kill when
and whom to target when which already
happens
right and you have no way to get off the
kill list once that happens once you
have been targeted according to some
automatic criterion by people right in a
bureaucracy
that that is the issue the issue is it's
not the ai it's the automation
so
there's something about
right it's automation but there's
something about the
there's a certain level of abstraction
where you give control to ai to do the
automation
there's a
scale that can be achieved that it feels
like the scale of bug and scale mistake
and scale of destruction that can be
achieved
of the kind that humans cannot achieve
so ai is much more able to destroy an
entire country accidentally versus
humans it feels like the more
civilians die as they react or suffer as
the consequences of your decisions
the more weight there is on the human
mind to make that decision
and so like it becomes more and more
unlikely to make that decision for
humans for ai it feels like it's harder
to encode that kind of weight
in a way the ai that we're currently
building is automating statistics right
intelligence is the ability to make
models so you can act on them and ai is
the tool to make better models
so in principle if you're using ai
wisely you're able to prevent more harm
and i think that the main issue is is
not on the side of the ai it's on the
side of the human command hierarchy that
is using technology irresponsibly so the
question is how hard is it uh to encode
to to pro to properly encode the right
incentives into the ai so
for instance there's this idea what
happens if we let our airplanes be flown
with ai systems and the neural network
is a black box and so on and it turns
out our neural networks are actually not
black boxes anymore there are uh
function approximators using linear
algebra
and uh there are performing things that
we can understand but we can also
instead of letting the neural network
fly the airplane use the neural network
to generate approval of the correct
program with the degree of accuracy of
the proof that a human could not achieve
right so we can use our ai by combining
different technologies to build systems
that are much more reliable than the
systems that a human being could create
and so in in this sense i would say that
if you use an early stage of technology
to save labor and don't employ employ
competent people but just to hack
something together because you can that
is very dangerous and if people are
acting under these incentives that they
get away with delivering shoddy work
more cheaply using ai is less human
oversight than before that's very
dangerous the thing is though ai is
still going to be unreliable perhaps
less so than humans but it'll be
unreliable in novel ways
and
yeah but this is an empirical question
and it's something that we can figure
out and work with so the issue is do we
trust the systems the social systems
that we have in place and the social
systems that we can build and maintain
that they're able to use ai responsibly
if they can then ai is good news if they
cannot then it's going to make the
existing problems worse
well and also who creates the ai who
controls it who makes money from it
because it's ultimately humans and then
you start talking about how much you
trust the humans so the question is what
does who mean i don't think that we have
identity per se i think that these the
story of a human being is somewhat
random
what happens is more or less that
everybody is acting on their local
incentives but they're perceived to be
their incentives and the question is
what are the incentives that the one
that is
pressing the button is operating under
yeah
it's nice for those incentives to be
transparent so
for example i'll give i'll give you
examples there seems to be a significant
uh distrust of um
a
tech
like entrepreneurs in the tech space or
or people that run for example social
media companies like mark zuckerberg
there's not a complete transparency of
incentives under
which that particular human being
operates
the tren you know we can listen to the
words he says or what the marketing team
says for a company but we don't know and
that's that's becomes a problem when
that the algorithms and the systems
created by
uh him and other people
in that company start having more and
more impact on society
and that it starts you know if if the
incentives were somehow
the definition and the explainability of
the incentives was um
decentralized such that no nobody can
manipulate it
no propaganda
uh type manipulation of
like how these systems actually operate
could be done then yes
it'd be not it i think i think ai could
achieve um
much fairer much more effective
uh
sort of
uh like solutions to to to difficult
ethical problems but when there's like
humans in the loop
manipulating the
the the dissemination the communication
of how the system actually works that
feels like you can run to a lot of
trouble and that's why there's currently
a lot of distrust for for people at the
heads of companies that have
increasingly powerful ai systems
i suspect what happened traditionally in
the u.s was that since our decision
making is much more central a
decentralized than in authoritarian
state right people are making decisions
autonomously at many many levels in a
society what happened that was uh we
created coherence and cohesion in
society by controlling what people
thought and what information they had
right the media synchronized public
opinion
and social media have disrupted this
it's not i think so much russian
influence or something it's everybody's
influence it's that a random person can
come up with a conspiracy theory and
disrupt what people think and if that
conspiracy theory is more compelling or
more attractive than the standardized
public conspiracy theory that we give
people as a default
then it might get more traction right
you suddenly have the situation that the
single individual somewhere on a farm in
texas has more listeners than cnn
which particular farmer you're referring
to in texas
[Laughter]
i probably know
yes i had dinner with him a couple times
okay
right this is an interesting situation
because you cannot get to be an anchor
and cnn if you don't go through a
complete uh complicated gatekeeping
process and suddenly you have random
people without that gatekeeping process
uh just optimizing for attention
not necessarily with a lot of
responsibility for the long-term effects
of
projecting these theories into the
public and now uh there is a push of
making social media more like
traditional media which means that the
opinion that is being projected in
social media is more limited to an
acceptable range with the goal of
getting society into safe waters and
increase the stability and cohesion of
society again which i think is a
laudable goal but of course it also is
an opportunity to seize the means of
indoctrination
and the incentives that people are under
when they do this are
in such a way that
the ai ethics that we would need
becomes very often
something like ai politics which is
basically partisan and ideological and
this means that
whatever one side says another side is
going to be disagreeing with right in
the same way as when you turn masks or
the vaccine into a political issue if
you say that it is politically virtuous
to get vaccinated it will mean that the
people that don't like you
will not want to get vaccinated right
and as soon as you have this partisan
discourse it's going to be very hard to
make the right decisions because the
incentives get to be the wrong ones ai
ethics needs to be super boring it needs
to be done by people who do statistics
all the time and have extremely boring
long-winded discussions that most people
cannot follow because they are too
complicated but that are that serious
these people need to be able to to be
better at statistics than the leading
machine learning researchers and at the
moment
the ia ethics the debate is the one but
you don't have any barrier to entry
right everybody who has a strong opinion
and is able to signal that opinion in
the right strong words
and
to me that is a very frustrating thing
because the field is so crucially
important to us so crucially important
but
uh the only qualification you currently
need is to be outraged by the injustice
in the world
it's more complicated right everybody
seems to be outraged but
so let's just say that the incentives
are not always the right ones so
basically i i suspect that
a lot of people that enter this debate
don't have a vision for what society
should be looking like in a way that is
non-violent where we preserve liberal
democracy where we make sure that we all
get along and
we are around in a few hundred years
from now
preferably with a comfortable
technological civilization around us
i i generally
have a very foggy view of that world but
i tend to try to follow and i think
society should in some degree follow
the gradient of love increasing the
amount of love in the world
and whenever i see different policies or
algorithms or ideas that are not doing
so
obviously that that's the ones that kind
of resist
so the thing that terrifies me about
this notion is i think that
german fascism was driven by love
it was just a very selective love
it was a love that now you're just
manipulating i mean that's
i
it's you have to be very careful uh
you're talking to the wrong person
in this way about love so let's talk
about what love is and i think that love
is the discovery of shared purpose it's
the recognition of the sacred and the
other
and this enables non-transactional
interactions
but the
the size of the other that you include
needs to be maximized
so it's
it's basically
appreciation
like deep appreciation
of uh
the world around you fully
like um
including the people that are very
different than you people that disagree
with you completely including people
including living creatures outside of
just people include including ideas and
it's like
appreciation of the full mess of it and
also it has to do with like empathy
which is coupled with a lack of
confidence
uncertainty about
of your own rightness it's like an open
a radical open-mindedness to the way
forward i agree with every part of what
you said and now if you scale it up what
you recognize is that life is in some
sense the service to a next level agency
to the highest level agency that you can
recognize it could be for instance life
on earth or beyond that
really you could say intelligent
complexity in the universe that you try
to maximize in a certain way but when
you think it's true it basically means a
certain aesthetic
and there is not one possible aesthetic
there are many possible aesthetics and
once you project an aesthetic into the
future you can see that there are some
which uh defect from it which are in
conflict with it that are corrupt that
are evil
right you and me would probably agree
that hitler was evil
because uh the aesthetic of the world
that he wanted is in conflict with the
aesthetic of the world that you and me
have in mind yeah
and
so they think that he destroyed we we
want to keep them in the world
there there's a kind of
uh there's kind of ways to deal i mean
hitler is an easier case but perhaps he
wasn't so easy in the 30s right to
understand
who is hitler and who is not
no
there was no consensus that the
aesthetics that he had in mind were
unacceptable yeah i mean it's difficult
a lot
love is complicated
because
you can't just be
uh so open-minded that you let eva walk
into the door
but you can't be so self-self-assured
that
um you you can always identify evil
perfectly
because that's what leads to nazi
germany
having a certainty of what isn't wasn't
evil like always drawing lines of good
versus evil
there seems to be a
there has to be a dance between
um
like
hard stances extending up against what
is wrong and at the same time
empathy and open-mindedness of
towards not knowing
what is right and wrong and like a dance
between those i found that when i watch
the miyazaki movies that there is nobody
who captures my spirituality as well as
he does
it's very interesting and just wishes
right there is something going on in his
movies that is very interesting so for
instance mononoker is discussing
not only an answer to
disney's simplistic notion of mowgli the
jungle boy was raised by wolves and as
soon as he sees people realizes that
he's one of them and
the way in which the
moral life and nature is simplified and
romanticized and turned into kitsch
right it's disgusting in the disney
movie and he answers to this you see
he's replaced by mononoker this wolf
girl who was raised by wolves and who is
fierce and dangerous and
who cannot be socialized because he
cannot be tamed it cannot be part of
human society and you see human society
it's something that is very very
complicated you see people extracting
resources and destroying nature but the
purpose is not to be evil uh but to be
able to have a life that is free from
for instance oppression and violence
and uh to curb death and disease
and you basically see this conflict
which cannot be resolved in a certain
way you see this moment when nature is
turned into a garden and it loses most
of what it actually is and humans no
longer submitting to life and death and
nature and to these questions there is
no easy answer so it just turns it into
something that is being observed as a
journey that happens and that happens
with a certain degree of
inevitability and the nice thing about
all his movies is there's a certain main
character and it's the same in all
movies
it's this little girl
that is basically heidi and i suspect
that happened because he when he did um
field work for working on the heidi
movies back then the heidi animations
before he did his own movies he
traveled to switzerland and south uh
western southeastern europe and adriatic
and so on and got an idea about a
certain aesthetic and a certain way of
life that informed is his future
thinking
and heidi has a very interesting
relationship
to herself and to the world
there is nothing that she takes for
herself
she is in a way fearless because she is
committed to a service to a greater
whole basically she is completely
committed to serving god it is not
an institutionalized god it has nothing
to do with the roman catholic church or
uh something like this but in some sense
heidi is an embodiment of the spirit of
europea protestantism
it's this idea of a being that is
completely perfect and pure and it's not
a feminist vision because
she is
not a girl boss or something like this
she is the justification for the men in
the audience to protect her to build the
civilization around her that makes her
possible
right so she is not uh just the
sacrifice of jesus who is innocent and
therefore nailed to the cross
she is not being sacrificed he is being
protected by everybody around her who
recognizes that she is sacred and there
are enough around her to
to see that
right so this is a very interesting
perspective there is a certain notion of
innocence and this notion of innocence
is not universal it's not in all
cultures right hitler was an innocent
his idea of it of germany was not that
there is a innocence that is being
protected there was a predator that was
going to triumph yeah and it's also
something that is not at the core of
every religion there are many religions
which don't care about innocence they
might care about um increasing
the the status of something
right and that's a very interesting
notion that is quite unique and not
claiming it's the optimal one it's just
a particular kind of aesthetic which i
think makes miyazaki into the most
relevant protestant philosopher today
and you're saying in terms of
of all the of all the ways that society
can operate perhaps the preservation of
innocence um might be one might be one
of the best
no it's just uh my aesthetic
so you're aesthetic it's
it's a particular way in which i feel
that i relate to the world that is
natural to my own socialization and
maybe it's not an accident that i uh
have cultural roots in europe yeah
in a particular world and yeah so
maybe it's a natural convergence point
and it's not something that you will
find in all other times in history
i'd like to ask you about solzhenitsyn
and our individual role as ants in this
very large society so he says that uh
some version of the line between good
and evil runs to the heart of every man
do you think all of us are capable of
good and evil like
what's our role in this play
uh in this game we're all playing
is all of us capable to play any role
like is there an ultimate responsibility
to um
you mentioned maintaining innocence or
whatever the
whatever the highest ideal for a society
you want
are all of us capable of living up to
that and that's our responsibility or or
is there significant limitations to what
we're able to do in terms of good
and evil
so there is a certain way if you are not
terrible if you are committed to
some kind of civilizational agency
a next level agent that you are serving
some kind of transcendent principle
uh in the eyes of that transcendental
principle you are able to discern good
from evil otherwise you cannot otherwise
you have just individual aesthetics
right the cat that is torturing a mouse
is not evil because the cat does not
envision or no part of the world
of the cat is envisioning a world where
there is no violence and nobody is
suffering yeah right if you have an
aesthetic where you want to protect
innocence then torturing
somebody needlessly is evil
but only then
no but within i guess the question is
within the aesthetic like within
your
sense of what is good and evil are we
still
it seems like
we're still able to uh commit evil
yes so basically if you are committing
to this next level agent you are not
necessarily are this next level agent
right you are part of it you have a
relationship to it like a cell does to
its organism it's hyper organism and it
only exists to the degree that it's
being implemented by you and others
and
that means that you're not completely
fully serving it you have freedom in
what you decide whether you are acting
on your impulses and local incentives on
your feral impulses so to speak or
whether you're committing to it and what
you perceive then is a tension between
what you would be doing with respect to
the thing that you uh recognize as the
sacred if you do
and what you're actually doing and this
is the line between good and evil right
where you see oh i'm here acting on my
local incentives or impulses
and here i'm acting on what i consider
to be sacred and there's a tension
between those and this is the line
between good and evil that might run
through your heart
and if you don't have that if you don't
have this relationship to a
transcendental agent you could call this
relationship to the next level agent
soul right it's not a thing it's not an
immortal thing that is intrinsically
valuable it's a certain kind of
relationship that you project to
understand what's happening somebody is
serving this transcendental sacredness
or they're not
if you don't have your soul you cannot
be evil you're just a complex
natural phenomenon so if you look at
life like starting today or starting
tomorrow when we leave here today
there's a bunch of trajectories that you
can take through life
may be countless
do you think some of these trajectories
in your own conception of yourself some
of those trajectories are the ideal life
a life that uh if you were to be the
hero of your life story
you would want to be like is there some
joshua bach that you're striving to be
like this is the question i ask myself
as an individual
trying to make
a better world in the best way that i
could conceive of
what is my responsibility there
and how much am i responsible for the
failure to do so because i'm a i'm i'm
lazy and incompetent too often
in my own perception
in my own worldview i'm not very
important
so it's uh i don't have place for me as
a hero in my own world
i'm trying to do the best that i can
which is often not very good
and
so it's not important for me to to have
status or to be seen in a particular way
it's helpful if others can see me or a
few people can see me that can be my
friends no sorry i i want to clarify
that hero i didn't mean status or
perception or
uh like
uh some kind of marketing thing but more
in private in the quiet of your own mind
is there the kind of man you want to be
and would consider it a failure if you
don't become that
that's what i'm meant by hero yeah not
really
i don't perceive myself as having such
an identity
and it's also sometimes frustrating
but it's basically a
lack of
of having this notion of father that i
need to be emulating
it's interesting it means the leaf
floating down the river
i worry that
sometimes it's more like being the river
i'm just a
fat frog sitting in a leaf
[Music]
on a dirty muddy lake
waiting for a princess to kiss me
or the other way i forgot which way it
goes somebody kisses somebody
i can ask you i don't know if you know
who michael malus is but um
in terms of constructing systems of
incentives it's it's uh interesting to
ask
i don't think i've talked to you about
this before
malice espouses anarchism so he sees all
government as fundamentally
getting in the way or even being
destructive to
collaborations between human beings
thriving
what do you think what's the role of
government in a society
that um
that thrives
is anarchism at all compelling to you
as a system so like
not just small government but no
government at all
yeah i don't see how this would work
the government is an agent that imposes
an offset on your reward
function on your payout matrix so it
your behavior becomes compatible with
the common good
so the argument there
is that you can have
collectives like
governing organizations but not
government like where you're born in a
particular
set of land
and therefore you must follow this
uh
rule or else
you're forced by what they call violence
because there's
there's an implied violence here so what
government the key the key aspect of
government is
is it uh
protects you from the rest of the world
with an army
and with with police right so there's
this
it has a monopoly on violence it's the
only one that's able to do violence
there are many forms of government not
all governments do that right but we uh
find that in the in successful countries
the government has a monopoly on
violence
and uh that means that you cannot get
ahead by starting your own army because
the government will come down on you and
destroy you if you try to do that and in
countries where you can build your own
army and get away with it some people
will do it right in these countries is
what we call failed countries in a way
and
you if you don't want to have violence
the point is not to appeal to the moral
intentions of people because some people
will use strategies if they get ahead
with them that feel a particular kind of
ecological niche so you need to destroy
that ecological niche
and if a
affective government has a monopoly on
violence it can create a world where
nobody is able to use violence and get
ahead right so you want to use that
monopoly on violence not to exert
violence but to make violence impossible
to raise the cost of violence so people
need to get ahead with nonviolent means
so the idea is that you might be able to
achieve that in an anarchist state
with companies so with the with the
forces of capitalism
is
create security companies
where the one that's most ethically
sound rises to the top basically it
would be much better representative of
the people because there is uh
less sort of stickiness to the the big
military force sticking around even
though it's long overlived
outlet so you have groups of militants
that are hopefully officially organized
because otherwise they're going to lose
against the other groups of militants
and they are coordinating themselves
with the rest of society
until they are having a monopoly on
violence how is that different from a
government i listen i'm doing right so
it's basically converging to the same
thing so i think it always i did in my
as i was trying to argue with malice i
feel like it always converges towards
government at scale
but i think the idea is you can have a
lot of collectives that are
you basically never let anything scale
too big
so
one of the problems with governments is
it gets too big in terms of like the the
the size of the group over which it has
control
my sense is that would happen anyway in
in this so successful company like
amazon or facebook i mean it starts
forming a monopoly over over entire
populations not over just a hundred
hundreds of millions but billions of
people so i don't know
but
there is something about the abuses of
power the government can have when it
has a monopoly on violence right
and so
that's that's a tension there but
so the question is how can you set the
incentives for government correctly and
this mostly applies at the highest
levels of government and we because we
haven't found a way to set them
correctly we made the highest levels of
government relatively weak
and this is i think part of the reason
why we had difficulty to coordinate the
pandemic response
and china didn't have that much
difficulty
and there is of course a much higher
risk of the abuse of power that exists
in china because the power is largely
unchecked
and
that's basically what happens in the
next generation for instance imagine
that we would agree that the current
government of china is largely correct
and but never limit and maybe we don't
agree on this but if if he did
how can we make sure that this stays
like this and if you don't have checks
and balances uh division of power
it's hard to achieve we don't have a
solution for that problem
but the abolishment of government
basically would remove the control
structure from a cybernetic perspective
there is an optimal point in the system
that the regulation should be happening
right that you can measure the current
incentives and the regulator would be
properly incentivized to make the right
decisions
and change the payout metrics of
everything below it in such a way that
the local prisoner's dilemmas get
resolved right you cannot resolve the
prisoner's dilemma without some kind of
eternal control
that emulates an infinite game in a way
yeah i mean there
there's a sense in which
it seems like the reason government
the parts of government that don't work
well currently
is because there's not good um
mechanisms
for
through which to interact for the
citizenry to interact with government is
basically it
it hasn't caught up in terms of
technology and i think once you
integrate some of the digital revolution
of of being able to have a lot of access
to data be able to vote on on different
ideas at a local level at all levels at
the at the optimal level like you're
saying that can resolve the the prisoner
dilemmas and to integrate ai to help you
out automate things that are like
um
that don't require the human ingenuity
i feel like that's that's where
government could operate that well and
can also
break apart the inefficient
bureaucracies if needed there'll be a
strong incentive to
to um
to be efficient and successful so out
human history we see an evolution and
evolutionary competition of modes of
government and of individual governments
is in these modes and every nation state
in some sense is some kind of organism
that has found different solutions for
the problem of government
and you could look at all these
different models and the different
scales at which it exists as empirical
attempts to validate the idea of how to
build a better government
and i suspect that the idea of anarchism
similar to the idea of communism
is the result of
being disenchanted with the ugliness of
the real existing solutions and the
attempt to get to
an utopia and i suspect that communism
originally was not a utopia i think
that's in the same way as original
christianity it had a particular kind of
vision
and this vision is a society a mode of
organization within the society in which
humans can
coexist at scale without coercion
the same way as we do in a healthy
family right in a good family you don't
terrorize each other into compliance
but you understand what everybody needs
and what everybody can is able to
contribute and what the intended future
of the whole thing is and you
everybody coordinates their behavior in
the right way and informs each other
about how to do this and all the
interactions that happen are
instrumental to making that happen
right could this happen at scale and i
think this is the idea of communism
communism is opposed to the idea that we
need economic terror or other forms of
terror to make that happen but in
practice what happened is that the
proto-communist countries the real
existing socialism replaced a part of
the economic terror of his moral terror
right so we were told to do the right
thing for moral reasons and of course it
didn't really work and the economy
eventually collapsed and the immoral
terror had actual real cause right
people were in prison because they were
morally non-compliant
and
that is
the other thing is that the idea of
communism became a utopia so it
basically was projected to the afterlife
we were told in in my childhood that
communism was a hypothetical society to
which we were in a permanent revolution
that justified everything was presently
wrong with society morally but it was
something that our grandchildren
probably would not ever see because it
was too ideal and too far in the future
to make it happen right now and people
were just not there yet morally
and the same thing happened with
christianity right this notion of heaven
was mythologized and projected into an
afterlife and i think this was just the
idea of god's kingdom of this world in
which we instantiate the next level
transcendental agent in the perfect form
so everything goes smoothly and without
violence and without conflict and
without this human messiness on this
economic messiness and the terror and
coercion that existed in the present
societies
and the idea of that the humans can
exist at scale in a harmonious way
non-coercively is untested right a lot
of people tested it but didn't get it to
work so far
and the utopia is a world in where you
get all the good things without any of
the bad things
and you are i think very susceptible to
believe in utopia when you are very
young and don't understand
that everything has to happen in causal
patterns that there is always feedback
loops that ultimately are closed there's
nothing that just happens because it's
good or bad good or bad don't exist in
isolation they only exist with respect
to larger systems
so
can you intuit why utopias fail as uh as
systems
so like having a utopia that's out there
beyond the horizon is it because then
um
so it's not only because it's impossible
to achieve utopias but it's because
what certain humans certain small number
of humans start
to uh
um
sort of greedily attain power and money
and control and influence
as they
become uh
uh
as they see the power in uh using this
uh
idea of a utopia it's probably like
saying why is my garden not perfect it's
because some evil weeds are overgrowing
it and they always do yeah right but
this is not how it works a good garden
is a system that is in balance and
requires minimal interactions by the
gardener
and
so you need to create a system that is
designed to self stabilize and the
design of social systems requires not
just the implementation of the desired
functionality but the next level design
but also in biological systems you need
to create a system that wants to
converge to the intended function
right so instead of just creating an
institution like the fda that is
performing a particular kind of rule a
role in society you need to make sure
that the fda is actually driven by a
system that wants to do this optimally
that is incentivize the root optimally
and then makes the performance that is
actually enacted in every generation
instrumental to that thing that actual
goal right and that is much harder to
design and to achieve so you have to
design a system where
i mean listen communism also was quote
unquote incentivized
to be a
feedback
loop
system that achieves that utopia it's
just it wasn't working given human
naturally the incentives were not
correct
how do you uh
incentivize people when they are getting
coal off the ground to work as hard as
possible because it's a terrible job and
it's very bad for your health and
right how do you do this and you can uh
give them prices and medals and status
to some degree right there's only so
much status to give for that and most
people will not fall for this yeah right
or you can pay them
or uh and you probably have to pay them
an asymmetric way because if you pay
everybody the same and they uh
you nationalize the coal mines
eventually people will figure out that
they can game the system yes so you're
you're describing capitalism
so capitalism is the present solution to
the system and what he also noticed that
i think that marx was correct in saying
that capitalism is prone to crisis that
capitalism is a system that in its
dynamics is not convergent but divergent
it's not a stable system
and that eventually it produces an
enormous
potential for productivity
but it also is systematically
misallocating resources so a lot of
people cannot participate
in the production and consumption
anymore and this is what we observe we
observe that the middle class in the us
is tiny
it's uh
a lot of people think that they're
middle class but if you are still flying
economy you're not middle class
right
every class is a magnitude smaller class
and
right
classes really like airline classes
[Laughter]
a lot of people are economy have we
really business class and very few are
first class and some are bridgette i
mean some i i understand i i i think
there's uh yeah
maybe some people probably i would push
back against that definition of the
middle class it does feel like the
middle class is pretty large but yes
there's a discrepancy in terms of wealth
um so there's a big gap in terms of the
productivity that our society could have
yeah there is no reason for anybody to
fly economy right we would be able to
let everybody travel in style
well but also some people like to be
frugal even when they're billionaires
okay so like that
let's take that into yes but uh i mean
you probably don't need to be a
traveling lavish but
you also don't need to be tortured right
there is a difference between frugal and
uh subjecting yourself to torture listen
i love economy i don't understand why
you're comparing a flying economy to
torture i don't although the the the
fight here there's two crying babies
next to me so that but that has nothing
to do with the comments it has to do
with crying babies they're very cute
though so yeah
i have two kids and sometimes i have to
go back to visit the grandparents
and
back means going from the west coast to
germany and it's a long flight is it
true that so when you're a father
you you grow immune to the crying and
all that kind of stuff like you know
because like me just not having kids
it can be other people's kids can be
quite annoying when they're crying and
screaming and all that kind of stuff
uh when you have children and you're
wired up in the default natural way
you're lucky in this regard you fall in
love with them yeah and this falling in
love with them means that you
basically start to see the world through
their eyes and you understand that in a
given situation they cannot do anything
but
um being
expressing despair
and so it becomes more differentiated i
noticed that for instance my son is
typically acting on
a pure
experience of what things are like right
now
and
has to do this right now and you have
this a small child that is um if
when he was a baby and so on where he
was just immediately expressing what he
felt and if you cannot regulate this
from the outside well there's no point
to be upset about it right it's like
dealing with weather or something like
this
you all have to get through it and it's
not easy for him either
but uh if you also have
a daughter maybe she is planning for
that maybe she understands that you know
she's sitting in the car behind you and
she's screaming at the top of her lungs
and you're almost doing an accident yeah
and you really don't know what to do
what should i have done to make you stop
screaming you could have given me candy
i think that's like a cat versus dog
discussion i love it so but
because you said the f
like a fundamental aspect of that is is
love
that makes it all like worth it
what in this monkey riding an elephant
in a dream world
what role does love play
in the human condition
i think that love is the facilitator of
non-transsexual interaction
you are um
observing your own purposes some of
these purposes go beyond your ego they
go beyond the particular organism that
you are and your local interests that's
what you mean by non-transactional yes
so basically when you are acting in
transactional way it means that you are
respecting something in return for you
from the one that you're interacting
with
right you are interacting with a random
stranger you buy something from them on
ebay you expect a fair value for the
money that you send them and vice versa
because you don't know that person you
don't have any kind of relationship to
them but when you know this person a
little bit better and you know the
situation that they're in you understand
what they try to achieve in their life
and you approve because you you realize
that they're in some sense serving the
same human sacredness as you are
and they need to think that you have
maybe you give it to them as a present
but
the i mean the feeling itself of joy is
a kind of benefit
is the kind of transaction
like yes but the joy is not the point
the joy is the signal that you get it's
the reinforcement signal that your brain
sends to you because you are acting on
the incentives of the agent that you're
part of
we are meant to be part of something
larger right this is the way in which we
out-competed other hominins
[Music]
uh take their neanderthals uh yeah right
and also other humans uh right there was
a population bottleneck for a human
society that
leads to an extreme lack of genetic
diversity among humans if you look at
bushmen in the kalahari that basically
tribes that are not that far distant to
each other have more genetic diversity
that exists between
europeans and chinese
and it's because basically the out of
africa population
at some point had a bottleneck of just a
few thousand individuals
and what probably happened is not that
at any time the number of people shrunk
below a few hundred thousand
what probably happened is that there was
a small group that had a decisive
mutation that produced an advantage and
this group multiplied and killed
everybody else and we are descendants of
that group
yeah i i wonder what uh the the the
peculiar characteristics of that group
yeah
i mean we never know and a lot of people
do and we can only we can only just
listen to the echoes in our like the the
ripples uh that are still within us so i
suspect what eventually made a big
difference was the ability to organize
at scale people to program each other
good ideas that we became programmable
that we are willing to work on lockstep
that we went below uh above the tribal
level that we no longer were hoops over
a few hundred individuals and uh acted
on direct deportation systems
transactionally but that we basically
evolved an adaptation to become state
building
yeah
to do to form collectives outside of the
direct collective yes and that's
basically a part of us became committed
to serving something outside
of what we know
yeah then that that's kind of what love
is and it's terrifying because it meant
that we eradicated the others
right it's a force it's an adaptive
force that gets us ahead in evolution
which means we displace something else
that doesn't have that
oh so we had to murder a lot of people
that weren't about love
so love led to destruction have the same
strong love as we did
right that's why i i mentioned this
thing with fascism when you see this uh
these speeches do you want total war and
everybody says yes right this is this
big uh oh my god we are part of
something that is more important than me
that gives meaning to my existence
fair enough
[Laughter]
uh do you have advice
for young people today
in high school in college
that are thinking about what to
do uh with their career with their life
so that uh at the end of the whole thing
they could be proud of what they did
don't cheat
have integrity
integrity so what does integrity look
like when you're the river or the leaf
or the fat frog in a lake
it basically means that you try to
figure out
what the thing is that is the most
right
and this doesn't mean that you don't uh
that you have to look for what other
people tell you what's right
but uh you have to aim for moral
autonomy so things need to be right
independently of what other people say
i always felt that um
when people told me to
listen to what others say like um read
the room build your uh ideas of what's
true based on the high status people of
your in-group that does not protect me
from fascism
the only way to protect yourself from
fascism is to decide is the world that
is being built here the world that i
want to be in
yeah and so in some sense uh try to make
your behavior sustainable act in such a
way that you would feel comfortable on
all sides of the transaction realize
that everybody is you in a different
timeline but is seeing things
differently and has reasons to do so
yeah there's uh
i've come to realize this recently there
is an inner voice that tells you what's
right and
wrong
and
speaking of reading the room there's
times what integrity looks like is
there's times when
a lot of people are doing something
wrong
and what integrity looks like is not
going on twitter and tweeting about it
but
not participating quietly
not doing so it's not like signaling or
not
all this kind of stuff but actually
living your what you think is right like
living it now that's also sometimes this
expectation that others are like us so
imagine the possibility that some of the
people around you are space aliens that
only look human
right so they don't have the same price
as you do they don't have don't have the
same impulses that's what's right and
wrong there is a large
diversity in these basic impulses that
people can have
in a given situation and now realize
that you are a space alien
right you are not actually human you you
think that you're human but you don't
know what it means like what it's like
to be human you just make it up as you
go along like everybody else
and you have to figure that out what it
means that you are
a full human being what it means to be
human in the world and how to connect
with others on that
and
there is also something don't be afraid
in the sense that you if you do this
you're not good enough because if you
are acting on these incentives of
integrity you become trustworthy that's
the way in which you can recognize each
other
there is a particular place where you
can meet you can figure out what that
place is
where you will give support to people
because you realize that they act with
integrity and they will also do that
so in some sense you are safe if you do
that
you're not always protected there are
people which will abuse you and that
might
that are bad actors in way that it's
hard to imagine before you meet them
but
there is also people which
will try to protect you
yeah this that's such a
thank you for saying that there's such a
hopeful message
that no matter what happens to you
there'll be a place
there's people you'll meet
that also have
what you have
and
you will find happiness there and safety
there yeah but it doesn't need to end
well it can also uh all go wrong so this
there's no guarantees in this life so
you can do everything right and you
still can fail and
you can still horrible things happening
to you that traumatize you and mutilate
you
and you have to be grateful if it
doesn't happen
and ultimately be grateful no matter
what happens because even just being
alive is pretty damn nice
yeah even that you know
um
the gratefulness in some sense is also
just generated by your brain uh to keep
you going
it's all the trick
speaking of which
camus said
i see many people die because they judge
that life is not worth living
i see others paradoxically getting
killed for the ideas or illusions that
give them a reason for living
what is called the reason for living is
also an excellent reason for dying
i therefore conclude that the meaning of
life is the most urgent of questions
so i have to ask
what
joshua bach is the meaning of life
it is an urgent question according to
kamu
i don't think that there's a single
answer to this
nothing makes sense unless the mind
makes it so so
you basically have to project a purpose
and
if you zoom out far enough there's the
heat test of the universe and everything
is meaningless everything is just a blip
in between and the question is do you
find meaning in this blab in between
do you find meaning and
observing squirrels
do you find meaning in raising children
and projecting a multi-generational
organism into the future do you find
meaning in projecting an aesthetic of
the world that you like to the future
and trying to serve that aesthetic and
if you do then life that has that
meaning
and if you don't then it doesn't
i kind of enjoy the idea that you just
create the most
vibrant the most weird the most unique
kind of blip you can given your
environment given your set of skills
just be the
the most weird
set of
uh
like local pocket of complexity you can
be
so that like when people study the
universe they'll pause and be like uh
that's weird
it looks like a useful strategy but of
course it's still motivated reasoning
[Laughter]
you're obviously acting on your
incentives here it's still a story we
tell ourselves within a dream that's
that's hardly in touch with reality it's
definitely a good strategy if you're a
podcaster
uh
and a human which i'm still trying to
figure out if i am yeah there's a mutual
relationship somehow somehow josh
you're you're one of the most incredible
people i know um i really love talking
to you i love talking to you again and
um
it's really an honor that you spend your
valuable time with me i hope we get to
talk many times throughout our
uh
through our short and meaningless lives
well meaningful
or meaningful thank you alex i enjoyed
this conversation very much
thanks for listening to this
conversation with joshua bach and thank
you to coinbase codecademy linode
netsuite and expressvpn check them out
in the description to support this
podcast
now let me leave you with some words
from carl young
people will do anything
no matter how absurd in order to avoid
facing their own souls
one does not become enlightened by
imagining figures of light but by making
the darkness conscious
thank you for listening and hope to see
you next time
you