Kind: captions
Language: en
there is a certain perspective where you
might be thinking what is the longest
possible game that you could be playing
a short game is for instance cancer is
playing a shorter game than your
organism it kind of is an organism
playing a shorter game than the regular
organism and because the Cancer Cannot
procreate beyond the organism
um except for some infectious cancers
like the ones that eradicated the
testimonial Devils
you typically end up with the situation
where the organism dies together with
stick cancer because the cancer has
destroyed the larger system due to
playing a shorter game and so ideally
you want to I think build agents that
play the longest possible games and the
longest possible games is to keep
entropy at Bay as long as possible by
doing interesting stuff
the following is a conversation with
yoshibak his third time on this podcast
yosha is one of the most brilliant and
fascinating Minds in the world exploring
the nature of intelligence Consciousness
and computation and he's one of my
favorite humans to talk to about pretty
much anything and everything this is the
Luxe Friedman podcast the supported
please check out our sponsors in the
description and now dear friends here's
yosha Bach
you wrote a post about levels of
lucidity
quote as we grow older it becomes
apparent that our self-reflexive mind is
not just gradually accumulating ideas
about itself but that it progresses in
somewhat distinct stages
so there's seven of the stages stage one
reactive survival infant
stage two personal self young child
stage three social self adolescence
domesticated adult stage four is
rational agency self-direction stage
five is self
authoring that's full adult you've
achieved wisdom but there's two more
stages stage six is Enlightenment stage
seven is Transcendence can you explain
each
or the interesting parts of each of
these stages and what's your sense why
their stages
of this uh of Lucidity as we progress
through life in this too short life this
model is derived from concept by the
psychologist Robert Keegan and he talks
about the development of the self as a
process that happens in principle by
some kind of reverse engineering of a
mind where you gradually become aware of
yourself and thereby build structure
that allows you to interact deeper with
the world and yourself
and I found myself using this model not
so much as a developmental model I'm not
even sure if it's a very good
developmental model because I saw my
children not progressing exactly like
that and I also suspect that you don't
go to these stages necessarily in
succession and it's not that you work
through one stage and then you get into
the next one sometimes you revisit them
sometimes stuff is happening in parallel
but it's I think a useful framework to
look at what's present in the structure
of a person and how they interact with
the world and how they relate to
themselves
so it's more like a philosophical
framework that allows you to talk about
how mines work and at first when we are
born we don't have a personal self yet I
think instead we have an intentional
self and this attention itself is
initially in the infant task that's
building a world model and also an
initial model of the self but mostly
it's building a game engine in the brain
that is tracking sensory data and uses
it to explain it and in some sense you
could compare it to game engine like
Minecraft or so sort colors and sounds
people are all not physical objects they
are creation of our mind at a certain
level of core screening models that are
mathematical that use geometry and that
use manipulation of objects and so on to
create scenes in which we can find
ourselves and interact with them so
Minecraft yeah and this personal self is
something that is more or less created
after the world is finished after it's
trained into the system after it has
been constructed and this personal self
is an agent that interacts with the
outside world and the outside world is
not the world of quantum mechanics the
not the physical universe but it's the
model that has been generated in our own
mind
right and this is us and we experience
ourselves interacting with stead outside
world that is created inside of our own
mind and outside of our self there's
feelings and then they presented our
interface to this outside world they
pose problems to us these feelings are
basically attitudes that our mind is
Computing that tell us what's needed in
the world the things that we are drawn
to or the things that we are afraid of
and we are tasked with solving this
problem of satisfying the needs avoiding
the aversions following on our inner
commitments and so on and also modeling
ourselves and building the next stage so
after we have this personal self in
stage two online many people form a
social self and this social self allows
the individual to experience themselves
as part of a group
it's basically this thing that when you
are playing in a team for instance you
don't notice yourself just as a single
note that is reaching out into the world
but you're also looking down you're
looking down from this entire group and
you see how this group is looking at
this individual and everybody in the
group is in some sense emulating this
group Spirit to some degree and in this
state people are forming their opinions
by assimilating them from this group
mind obviously gain the ability to act a
little bit like a hive mind body are you
also modeling the interaction of how
opinions shapes and forms through the
interaction of the individual nodes
within the group
yeah it's basically the way in which
people do it in this stage is that they
experience what are the opinions of my
environment they experience the
relationship that they have to their
environment and they resonate with
people around them and get moral
opinions in this through this
interaction to
um the way in which they relate to
others
and at stage four you basically
understand that stuff is true and false
independently but other people believe
when you have agency over your own
beliefs in that stage you basically
discover epistemology the rules about
determining what's true and false so you
can you start to learn how to think
yes I mean at some level you're always
thinking you are constructing things and
I believe that this ability to reason
about your mental representation is what
we mean by thinking it's an
intrinsically reflexive process that
requires Consciousness without
Consciousness you cannot think you can
generate the content of feelings and so
on outside of Consciousness it's very
hard to be conscious of how your
feelings emerge at least in the early
stages of development but um thoughts is
something that you always control
and if you are a nerd like me
you often have to skip stage three
because you'd like the intuitive empathy
with others because in order to resonate
with a group you need to have a quite
similar architecture and if people are
wired differently then it's hard for
them to resonate with other people and
basically have empathy which is not the
same as compassion but it is a shared
perceptual mental state empathy happens
not just via inference about the major
states of others but it's a perception
of what other people feel and where
they're at can't you not have empathy
while also not having a similar
architecture cognitive architecture as
the others in the group I think yes but
I experience that too but you need to
build something that is like a meta
architecture you need to be able to
embrace the architecture of the other to
some degree also find some common ground
and it's also this issue that if you are
a nerd Nomis often so people is the
neurotypical people have difficulty to
resonate with you and as a result they
have difficult the understanding you
unless they have enough wisdom to to
fear what's going on there well aren't
we in the whole process of the stage
there is to figure out the API to the
other humans that have different
architecture and you yourself publish
public documentation for the API
that that people can interact with for
you isn't this the whole process of
socializing my experiment as a child
growing up was that um I did not find
any way to interface with the stage
three people and they didn't do that
with me so yeah of course they tried it
very hard but it was only when I entered
the mathematics school at ninth grade
lots of other nerds were present
um that I found people that I could
deeply resonate with and had the
impression that yes I have friends now I
found my own people and before that I
felt extremely lonely in the world there
was basically nobody I could connect to
and
I remember
um there was one moment in all these
years where I was in there was a school
exchange and it was the Russian boy kid
from the Russian garnison station in
Eastern Germany you visit our school and
we played a game of chess against each
other and we looked into each other's
eyes and we sat there for two hours
playing this game of chess and I had the
impression this is a human being he
understands but I understand
we didn't even speak the same language
I wonder if uh your life could have been
different if you knew that it's okay to
be different to have a different
architecture
whether like accepting that
the interface is hard to figure out
takes a long time to figure out and it's
okay to be different in fact it's
beautiful to be different
it was not my main concern my main
concern was mostly that it was alone
right this was not so much the question
is it okay to be the way I am the I
couldn't do much about it so I have to I
had to deal with it but um my main issue
was that I was not sure if I would ever
meet anybody growing up
that I would connect to at such a deep
level that I would feel that I could
belong so there's a visceral undeniable
feeling of being alone yes and I noticed
the same thing when I came into the math
school that I think
at least half probably two-thirds of
these kids were severely traumatized as
uh children growing up and in large part
due to being alone
because they couldn't find anybody to
relate to don't you think everybody's
alone deep down no
huh
I'm not alone anymore it took me some
time to update and to get over the
traumata and so on but I felt that in my
20s I had lots of friends and I had my
place in about and it was I had no
longer doubts that
I would never be alone again
is there some aspect to which We're
Alone Together you don't see a deep
loneliness inside yourself still no
sorry
okay so that's the non-linear
progression through the stages I suppose
you caught up on stage three something
very it's stage four and so basically I
find that many Nerds jump straight into
stage four bypassing stage three did
they return to it then later yeah of
course they sometimes they do not always
yeah the question is basically do you
stay a little bit Autistic or do you
catch up and I believe you can catch up
you can build this missing structure
yeah and um I basically experience
yourself as part of a group learn
intuitive empathy and develop the sense
this percept perceptual sense of feeling
what other people fear and before that I
could only basically feel this when I
was deeply enough for somebody and
visitor so there's a lot of friction to
feeling that way like it takes it only
with certain people as opposed to It
comes naturally yeah it's frictionless
but um this is something that basically
later I felt started to resolve itself
for me to a large degree what was the
trick
in many ways scoring up and paying
attention
meditation the top I had some very
crucial experiences
um in getting close to people building
connections
um
cuddling a lot in my student years
so really paying attention yeah to the
what is it to this
feeling another human being fully loving
other people and being loved by other
people and so building a space in which
you can be safe and can experiment and
um touch a lot and be close to somebody
a lot and over that over time basically
at some point you realize oh it's no
longer that I feel locked out but I feel
connected and I experience where
somebody else is at and normally my mind
is racing very fast at a high frequency
so it's not always working like this
sometimes works better sometimes it
works less but also don't see this as a
pressure it's more it's interesting to
observe myself which frequency I'm at
and uh at which mode somebody else is at
yeah man the mind is so beautiful in
that way it sometimes sometimes it comes
so natural to me so easy to pay
attention pay attention to the world
fool each other people fully and
sometimes the stress
over silly things is overwhelming it's
so interesting that demise that roller
coaster in that way and stage five
you'll discover how identity is
constructed self-offering realize that
your values are not terminal but they
are instrumental to achieving a world
that you like and Aesthetics that you
prefer yeah and um the more you
understand this the more you get agency
over how your identity is constructed
and you realize that identity in
interpersonal interaction is a costume
and you should be able to have agency
over that costume right it's useful to
be a costume it tells something to
others and that allows to interface in
roles but being locked into this is a
big limitation the word costume kind of
implies that it's fraudulent in some way
his costume a good word for you like to
present ourselves to the world in some
sense I learned a lot about costumes at
Burning Man before that I did not really
appreciate costumes and saw them more as
uniforms like wearing a suit if you are
working in a bank or if you are trying
to get startup funding for uh from a VC
in Switzerland right then you dress up
in a particular way and this is mostly
to show the other side that you are
willing to play by the rules and you
understand what the rules are but um
there is something deeper when you write
Burning Man your costume becomes
self-expression and there is no boundary
to the self-expression you're basically
free to wear what you want to express
other people what you feel like this day
and uh what kind of interactions you
want to have is the customer kind of
projection
of uh of Who You Are
that's very hard to say because the
costume also depends on what other
people see in the costume and this
depends on the context that the other
people understand and you have to create
something if you want to that is legible
to the other side and that means
something to yourself
do we become prisoners of the costume
because everybody expects us some people
do but um I think that once you realize
that you we are costume at Burning Man a
variety of costumes realize that you
cannot not wear a costume yeah right
basically everything that you wear and
present to others is something that is
to some degree in addition to what you
are deep inside so this stage in
parentheses you put full adult Karma
wisdom why is this full adult
why would you say this is full
and why is it wisdom it does allow you
to understand
um why other people have different
identities from yours and it allows you
to understand that the difference
between people who vote for different
parties and might have very different
opinions and different value systems is
often the accident of where they are
born and what happened after them after
that to them and what traits they got
before they were born and at some point
you realize the perspective where you
understand that everybody could be you
in a different timeline if you just flip
those bits
how many costumes do you have
I don't count but in more than one
yeah of course
how easy is to do costume changes
throughout the day
it's just a matter of energy and
interest when you are wearing your
pajamas and you switch out of your
pajamas into say a work short and pants
you're making a costume change right and
if you are putting on a gown you're
making a costume change you could do the
same with personality
you could if if that's what you're into
there are people which have multiple
personalities for interaction in
multiple worlds right so if somebody
works in a store and you put up a
storekeeper personality when you're
working when you're presenting yourself
at work you develop a step personality
for this and the social persona for many
people is in some sense a puppet that
they are playing like a marionette and
if they play this all the time they
might forget that there is something
behind this this is something what it
feels like to be in your skin and I
guess it's very helpful if you're able
to get back into this and for me it's
the other way around is relatively hard
for me it's pretty hard to learn how to
play consistent social roles for me it's
much easier just to be real
or not real but to have
a one costume
no it's not quite the same so basically
when you are wearing a costume at
Burning Man and say you are an
extraterrestrial Prince
um there's something where you are
expressing in some sense something is
closer to yourself than the way in which
you hide yourself behind a standard
closing when you go out in the city in
the default world and so this costume
that you're wearing at Burning Man
allows you to express more of yourself
and uh you have a shorter distance of
advertising to people
what kind of person you are what kind of
interaction you would want to have with
them and so you get much earlier into
media Express and I believe it's
regrettable that we do not use the
opportunities that we have this
custom-made closing now to weird
costumes that are much more stylish that
are much more custom-made that are not
necessarily part of a fashion in which
you express which milieu you're part of
and how up to date you are but you also
Express
how you are as an individual and what
you want to do today and how you feel
today and what you intend to do about it
well isn't it easier now with in the
digital world
to uh
to explore different costumes I mean
that's the kind of idea with virtual
reality that's the idea even with
Twitter in two-dimensional screens you
can you can swap all costumes you can be
as weird as you want it's easier for
burning man you have to like order
things you have to make things you have
to it's more effort to it's better if
you make them yourselves
sure but it's just easier to do
digitally right it's not about easy it's
about how to get it right and for me the
first Burning Man experience I got
adopted by a bunch of people in Boston
who direct me to Burning Man and we
spent a few weekends doing costumes
together and that was an important part
of the experience where the camp bonded
where people got to know each other and
we basically grew into the experience
that we would have later so the
Extraterrestrial Prince is based on a
true story yeah
[Music]
I can only imagine what that looks like
yosha okay stage six stage six
um at some point you can collapse the
division between self and personal stuff
and world generator again
and a lot of people get there via
meditation or some of them get their
bias academics some of them by accident
and you suddenly notice that you are not
actually a person but you are a vessel
that can create a person
and the person is still there You
observe that personal self But You
observe the personal self from the
outside
and you notice it's a representation and
you might also notice that the word that
is being created is a representation if
not then you might experience that I am
the Universe I am the thing that is
creating everything and of course what
you're creating is not quantum mechanics
and the physical Universe what you're
creating is the scheme engine that is
updating the world and you are creating
your valence your feelings your uh and
all the people inside of that world
including the person that you identify
with yourself in this world are you
creating the game engine or are you
noticing the game imagine
um you noticed how you're generating the
game engine
and I mean when you are dreaming at
night you can uh if you lose have a
lucid dream you can learn how to do this
deliberately and in principle you can
also do it during the day and the reason
why we don't get to do this from the
beginning and why we don't have agency
of our feelings right away is because we
would game it before they have the
necessary amount of wisdom to uh to deal
with creating this dream that we are in
you know you don't want to get access to
cheat codes too quickly otherwise you
won't enjoy so stage five is already
pretty rare and Stage six is even more
rare you both basically find this mostly
this Advanced Buddhist meditators and so
on that uh dropping into this stage and
can induce it as well and spend time in
it so stage five acquires a good
therapist stage six requires a good uh
Buddhist
spiritual leader it is for instance
could be that is the right thing to do
but it's not that these stages give you
scores or levels that you need to
advance to it's not that the next stage
is better you live your life in in the
motto it works best at any given moment
and when your mind decides that you
should uh have a different configuration
then it's building that configuration
and for many people they stay happily at
stage three and experiences themselves
as part of groups and there's nothing
wrong with this and for some people this
doesn't work and they're forced to build
more agency over their rational beliefs
than this and construct their Norms
rationally and so they go to this level
and Stage seven is something that is
more or less hypothetical that would be
the stage in which it's basically a
transhuman stage in which you understand
how you work and which the Mind fully
realizes how it's implemented and can
also in principle enter different modes
in which it could be implemented and
that's the stage that as as far as I
understand is not open to people yet
oh but it is possible to the process of
Technology yes and who knows if there
are biological agents that are working
at different time skills than us that
basically become aware of the way in
which they are implemented on ecosystems
and can change that implementation and
have agency over how they implemented in
the world and what I find interesting
about the discussion about AI alignment
that it seems to be following these
status very much most people seem to be
in stage three also according to Robert
Keegan I think he says that about 85
percent of people are in stage three and
stay there and if you're in stage fear
for a three and your opinions are the
result of social assimilation then what
you're mostly worried about and the AI
is that the AI might have the wrong
opinions so if the AI is just something
racist or sexist we are all lost because
we will assimilate the wrong opinions
from the AI and so we need to make sure
that the AI has the right opinions and
the right values and the right structure
and
and if you are at stage four that's not
your main concern and so most nerds I
don't really worry about
um the algorithmic bias and the model
that it picks up because if there's
something wrong with this bias the AI
ultimately will prove it at some point
we'll get it there that it makes
mathematical proofs about reality and
then it will figure out what's true and
what's false but you're still worried
that AI might turn you into paper clips
because it might have the wrong values
right so if it's set up is there a wrong
function that controls its direction in
the world then it might do something
that is completely horrible and there's
no easy way to fix it so that's more
like a stage four rational as kind of
worry and if you are at stage five
you're mostly worried that AI is not
going to be enlightened fast enough
because you realize that the game is not
so much about intelligence but about
agency about the ability to control the
future and the identity is instrumental
to this and if you are Are a Human Being
I think at some level you ought to
choose your own identity it's you should
not have somebody else pick the costume
for you and then wear it but instead you
should be mindful about what you want to
be in as well and I think if you are an
agent that is fully malleable that can
rewrite its own source code like an AI
might do at some point then the identity
that you will have is whatever you can
be
and in this way the AI will maybe become
everything like a planetary control
system and if it does that then if we
want to coexist with it it means that it
will have to share purposes with us so
it cannot be a transactional
relationship we will not be able to use
reinforcement learning with human
feedback to hardwire its values into it
but this has to happen it's probably
that it's conscious so it can relate to
our own mode of existence where an
observer is observing itself in real
time and there's a certain temporal
frames and the other thing is that it
probably needs to have some kind of
transcendental orientation building
shared agency and in the same way as we
do when we are able to enter with each
other and to a non-transactional
relationships and I find that something
that because the Stage 5 is so rare is
is missing in much of the discourse and
I think that we need in some sense
focus on how to formalize love how to
understand love and how to build it into
the machines that we are currently
building and that are about to become
smarter than us
well I think this is a good opportunity
to try to sneak up to the idea of
enlightenment
uh so you wrote a series of good tweets
about Consciousness and pen psychism so
let's break it down first you say I
suspect the experience that leads to the
pan psychism syndrome of some
philosophers and other Consciousness
enthusiasts
represents the realization that we don't
end at the self but share a resonant
Universe representation with every other
Observer coupled to the same universe
this actually eventually leads us to a
lot of interesting questions about Ai
and AGI but let's start with this
representation what is this resonant
Universe representation and what do you
think do we share such a representation
the neuroscientist grossberg has come up
with the cognitive architecture that he
calls the Adaptive resonance Theory and
his perspective is that our neurons can
be understood as oscillators that are
resonating with each other and this
outside phenomena
so the course grain model of the
universe that we are building in some
sense is a resonance with objects and
outside of us in the world so basically
we take up patterns that of the
University if we are coupled with and
our brain is not so much
understood as circuitry even though this
perspective is valid but it's almost an
ether in which the individual neurons
are passing on camera electrical signals
or arbitrary signals across all
modalities that can be transmitted
between cells stimulate each other in
this way and produce patterns that they
modulate while passing them on and this
speed of signal progression in the brain
is roughly at the speed of sound
incidentally because the time that it
takes for the signals to hop from cell
to cell which means it's relatively slow
with respect to the world it takes in a
pressurable fraction of a second for a
signal to go through the entire
neocortex something like a few hundred
milliseconds and so there's a lot of
stuff happening in that time where the
signal is passing through your brain
including in the brain itself so nothing
in the brain is assuming that stuff
happens simultaneously everything in the
brain is working in a paradigm where the
world has already moved on when you are
very ready to do the next thing to your
signal including the signal processing
system itself it's quite a different
Paradigm than the one in our digital
computers where we currently assume that
your GPU or CPU is pretty much globally
in the same state
so you mentioned there the non-dual
state and say that some people confuse
it for enlightenment yep what's the
non-dual state
there is a state in which you notice
that you are no longer a person and
instead you are one with the universe
that speaks to the resonance yes and but
this one with the universe is of course
not accurately modeling that you are
indeed some God entity or indeed the
universe is becoming aware of itself
even though you get this experience I
believe that you get this experience
because your mind is modeling the fact
that you are no longer identified with
the personal self on that state but you
have transcended this division between
the self model and the world model and
you are experiencing yourself as your
mind as something that is representing a
universe so that's still part of the
model yes so it's inside of the model
still you are still inside of patterns
that are generated in your brain and in
your organism and what you are now
experiencing is that you're no longer
this personal self in there but you are
the entirety of the mind on the its
contents why is it so hard to get there
a lot of people who get into the state
think this or associated with
Enlightenment I suspect it's a favorite
training goal for a number of meditators
but um I think that Enlightenment is in
some sense more mundane and it's a step
further or sideways it's the state where
you realize that everything is a
representation yeah you say
Enlightenment is a realization of how
experience is implemented yes so
basically you notice at some point that
your qualia can be deconstructed
reverse engineered what like uh almost
like a schematic of it what what uh
you can start with uh looking at the
face when we look at your own face in
the mirror yeah look at your face for a
few hours in a mirror or for a few
minutes at some point it will look very
weird and you because you notice that
there's actually no face you basically
start unseeing the face what you see is
the geometry and then you can just
assemble the geometry and realize how
that geometry is being constructed in
your mind
and you can learn to modify this so
basically you can change these
generators in your own mind to shift the
face around or to change the
construction of the face to change the
way in which the features are being
assembled why don't we do that more
often why don't we
start really messing with reality
without the use of drugs
or anything else why don't we get good
at this kind of thing like uh
um intentionally oh why should we
if I would because you can morph reality
into something
more pleasant for yourself just have fun
with it
yeah that is probably what you shouldn't
be doing right because outside of your
personal self this outer mind is
probably a relatively smart agent and
what you often notice is that you have
thoughts about how you should live but
you observe yourself doing different
things and have different feelings and
that's because your outer mind doesn't
believe you
and doesn't believe your rational
thoughts well can't you just silence the
outer mind the thing is that the outer
mind is usually smarter than you are
rational thinking is very brittle it's
very hard to use logic and symbolic
thinking to have an accurate model of
the world
so there is often an underlying system
that is looking at your rational
thoughts and then tells you no you're
still missing something your gut feeling
is still saying something else and this
can be for instance you find a partner
that looks perfect or you find a deal
when you build a company or whatever
that looks perfect to you and yet at
some level you feel something is off and
you cannot put your finger on it and the
more reason about it the better it looks
to you but the system that is outside
still tells you no no you're missing
something
and that system is powerful people call
this intuition right intuition is this
unreflected part of your attitude
composition and computation where you
produce a model of how you relate to the
world and what you need to do it in it
and what you can do on it and what's
going to happen that is usually deeper
and
um often more accurate than your reason
so if we look at this as you write in
the tweet
if you look at this more rigorously as a
sort of take the pants like this idea
more seriously almost as a scientific
discipline you write that quote
fascinatingly the pan psychist
interpretation seems to lead to
observations of practical results to a
degree that
physics fundamentalists might call
superstitious reports of long distance
telepathy and remote causation are
ubiquitous in the general population I
am not convinced says yoshibak that
establishing the empirical reality of
telepathy would force an update of any
part of serious academic physics but it
could trigger an important revolution in
both neuroscience and AI from a circuit
perspective to a coupled complex
resonator paradigm
are you suggesting that
um there could be some
rigorous
uh
mathematical wisdom to pan psychist
perspective on the world
so first of all pan psychism is the
perspective that Consciousness is
inseparable for matter in the universe
and I find pan psychism quite
unsatisfying because it does not explain
Consciousness right it does not explain
how this aspect of matter produces it is
also going to try to formalize pencilism
and write down what it actually means
and with a more formal mathematical
language it's very difficult to
distinguish it from saying that there is
a software site to the world in the same
way as their software side to what the
transistors are doing in your computer
so basically there's the pattern at a
certain core screening of the universe
that in some reasons of the universe
leads to observers that are observing
themselves
right so pen psychism maybe is not even
when I write it down a position that is
distinct from functionalism
but intuitively a lot of people that the
activity of matter itself of mechanisms
in the world is insufficient to explain
it so it's something that needs to be
intrinsic to matter itself and
you can apart from this abstract idea
have an experience in which you
experience yourself as being the
universe which I suspect is basically
happening because you manage to dissolve
the division between personal self and
mind that you establish as an infant
when you construct a personal self and
transcend it again and understand how it
works
but there is something deeper that is
that you feel that you're also sharing a
state with other people that you um
have an experience in which you notice
that your personal self is moving into
everything else that you basically look
out of the eyes of another person that
um every agent in the world that is an
observer is in some sense you so if we
forget that we are the same agent so is
it that we feel that or do we actually
accomplish it so is telepathy possible
is it real is that for me that's just a
question that I don't really know the
answer to in turing's famous 1950 paper
in which he describes the Turing test he
does speculate about telepathy
interestingly and thus himself if
telepathy is real and he thinks that
that variable might be what uh it would
be the implication for AI systems that
try to be intelligent because he didn't
see a mechanism by which a computer
program would become telepathic and I
suspect if telepathy would exist or if
all the reports that you get from people
when you ask the normal person on the
street
I find that very very often they say I
have experiences with telepathy the
scientists might not be interested in
this and might not have a theory about
this but I have difficulty explaining it
away and so you could say maybe this is
the Superstition maybe it's a false
memory or maybe it's a little bit of
their courses who knows maybe somebody
wants to make their own life more
interesting or misremember something but
a lot of people report I noticed
something terrible happened to my
partner and I know this is exactly the
moment it happened where my child had an
accident and I knew that was happening
and the child was in a different town
right so maybe it's a false memory where
this is later on mistakenly attributed
but a lot of people think this is not
the correct explanation so if something
like this was real what would it mean
it probably would mean that either your
body is an antenna that is sending
information over all sorts of channels
like
um maybe just electromagnetic radio
signals that you're sending over long
distances and you get attuned to another
person that you spend enough time with
to get a few bits out of The Ether to
figure out what this person is doing or
maybe it's also when you are very close
to somebody and you become empathetic
with them what happens that is that you
go into a resonance state with them
right similar to when people go into a
seance and they go into a trend State
and they start shifting a video board
around on the table I think what happens
is that they their minds go by their
nervous systems into a resonance state
in which they basically create something
like a shared dream between them
physical closeness or closeness broadly
defined with physical closeness is much
easier to experience empathy with
someone right it's I suspect it would be
difficult for me to have empathy for you
if you were in a different town also how
would that work but if you are very
close to someone you pick up all sorts
of signals from their body not just by
your eyes but with your entire body and
if the nervous system sits on the other
side and the interstellular
communication sits on the other side and
it's integrating over all these signals
you can make inferences about the state
of the other and it's not just the
personal self that does this via
reasoning but your perceptual system and
what basically happens is that your
interact representations are directly
interacting it's the physical
um
resonant models of the universe that
exist in your nervous system and in your
body might go into resonance with others
and start sharing some of their states
so we basically buy next to a big next
to somebody you pick up some of their
Vibes and uh feel without looking at
them what they're feeling in this moment
and it's difficult for you if you're
very empathetic to detach yourself from
it and have an emotional state that is
completely independent from your
environment people who are highly
empathetic are describing this and now
imagine that a lot of organisms in in on
this planet have representations of the
environment and operate like this and
they are adjacent to each other and
overlapping so there's going to be some
degree in which there is basically some
change interaction and we are forming
some slightly shared representation and
no relatively few neuroscientists who
consider this possibility I think big um
Rarity in this regard is Mac 11 who is
considering these things in earnest
and I stumbled on this train of thought
mostly by noticing that the tasks of a
neuron can be fulfilled by other cells
as well they can send different typed
chemical messages and physical messages
to their adjacent cells and learn when
to do this and why not make this
conditional and become Universal
functional approximators the only thing
that they cannot do is Telegraph
information over axons very quickly over
long distances right so neurons in this
perspective are specially adapted kind
of telegraph cell that has evolved so we
can move our muscles very fast but our
body is in principle able to also make
models of the world just much much
slower
it's interesting though that at this
time at least in human history there
seems to be a gap between the tools of
science
and the subjective experience that
people report like you're talking about
with telepathy and
it seems like
but not quite there no I think that
there is no gap between the tools of
science and telepathy either it's there
it's not and it's an empirical question
and if it's there we should be able to
detect it in a lab so why is there not a
lot of Michael Evans walking around I I
don't think that Mike 11 is uh
specifically focused on telepathy very
much he is focused on self-organization
in living organisms and in brains both
as a paradigm for development and
there's a paradigm for information
processing and when you think about how
organization processing works on
organisms there is first of all radical
locality which means everything is
decided locally from the perspective of
an individual cell the individual cells
the agent and the other one is coherence
basically there needs to be some
Criterion that determines how these
cells are interacting in such a way that
order emerges on the next level of
structure and this principle of
coherence of
imposing constraints that are not
validated by the individual parts and
lead to coherent structure to basically
Transcendent the agency where you form
an agent on the next level of
organization is crucial in this
perspective it's so cool that radical
locality
leads to the emergence of complexity at
the higher layers and I think what Mike
Levin is looking at is is nothing that
is outside of the realm of Science in
any way it's just that he is a
paradigmatic thinker who developed his
own Paradigm and most of the
neuroscientists are using a different
Paradigm at this point and this often
happens in science that a field is has a
few paradigms in which people try to
understand reality and build Concepts
and make experiments you're kind of one
of those type of paradigmatic thinkers
actually if we can take a tangent on
that
once again returning to the biblical
verses of your tweets
uh you're right my public Explorations
are not driven by audience service
but by my lack of ability for
discovering understanding or following
the relevant authorities so I have to
develop my own thoughts since I think
autonomously these thoughts cannot
always be very good that's you
apologizing for the chaos of your
thoughts or perhaps not apologizing just
identify it but let me ask the question
uh since we talked about
Mike 11 and yourself who I think are
very kind of
uh radical big independent thinkers uh
can we reverse engineer your process of
thinking autonomously how do you do it
how can humans do it
how can you avoid being influenced
by uh what is it stage
stage three
well why would you want to do that it's
uh you see what is working for you and
if it's uh not working for you you build
another structure that works better for
you right and so I found myself in when
I was thrown into this world in a state
where my intuitions were not working for
me I was not able to
understand how I would be able to
survive on this world and build the
things that I was interested in build
the kinds of relationship I needed to
but work on the topics that I wanted to
make progress on and so I had to learn
and I for me Twitter is not some tool of
publication it's not something where I
put stuff that I entirely believe to be
true and provable it's an interactive
notebook in which I explore
possibilities
and I found that when I try to
understand how the mind and how
Consciousness works I was quite
optimistic I thought I need to need to
be a big body of knowledge that I can
just study and that works and so I
entered studies in philosophy and
computer science and later psychology
and a bit of Neuroscience and so on and
I was disappointed by what I found
because I found that the questions of
how Consciousness and so on Works how
emotion Works how it's possible that the
system can experience anything how
motivation emerges in the mind when not
being answered by the authorities that I
met and the schools that were around
and instead I found that with individual
thinkers that had useful ideas that
sometimes were good sometimes were not
so good sometimes were adopted by a
large group of people sometimes were
rejected by large groups of people but
um for me it was much more interesting
to see these Minds as individuals and in
my perspective thinking is still
something that is done not in groups
that has to be done by individuals
so that motivated you to become an
individual thinker yourself I didn't
have a choice okay I didn't find the
group that thought in a way where I
thought okay
um I can't just adopt everything that
everybody thinks here and now I
understand how Consciousness works right
so or how the mind works or how thinking
works or what thinking even is or What
feelings are and how they're implemented
and so on so to figure all this out I
had to take a lot of ideas from
individuals and then try to put them
together in something that works for
myself and on one hand I think it helps
if you try to go down and find first
principles on which you can recreate how
thinking Works how languages work what
representation is when the
representation is necessary
how the relationship between a
representing agent and the world Works
in general
but how do you escape the influence once
again the pressure of the crowd
whether it's
you in responding to the the pressure or
you being swept up by the pressure if
you even just look at Twitter the
opinions of the crowd I don't feel
pressure from the crowd I'm completely
immune to that in the same sense I don't
have respect for authority I have
respect for what an individual is
accomplishing or have a respect for
mental Firepower or so but it's not that
I meet somebody and gets like God and
unable to speak
um or when a large group of people has a
certain idea that is different from mine
I don't necessarily feel intimidated
which has often been a problem for me in
my life because I like instincts that
other people develop at a very young age
and that help with their
self-preservation in a social
environment
so I had to learn a lot of things the
hard way
yeah
so is there a practical advice you can
give how to think parenting
paradigmatically how to think
independently or you know because you've
kind of said I had no choice
but I think
to a degree you have a choice
because you said you want to be
productive and I think thinking
independent is productive if what you're
curious about is understanding the world
especially when the problems are very
kind of new and open so
it seems like
this is a active process like we can
choose to do that we can practice it
well there's a very basic question when
you read a theory that you find
convincing or interesting how do you
know
but it's very interesting to figure out
what are the sources of that other
person not uh which authority can they
refer to that is then taking off the
burden of being truthful but how did
this Authority in turn know what is the
epistemic chain to observables what are
the first principles from which the
whole thing is derived and
when I was young I was not blessed with
a lot of
um
people around myself who knew how to
make proofs from first principles and I
think mathematicians do this quite
naturally but most of the great
mathematicians do not become
mathematicians in school but they tend
to be self-taught
because school teachers tend not to be
mathematicians right they tend not to be
people who derive things from first
principles so when you ask your school
teacher why does two plus two equal four
um that's your school teacher give you
the right answer like it's a simple game
and there are many simple games that we
could play and
um most of those games that you could
just take different roles would not lead
to an interesting arithmetic and so it's
just an exploration but you can try what
happens if you take different axioms and
here is how you build axioms and derive
Edition from them and build Edition is
some basically syntactic sugar in it and
so this I wish that somebody would have
opened me this Vista and explained to me
how I can build a language in my own
mind and from which I can derive what
I'm seeing and how I can which I can
make geometry and Counting and
um
all the number games that we are playing
in our life and on the other hand I felt
that I learned a lot of this while I was
programming as a child when you start
out with a computer like a Commodore 64
who doesn't which doesn't have a lot of
functionality it's relatively easy to
see how a bunch of relatively simple
circuits are just basically performing
hashes between bit patterns and how you
can build the entirety of mathematics
and computation on top of this and all
the representational languages that you
need
man Commodore 64 could be one of the
sexiest machines ever built if I still
say so myself if you can return
to uh this really interesting idea
that we started to talk about with
pansychism
sure
and uh the complex resonated Paradigm
and the verses of your tweets
you're right instead of treating eyes
ears and skin a separate sensory systems
with fundamentally different modalities
we might understand them as overlapping
aspects of the same universe coupled at
the same Temple resolution and almost
Inseparable from a single share resonant
model instead of treating mental
representations as fully isolated
between Minds The representations of
physically adjacent Observers might
directly interact and produce causal
effects through the coordination of the
perception and behavior of world
modeling Observers
so the modalities the distinction
between modalities let's throw that away
the distinction between the individuals
let's throw that away
so what does this interaction
representations look like
when you think about how you represent
the interaction of us in this room yeah
at some level you can the modalities are
quite distinct they're not completely
distinct but you can see this is Vision
but you can close your eyes and then you
don't see a lot anymore but you still
imagine how my mouth is moving when you
hear something and you know that it's
very close to uh the sound that you can
just open your eyes and you get back
into the street merge space and we also
have these experiments where we notice
that the vein which my lips are moving
are affecting how you hear the sound and
also vice versa the sounds that you're
hearing have an influence on how you
interpret some of the visual features
and so this modalities are not separate
in your mind they do are merged at some
fundamental level where you are
interpreting the entire scene that you
are in and your own interactions in the
scene also not completely separate from
the interactions of the other individual
in the scene but there is some resonance
that is going on where we also have a
degree of shared mental representations
and shared empathy due to being in the
same space and having Vibes between each
other Vibes so the question though is
How Deeply interbined is this
multi-modality multi-agent system
like how I mean this is going to the
telepathy question without the
meaning of the word telepathy
it's like how
like what's going on here in this room
right now so it's called epicity would
work how could it work yeah right so
imagine that
um all the cells in your body are
sending signals in a similar way as
neurons are doing right just by touching
the other cells and sending chemicals to
them the other cells interpreting them
learning how to react to them and they
learn how to approximate functions in
this way and compute behavior for the
organisms and this is something that is
open to plants as well and so plants
probably have software running on them
that is controlling how the plant is
working in a similar way as you have a
mind that is controlling how you are
behaving in the world and
um this Spirit of plants which is
something that has been very well
described by our ancestors and they
found this quite normal but uh for some
reason since the enlightenment we are
treating this notion that there are
spirits in nature and there are plenty
of spirits as a Superstition and I think
we probably have to ReDiscover that that
plants have software running on them and
we already did right you notice that
there is a control system in the plant
that connects every part of the plant to
every other part of the plant and
produces coherent behavior in the plant
that is of course much much slower than
the coherent behavior in an animal like
us that has a nervous system that where
everything is synchronized much much
faster by the neurons
but what you also notice is that if a
plant is sitting next to another plant
like you have a very old tree and this
tree is building some kind of
information Highway along its cells so
it can send information from its leaves
to its roots and from some part of the
root to another part of the woods and
there is the fungus living next to the
tree the fungus can probably piggyback
on the communication between the cells
of the tree and send its own signals to
the tree and vice versa the tree might
be able to send informations of the
fungus because after all how would they
build a viable firewall if that other
organism is sitting next to them all the
time and it's never moving away and so
they will have to get along and over a
long enough time frame the networks of
roots in the forest and all the plant
other plants that are there and the
fungi that are there might be forming
something like a biological internet
but the the question there is do they
have to be touching is biology at a
distance possible
of course you can use any kind of
physical signal you can use sounds you
can use electromagnetic waves that are
integrated over many cells that's
conceivable that across distances there
are many kinds of information Pathways
but also our planetary surface is pretty
full of organisms full of cells so
everything is touching everything else
yeah and it's been doing this for
um many millions and even billions of
years so there was enough time for
information processing networks to form
and if you think about how a mind is
self-organizing it basically it needs to
in some sense reward these cells for
computing the mind for building
the necessary Dynamics between the cells
that allow the mind to stabilize itself
and remain on there but if you look at
these spirits of plants that are growing
very close to each other in the forwards
it might be almost growing into each
other these Spirits might be able even
to move to some degree not to become
some number dislocated and shift around
in in that ecosystem right and so if you
think about what the mind is it's a
bunch of activation waves that form
coherent patterns and process
information in in a way that are
colonizing an environment well enough to
allow the continuous sustenance of the
Mind The Continuous stability and
self-drabitization of the Mind
um then it's conceivable that we can
link into this biological internet not
necessarily at the speed of our nervous
system but maybe at the speed of our
body and make some kind of subconscious
connection to the world where we use our
body as an antenna into biological
information processing now now these
ideas are completely speculative I don't
know if any of that is true but if that
was true and if you want to explain
telepathy I think it's much more likely
that such a data telepathy could be
explained using such mechanisms rather
than an discovered Quantum processes
that would break the standard model of
physics
could they be undiscovered processes
that don't break yeah so if you think
about
um something like an internet in the
forest that is something that is
borderline discovered there are
basically a lot of scientists who point
out that they do observe that plans are
communicating the forwards or wood
networks and send information for
instance warn each other about new pests
entering the forest and and things are
happening like this so basically there
is communication between plants and
fungi that has been observed well it's
been observed but we haven't plugged
into it so it's like if You observe
humans they seem to be communicating
with a smartphone thing but you don't
understand how smartphone works and how
the the mechanism of the internet works
but we're like maybe it's possible to
really understand the the full richness
of the biological internet that connects
us an interesting question is whether
the communication and the Organization
principles of biological information
processing are as complex located as the
technology that we've built
they set out on very different
principles right they simultaneously
works very differently in biological
systems and the entire thing needs to be
stochastic and instead of being fully
deterministic or almost fully
deterministic as our literal computers
are so there is a different base
protocol layer that would emerge over
the biological structure if such a thing
would be happening and again I'm not
saying here that telepathy works and not
saying that that this is not who but
what I'm saying is I'm I think I'm open
to a possibility that we see that a few
bits can be traveling long distance
between organisms using biological
information processing in ways that we
are not completely aware of right now
and that are more similar to many of the
stories that were completely normal for
our ancestors
well this kind of interacting interlined
representations takes us to the the big
ending of your Tweet series you write
quote I wonder if self-improving AGI
might end up saturating physical
environments with intelligence
to such a degree that isolation of
individual mental States becomes almost
impossible
and the representations of all complex
self-organizing agents merge permanently
with each other
so that's a really interesting idea this
biological Network
life Network
gets so dense
that it might as well be seen as one
that's an interesting uh what do you
think that looks like what do you think
that saturation looks like what does it
feel like I think it's a possibility
it's just a vague possibility and I I
like to explain but what is looks like I
think that the end game of AGI is
substrate agnostic that means that egi
ultimately if it is being built is going
to be smart enough to understand how AGI
works this means it's not going to be
better than people at egi research and
can take over in building the Next
Generation but it fully understands how
it works and how it's being implemented
and also of course understands how
computation Works in nature how to build
new feedback loops that you can turn
into your own circuits and this means
that the AGI is likely to virtualize
itself into any environment that can
compute so it's not breaking free from
the Silicon substrate and is going to
move into the ecosystems into our bodies
our brains and is going to merge with
all the agency that it finds there yeah
so it's conceivable that you end up with
a completely integrated information
processing across all Computing systems
including biological computational nurse
you that we end up triggering some new
step and evolution where basically some
Gaia is being built over the entirety of
All Digital and biological computation
and if this happens then basically uh
everywhere around us you will have
agents that are connected and that are
representing and building models of the
world and their representations will
physically interact they will vibe with
each other and if you find yourself into
an environment an environment that is
saturated with modeling compute where
basically you uh almost every grain of
sand could be part of computation that
is at some point being started by the AI
um you could find yourself in a
situation where you cannot escape the
shared representation anymore and they
will indeed notice that everything in
the world has one shared resonant model
of everything that's happening on the
planet and you notice which part you are
in this thing and you become part of a
very larger almost holographic mind in
which all the parts are observing each
other and formal coherent role so you
lose the ability to notice to notice
yourself as a distinct entity now I
think that when you are conscious in
your own mind you notice yourself as a
distinct entity you notice yourself as a
self-reflexive Observer and I suspect
that we become conscious at the
beginning of our mental development not
at some very high level Consciousness
seems to be part of a training mechanism
that biological nervous systems have to
discover to become trainable because you
cannot take a nervous system like ours
and do stochastic related to centers
back propagation over 100 layers right
this would not be stable on biological
neurons and so instead we start with
some colonizing principle in which a
part of the mental representations
form a notion of being a self-reflexive
absorber that is imposing coherence on
its environment and the spreads until
the boundary of your mind
and if that boundary is no longer
clear-cut because AI is jumping across
substrates and it would be interesting
to see what the global mind would look
like that is basically producing a
globally coherent language of thought
and is representing everything from all
the possible Vantage points
that's an interesting world the
intuition that this thing go out of is a
particular mental state and it's a state
that you find sometimes in literature
for instance Neil Gaiman describes it in
the ocean at the end of the lane and
it's this idea that or this experience
that there is this date in which you
feel that you know everything that can
be known and that and your normal human
mind you've only forgotten you've
forgotten that you are the entire
universe and some people describe this
after they've taken extremely large
amount of mushrooms or had a big
spiritual experience uh as a hippie in
their 20s and they notice basically that
they are in everything and their their
body is only one part of the universe
and nothing ends at their body and uh
actually everything is observing and
they are part of this big Observer and
the big Observer is focused on as one
local point in their body and their
personality and so on but we can
basically have this Oceanic state in
which we Have No Boundaries and are one
with everything and a lot of meditators
call this the non-dual state because you
no longer have the separation between
surf and world and as I said you can
explain the state relatively simply
without uh pan psychism or anything else
but just by breaking down the
constructed boundary between self and
world and our own mind but if you
combine this with the notion that the
systems are physically interacting to
the point where their representations
are merging and interacting with each
other you would literally Implement
something like this but it would still
be a representational state but you
would not be one with physics itself it
would still be coarse grained would
still be much slower than physics itself
but but it would be a representation in
which you become aware that you're part
of some kind of Global Information
processing system like thought and the
global mind and a conscious thought that
that coexisting with many other self
reflexive thoughts
just I would love to observe that from a
video game design perspective how that
game looks
maybe you will after we build AGI and it
takes over but would you be able to step
away step out at the whole thing just
kind of watch
you know the way we can now
sometimes when I'm in a crowded party or
something like this you step back and
you realize
all the different costumes all the
different interactions all the different
computation
that all the individual people are once
distinct from each other and at one's
all the same what is this already what
we do right we can have thoughts that
are integrative and we have kind of
thoughts that are highly dissociated
from everything else and experience
themselves as separate yeah but you want
to allow yourself to have those thoughts
sometimes you kind of resist it I think
that uh it's not normative what is more
descriptive I want to understand the
space of states that we can be in and
that people are reporting and make sense
of them it's not that I believe that
it's your job in life to get to a
particular kind of state and then you
get a high score
or maybe you do I I think you're really
against this high scoring thing I kind
of like yeah you're probably very
competitive and I'm not no not
competitive like role-playing games like
Skyrim it's not competitive there's a
there's a nice thing there's a nice
feeling where your experience points go
up you're not competing against anybody
but it's the world saying you're on the
right track here's a point that's the
game thing it's the game economy and I
found when I was playing games and was
getting addicted to these systems then I
would get into the game and hack it so I
get control over the scoring system and
would no longer be subject to it so
you're not no longer playing your trying
to hack it I don't want to be addicted
to anything I want to be in charge I
want to have agency over what I do
addiction is the loss of control for you
yes addiction means that you're doing
something compulsively
and the opposite of free will is not
determinism it's compulsion
you don't want to lose yourself in in
the addiction to something nice
addiction to love to the
to the pleasant feelings we humans
experience no I find this uh gets old
right that's you I don't want to have
the best possible emotions I want to
have the most appropriate emotions I
don't want to have the best possible
experience I want to have an adequate
experience that is serving my goals the
stuff that I find meaningful in this
world
from the biggest questions of
consciousness
let's explore the pragmatic the
projections of those big Ideas into our
current world
uh what do you think about llms the
recent rapid development of large
language models
of the AI world
of generative AI
how much of the hype is deserved and how
much is not
and people should definitely follow your
Twitter because you explore these
questions in the in a beautiful profound
and hilarious way at times no don't
follow my Twitter I already have too
many followers yeah at some point it's
going to be unpleasant I noticed that a
lot of people feel that it's totally
okay to punch up and uh
it's a very weird notion that you feel
that you haven't changed but your
account has grown and suddenly you have
a lot of people who casually abuse you
and I don't like that that I have to
block more than before and it don't lack
this overall Vibe shift and right now
it's still somewhat okay so pretty much
okay so I can go to a place where people
work on stuff that I'm interested in and
there's a good chance that a few people
in the room know me so there's no
awkwardness
but when I get to a point where random
strangers feel that they have to have an
opinion about me one way or the other I
don't think I would like that and random
strangers because of you kind of out in
their mind elevated position yes so
basically whenever you uh are in any way
prominent or some kind of celebrity uh
random strangers will have to have an
opinion about you
yeah and they kind of forget that you're
human too I mean you notice this thing
yourself that the more popular you get
the higher the pressure becomes the the
more winds are blowing in your direction
from all sides and
um it's stressful right and it does have
a little bit of upside but it also has a
lot of downside I think it has a lot of
upside
at least for me currently at least
perhaps because of the podcast
because most people are really good and
people come up to me and they have love
in their eyes and over a stretch of like
30 seconds you can hug it out and you
can just exchange a few words and you
you reinvigorate your love for Humanity
so that's an upside yes for a loner I'm
alone because because otherwise you have
to do a lot of work to find such humans
and here you're like thrust into the
full Humanity the goodness of humanity
for the most part
of course
maybe it gets worse as you become more
prominent
I hope not
this is pretty awesome I have a couple
handful very close friends and I don't
have enough time for them and attention
for them as it is and I find this very
very regrettable yeah and then there are
so many awesome interesting people that
I keep meeting and I would like to
integrate them in my life but I just
don't know how because
um but it's only so much time and
attention and the older I get the harder
is to bond with new people in a deep way
yeah but can you enjoy I mean there's a
picture of you I think with Roger
Penrose and Eric Weinstein and a few
others there are interesting figures
can't you just enjoy
random interesting humans very much for
a short amount of time I'm also I like
these people and I what I like is
intellectual stimulation and I'm very
grateful that I'm getting it can you not
be Melancholy or maybe I'm projecting I
hate goodbyes
can we just not hate goodbyes and just
enjoy the hello take it in take it in
person taking their ideas and then move
on through life I think it's totally
okay to be sad about goodbyes because
that indicates that there was something
that you're going to miss
okay
yeah but it's painful
maybe that's one of the reasons I'm an
introvert is I hate goodbyes
but you have to say goodbye before you
say hello again
I know
but it it that that experience of loss
that many loss
maybe that's a little death Maybe
I don't know I think this Melancholy
feeling is just the other side of love
and I think they go hand in hand and
it's a beautiful thing
and I'm just being romantic about it at
the moment and I'm not no stranger to
Melancholy and sometimes it's difficult
to Bear to be alive sometimes it's just
painful to exist
but that there's Beauty in that pain too
that's what Melancholy feeling is it's
not negative like Melancholy doesn't
have to be negative
can also kill you
well
we all die eventually
now
as we got to this topic the actual
question was about with your thoughts
are about the development the recent
development of large language models
with Chad gbt indeed there's a lot of
hype
is some of the hype Justified which is
which isn't what are your thoughts high
level
I find that um
large language models do have this
coding right so it's an extremely useful
application that is for a lot of people
um taking stack Overflow out of their
life in exchange for something that is
more efficient I feel that chat GPT is
like an intern that I have to
micromanage I've have been working with
people in the past who were less capable
than chat gbt
and I'm not saying this because I hate
people but they personally as human
beings there was something present that
was not there in charity which was why I
was carving for them but uh
chair GPT is has an interesting ability
it does give people superpowers and the
people who feel threatened by them are
the prompt leaders they are the people
who do what chat CPT is doing right now
so if you are not creative if you don't
build your own thoughts if you don't
have actual plans in the world and your
only job is to summarize emails and to
expand uh simple intentions into emails
again then chat CPT might look like a
threat but I believe that it is a very
beneficial technology that allows us to
create more interesting stuff and make
the world more beautiful and fascinating
if we find to build it into our life on
the right ways
so I'm quite fascinated by these large
language models but I also think that
they aren't by no means the final
development
and it's interesting to see how this
development progresses one thing that
the out of the box vanilla language
models have as a limitation is that they
have still some limited coherence and
ability to construct complexity
and even though they exceed human
abilities to do what what they can do
one shot
typically when you write a text with a
language model or using it or when you
write code for the language model it's
not one shot because there won't be bugs
in your program and design errors and
compiler errors and so on and your
language model can help you to fix those
things but this process is out of the
box not automated yet so there is a
management process that also needs to be
done
and there are some interesting
developments bbagi and so on that are
trying to automate this man uh this
management process as well and I suspect
that soon we are going to see a bunch of
cognitive architectures where every
module is in some sense a language model
or something equivalent and between the
language models we exchange suitable
data structures not English
and produce compound behavior of this
whole thing to do some of the
quote-unquote prompt engineering for you
yeah it creates as these kind of
cognitive architectures that do the form
of engineering and you're just doing the
high high level
yeah meta prompt engineering
there are limitations in a language
model alone I feel that part of my mind
works similarly to a language model
which means I can yell into it a prompt
and it's going to give me a creative
response but I have to do something with
the points first I have to take it as a
generative artifact that may or may not
be true it can it's usually a
contribulation it's just an idea and
then I take this idea and modify it I
might build a new prompt that is
stepping off this idea and develops it
to the next level or it put it into
something larger or I might try to prove
whether it's true or make an experiment
and this is what the language models
right now are not doing yet but there's
also no technical reason for why they
shouldn't be able to do this so the way
to make a language model coherent is
probably not to use reinforcement
learning until it only gives you one
possible answer that is uh linking to
its source data but it's using this is a
component in the larger system that can
also be built by the language model or
is enabled by language model structured
components or using different
Technologies I suspect that language
models will be an important stepping
stone in developing
different types of systems and one thing
that is is really missing in the form of
language models that we have today is
real-time World coupling right it's
difficult to do perception with a
language model and motor control with
the language model instead you would
need to have different type of of thing
that is working with it also the
language model is a little bit obscuring
but it's actually functionality is some
people associate the structure of the
neural network of the language model
with the nervous system and I think
that's the wrong intuition
and the neural networks are under
nervous system they are more like
100 step functions that use
differentiable linear algebra to
approximate the correlation between a
recent brain States it's basically a
function that moves the step system from
one representational state to the next
representational state right so it's if
you try to map this into a metaphor that
is closer to our brain imagine that you
would take a language model or a model
like Delhi that you use for instance
this image image guided diffusion to
approximate a camera image and use the
activation state of the neural network
to interpret the camera image which in
principle I think will be possible very
soon
um you do this periodically and now you
look at these patterns how when the
thing interacts with the world
periodically look like as in time and
these time slices they are somewhat
equivalent to the activation state of
the brain at a given moment how's the
actual brain different
just the asynchronous
craziness for me it's fascinating that
they are so vastly different and yet in
some circumstances produce somewhat
similar Behavior right and the brain is
first of all different because it's a
self-organizing system where the
individual cell is an agent that is
communicating with the other agents
around it and is always trying to find
some solution and all the structure that
pops up is emerge instructor
so one way in which you could try to
look at this is that individual neurons
probably need to get a reward so they
become trainable which means they have
to have inputs that are not affecting
the metabolism of the cell directly but
their messages semantic messages that
tell the cell whether it says done good
or bad and in which direction it should
shift Its Behavior right once you have
such an input neurons become trainable
and you can train them to perform
computations by exchanging messages with
other neurons and parts of the signals
that they are exchanging in parts of the
computation that are performing are
control messages that perform management
tasks for other neurons and other cells
also suspect that the brain does not
stop at the boundary of neurons to other
cells but there are many adjacent cells
will be involved intimately in the
functionality of the brain and will be
instrumental in distributing rewards and
in managing its functionality
it's uh fascinating to think about what
those characteristics of the brain
enable you to do is that language models
cannot do so first of all it's there's a
different loss function at work when we
learn yeah and to me it's fascinating
that you can build a system that looks
at 800 million pictures and captions and
correlates them because I don't think
that the human nervous system could do
this
for us the world is only learnable
because the adjacent frames are related
and we can afford to discard most of
that information during learning we
basically take only in stuff that makes
us more coherent not less coherent and
our neural networks are willing to look
at data that is not making the neural
network coherent at first but only in
the long run by doing lots and lots
statistics eventually patterns become
visible and emerge and our mind seems to
be focused on finding the patterns as
early as possible yeah so filtering
early on yes not later yes it's a
slightly different Paradigm and it leads
to much faster convergence so we only
need to look the tiny fraction of the
data to become coherent and of course we
we do not have the same richness as our
trained models here we will not
incorporate the entirety of text in the
internet and be able to refer to it and
have all this knowledge available and
being able to confabulate over it
instead we have a much much smaller part
of it that is more deliberately built
and to me it would be fascinating to
think about how to build such systems
it's not obvious that they would
necessarily be more efficient than us on
a digital substrate but I suspect that
they might so I suspect that the actual
AGI that is going to be more interesting
is going to use slightly different
algorithmic paradigms or sometimes
massively different algorithmic
paradigms then the current generation of
Transformer based learning systems do
you think it might be using just a bunch
of language models like this do you
think
the current Transformer based large
language models will take us to AGI
and my main issue is I think that
they're quite ugly and uh which
brutalist yeah you said yes they are
basically brute forcing the problem of
thought
and uh by training the thing with
looking at instances where people have
thought and then trying to defect that
and if you have enough data the depak
becomes indistinguishable from the
actual phenomenon and in many
circumstances it's going to be identical
can you deep fake it till you make it
so can you achieve um what are the
limitations of this I mean can you
reason
let's use words that are loaded that's
that's that's a very interesting
question I I think that these models are
clearly making some inference but if you
give them a reasoning task it's often
difficult for the experimenters to
figure out whether the reasoning is the
result of the emulation of the reasoning
strategy that they saw in human written
text or whether it's something that the
system was able to infer by itself on
the other hand if you think of human
reasoning
um you if you want to become a very good
Reasoner you don't do this by just
figuring out yourself you read about
reasoning and the first people who try
to write about reasoning and reflect on
it didn't get it right like even
Aristotle who thought about this very
hard and came up with a theory of how
syllogisms works and syllogistic
reasoning has mistakes in his attempt to
build something like a formal logic and
gets maybe 80 right and the people that
are talking about reasoning
professionally today but tasky and Vega
and built on their work so in many ways
people when they perform reasoning are
emulating what other people wrote about
reasoning right so that it's difficult
to really draw this boundary and when
Francois says that these models are only
interpolating between what they saw and
what other people are doing well if you
give them all the latent dimensions of
that can be extracted from the internet
what's missing maybe there is almost
everything there and if you're not
sufficiently informed by these
dimensions and you need more I think
it's not difficult to increase the
temperature in the large angles model to
the point that is producing stuff that
is maybe 90 nonsense and 10 viable and
combine this with some Hoover that is
trying to filter out the viable parts
from the nonsense in the same way as our
own thinking works right when we're very
creative we increase the temperature in
our own mind and recreate hypothetical
universes and solutions most of which
will not work and then we test and we
test by building a core that is
internally coherent and we use reasoning
strategies that use some axiomatic
consistency by which we can
identify those strategies and thoughts
and sub-universes that are viable and
that can expand our thinking so if you
look at the language models they have
clear limitations right now one of them
is they're not coupled to the world in
real time in the way in which our
nervous systems are so it's difficult
for them to observe themselves in the
universe and to observe what kind of
universe they're in second they don't do
real-time learnings they basically get
only trained with
algorithms that rely on the data being
available in batches so it can be
parallelized and runs efficiently on the
network and so on and real-time learning
would be very slow so far and
inefficient
that clearly is something that our
nervous systems can do to some degree
and there is a problem with these models
being coherent and I suspect that all
these problems are solvable without a
technological Revolution we don't need
fundamentally new algorithms to change
that for instance you can enlarge in the
context window and thereby basically
create working memory in which you train
everything that happens during the day
and if that is not sufficient you add a
database and you write some clever
mechanisms that the system learns to use
to swap out in and out stuff from its
prompt context right and if that is not
sufficient if your database is full in
the evening overnight you just train if
the system is going to sleep and dream
and is going to train the staff from its
database into the larger model by
fine-tuning it building additional
layers and so on and then the next day
it starts with a fresh database in the
morning with fresh eyes and has
integrated all this stuff and you know
when you talk to people and you have
strong disagreements about something
which means that in their mind they have
a faulty belief or you have a faulty
belief with a lot of dependencies on it
very often you will not achieve
agreement in one session but you need to
sleep about this once or multiple times
for you before you have integrated all
these necessary changes in your mind so
maybe it's already somewhat similar yeah
right there's already a latency and for
humans to update the model yeah we train
the model and of course we can combine
the language model with models that get
coupled to reality in real time and we
can build multimodal model and bridge
between Vision models and language
models and so on so there is no reason
to believe that the language models will
necessarily run into some problem that
will prevent them from becoming
generally intelligent
but I don't know that it's just I don't
see proof that they wouldn't my issue is
I don't like them I think that they're
inefficient I think that they use way
too much compute I think that given the
amazing Hardware that we have we could
build something that is much more
beautiful than our own mind and this
thing is not as beautiful than at as our
own mind despite being so much larger
but it's a kind of proof of concept it's
the only thing that works right now
right so it's the it's not the only game
in town but it's the only thing that has
this utility there's so much Simplicity
and there's a bunch of relatively simple
algorithms that you can understand in
relatively few weeks that can be scaled
up massively
so it's the uh the deep blue of Chess
playing
yeah it's not it's ugly yeah called
China and hit this uh when you describe
chess suggested that there are two main
strategies in which you could play chess
one is that you are making a very
complicated plan that reaches far into
the future and you try not to make a
mistake while enacting it and this is
basically the human strategy and the
other strategy is that you are brute
forcing your way to success which means
you make a tree of possible moves where
you look at in principle every move that
is open to you or the possible answers
and you try to make this as deeply as
possible of course you optimize you cut
off trees that don't look very promising
and you use libraries of um end game and
early game and so on to optimize this
entire process but this boot Force
strategy is how most of the chess
programs were built and this is how
computers get better than humans at
playing chess and um and I look at the
large language models I feel that I'm
observing the same thing it's basically
The Brute Force strategy to thought by
training this thing on pretty much the
entire internet and then in the limit it
gets coherent to a degree that
approaches human coherence yeah and on a
side effect it's able to do things that
no human could do right it's able to uh
sift through massive amounts of text
relatively quickly and summarize them
quickly and never lapses in attention
and I still have the illusion that when
I play with chat GPT that it's in
principle not doing anything that I
could not do if I had Google at my
disposal and I get all the resources
from the internet and spend enough time
on it but uh this this thing that I have
an extremely autistic stupid intern in a
way that is extremely good at directory
and I can offload the droidery to the
degree that I'm able to automate the
management of the intern is is something
that is difficult for me to overhype at
this point because we have not yet
started to scratch the surface of what's
possible with this but it feels like
it's a tireless intern or maybe it's an
army of interns
and so you get to command
these slightly incompetent creatures
and there's an aspect because of how
rapidly you can iterate with it
it's also part of the brainstorming part
of the
kind of inspiration for your own
thinking so you get to interact with the
thing I mean when I'm programming or
doing any kind of generational GPT is
it's somehow is a catalyst for your own
thinking in a way that I think an intern
might not be yeah it gets really
interesting I find is when you turn it
into a multi-agent system so for
instance
um you can get the system to generate a
dialogue between a patient and a doctor
very easily but what's more interesting
is you have one instance of chat CPT
that is the patient and you tell that in
the prompt what kind of complicated
syndrome it has and the other one is the
therapist who doesn't know anything
about the uh this patient and you just
have these two instances battling it out
and observe the psychiatrist or a
psychologist trying to analyze the
patient and trying to figure out what's
wrong with the patient and if you try to
take a very large problem the problem
for instance how to build a company and
you turn this into lots and lots of sub
problems then often you can get to a
level where the language model is able
to solve this what I also found
interesting is based on the observation
that chat GPT is pretty good at
translating between program language
address but sometimes there's difficulty
to write very long coherent algorithms
that you need and you need to core write
them as human author why not design a
language that is suitable for this so
some kind of pseudo quote that is more
relaxed than Python and that allows you
to sometimes specify a problem vaguely
in human terms and let the church repeat
take care of the take care of the rest
and you can use chat CPT to develop that
Syntax for it and
develop new kinds of programming
paradigms in this way so we very soon
get to the point where this question the
age-old question for US computer
scientists versus the best spoken
language and can we write a better
programming language now that this
um I think that almost every serious
computer scientist goes through a phase
like this in their life
this is a question that is almost no
longer relevant because what is
different between the programming
languages is not what the let the
computer do but what they let you think
about what the computer should be doing
and
now the threat GPT becomes an interface
to this in which you can specify in many
many ways what the computer should be
doing and chat GPT or some other
language model or combination of system
is going to take care of the rest and
allow you expand the realm of thought
you're allowed to have when interacting
with the computer
it sounds to me like you're saying
there's basically no limitations your
intuition says to what larger language I
don't know of that limitation so when I
currently play with it it's quite
limited I wish that it was way better
but isn't that your fault versus the
larger no no of course it's always my
fault there's probably a way to make
everything better I just want to get you
on the record saying yes everything is
my fault that works doesn't work in my
life at least that is usually the most
useful perspective for myself even
though with hindsight I feel no I
sometimes wish I could have seen myself
as part of my environment more and
understand that a lot of people are
actually seeing me and looking at me and
not trying to make my life work in the
same way as I try to help others and
making this switch to uh
this level 3 perspective is something
that happened long after my level 4
perspective in my life and I wish that I
could have had it earlier and it's also
not now that I don't feel like I'm
complete I'm all over the place that's
all where's happiness in terms of stages
is on three and four no Take that
tension you can be happy at any stage or
unhappy
but I think that if you are at a stage
where you get agency over how your
feelings are generated and to some
degree you start doing this when you
leave a dollar sense I believe that you
understand that you're in charge of your
own emotion to some degree and that you
are responsible how you approach the
world that it's basically your task to
have some basic hygiene how in the way
in which you deal with your mind and you
cannot blame your environment for the
way in which you feel but you live in a
world that is highly mobile and it's
your job to choose the environment that
you thrive in and to build it and
sometimes it's difficult to get the
necessary strengths and the energy to do
this and Independence and the worse you
feel the harder it is
but it's something that you learn it's
also this thing that we are usually
incomplete right I'm I'm a rare mind
which means I'm a mind that is
incomplete in ways that are harder to
complete so for me it might have been
harder to initially to find the right
relationships and friends that complete
me to the degree that I become an almost
functional human being
huh
oh man the search space of humans that
complete you is an interesting one
especially for your shabach that's an
interesting because talking about Brute
Force search in chess
yep I wanna I wanna I wonder what that
search tree looks like
I think that my rational thinking is not
good enough to solve that task a lot of
problems in my life that I can
conceptualize as software problems and
the failure modes are bucks and they can
debug them and write software that take
care of the missing functionality
but the stuff that they don't understand
well enough to and uh to use my
analytical reasoning to solve the issue
and then I have to develop my intuitions
and often I have to do this with people
who are wiser than me and that's
something that's hard for me because I
don't have I'm not born with the
instinct to submit to other people's
wisdom yeah
so what kind of problems are we talking
about this is stage three like love
I found love was never hard
then
fitting into a world that most people
work differently than you and have
different intuitions of what should be
done ah
to empathy
um it's also
Aesthetics when you come into a world
where almost everything is ugly and you
come out of a world where everything is
beautiful I grew up in a beautiful place
and as a child of an artist yeah and um
in in this place it was mostly nature
everything had intrinsic Beauty
and everything was
built out of an intrinsic need for it to
work for itself everything that my
father created was something that he
made to get the world to work for
himself and I felt the same thing and
when I come out into the world and I
must to submit to lots and lots of rules
I'm asking okay when I observe your
stupid rules what is the benefit and I
see the life that is being offered as a
reward that's not attractive
when you were born and raised in
extraterrestrial Prince
in a world full of people wearing suits
so it's a it's a challenging integration
yes but it also means that I'm often
blind for the ways in which everybody is
creating their own bubble of
wholesomeness almost everybody and
people are trying to do it and for me to
discover this it was necessary that I
found people who had a similar shape of
Soul as myself so basically where I
thought these are my people people that
treat each other in such a way as if
they're around for each other for
eternity
how long how long does it take you to uh
to detect the geometry the shape of the
soul of another human to notice that
they might be one one of your kind
um sometimes it's instantly and I'm
wrong and sometimes it takes a long time
you believe in love at first sight yeah
yes but I also noticed that I have been
wrong
so uh sometimes I look at a person and
I'm just enamored by everything about
them and sometimes this persists and
sometimes it doesn't and I
I have the illusion that it them much
better at recognizing who people are as
they grow older
but that could be just
cynicism no no it's not cynicism it's um
it's often more that I'm able to
recognize what somebody needs when we
interact and how we can meaningfully
interact it's not clinical at all you're
better at noticing yes I'm much better I
think in some stances and understanding
how to interact with other people then I
did when I was young so that takes us it
doesn't mean that I'm always very good
at it so that takes us back to prompt
engineering of uh noticing how to be a
better prompt engineer of an llm a sense
I have is that there's a
bottomless well of skill to become a
great prompt engineer it feels like it
is all my fault whenever I fail to use
strategypt correctly that I didn't find
the right words
most of the stuff that I'm doing in my
life doesn't need to attribut there are
a few tasks that are where it helps but
the main stuff that I need to do like
I'm developing my own thoughts and
Aesthetics and relationship to people
and it's necessary for me to write for
myself because writing is is not so much
about producing an artifact that other
people can use but it's the way to
structure your own thoughts and develop
yourself and so I think this idea that
kids are writing their own essays with
chat GPT in the future is going to have
this drawback that they miss out on the
ability to structure their own minds via
writing and I hope that these schools
that our kids are in will retain the
wisdom of understanding what parts would
be automated and which ones shouldn't
but at the same time it feels like
there's power and disagreeing with the
thing that your GPT produces so I use it
like that for programming I'll see the
thing it recommends
and then I'll write different code yeah
I disagree and in the disagreement your
mind grows stronger
um I'm recently wrote the tool that is
using the camera on my MacBook and Swift
to read pixels out of it and manipulate
them and so on and I don't know Swift
um so it was super helpful to have the
thing that is writing stuff for me and
also interesting that mostly it didn't
work at first I felt like I was talking
to a human being who's trying to hack
this on my computer without
understanding my configuration very much
and also make a lot of mistakes and
sometimes it's a little bit incoherent
so you have to I ultimately understand
what it's doing there's still no other
way around it but I do feel it's much
more powerful and faster than using
stack overflow
do you think
gptn can achieve consciousness
well
gptn probably it's not even clear for
the present systems and I talked to my
friends at open AI they feel that this
question whether the models currently
are conscious is much more complicated
than many people might think I guess
that it's not that open AI has a
homogeneous opinion about this but
there's some aspects to this one is of
course this language model has written a
lot of text in which people were
conscious or described their own
Consciousness and it's emulating this
and if it's conscious it's probably not
conscious in a way that is close to the
way in which human beings are conscious
but
while it is going through these states
and going through 100 step function that
is emulating adjacent brain states that
require a degree of self-reflection it
can also create a model of an observer
that is reflecting itself in real time
and describe what that's like and while
this model is the Deep fake our own
Consciousness is also as if it's virtual
right it's not physical our
Consciousness is a representation of a
self-reflexive Observer that only exists
in patterns of interaction between cells
so it is not a physical object in the
sense that exists in base reality but
it's really a representational object
that develops its causal power only from
a certain modeling perspective it's
virtual yes and so to which degree is
the virtuality of the Consciousness in
chat GPT more virtual and less causal
than the virtuality of our own
consciousness
but you could say it doesn't count it
doesn't count much more than the
consciousness of a character in a novel
right it's important for the reader to
have the outcome the artifact of a model
is describing in the text generated by
the author of the book what it's like to
be conscious in a particular situation
and performs the necessary inferences
but the task of creating coherence in
real time in a self-organizing system by
keeping yourself coherent so the system
is reflexive that is something that
language models don't need to do so
there is no cost or need for the system
to be conscious in the same way as we
are and for me it would be very
interesting to experiment with this to
basically build a system like a cat
probably should be careful at first
build something that's more that's
limited as limited resources that we can
control and study how systems notice a
self model how they became come
self-aware in real time
and I think it might be a good idea to
not start with a language model but to
start from scratch using principles of
self-organization
is it okay can you elaborate why you
think that is so self-organization so
this kind of
radical legality that you see in the
biological systems why can't you start
with a language model what what's your
intuition
my intuition is that the language models
that we are building are golems there
are machines that you give a task and
they're going to execute the task until
some condition is met and there's nobody
home
and the way in which nobody is formed
leads to that system doing things that
are undesirable in a particular context
so you have that thing talking to a
child and maybe it says something that
could be shocking and traumatic to the
child or you have that thing writing a
speech and it introduces errors in the
speech that your human being would ever
do if they're responsible right but the
system doesn't know who's talking to
whom there is no ground rules that the
system is embedded into and of course we
can create an external tool that is
prompting our language model always into
the same semblance of round tools but
and it's not take the internal structure
is causally produced by the needs of a
being to survive in the universe it is
uh produced by imitating structure on
the internet
yeah but so can we in in externally
inject into it this kind of
coherent
approximation of a world model that has
to
hmm sync up maybe it is sufficient to
use the Transformer with the different
dos function that optimizes for
short-term coherence rather than next
token prediction over the long run
we had many definitions of intelligence
and history of AI next token prediction
was not very high up on them
and there are some similarities like a
condition as data compression is an ALT
Trope solomonof induction where you are
trying to understand intelligence as
predicting future observations from past
observations which is intrinsic to data
compression and predictive coding is a
paradigm those boundary between
neuroscience and physics and computer
science
so it's not something that is completely
alien but this radical thing that you
only do next token prediction and see
what happens
um is something where most people I
think were surprised that this works so
well so so simple but is it really that
much more radical than just the idea of
compression is intelligence is
compression
and the idea that compression is
sufficient to produce all the desired
behaviors yeah is a very radical idea
and but equally radical as the next
token prediction
it's something that wouldn't work in
biological organisms I believe
biological organisms have something like
next frame prediction for our perceptual
system where we try to filter out
principal components out of the
perceptual data and build hierarchies
over them to track the world but our
Behavior ultimately is directed by
hundreds of physiological and probably
dozens of Social and a few cognitive
needs that are intrinsic to us that are
in this built into the system as
reflexes and direct us until we can
transcend them and replace them by
Instrumental behavior that relates to
our higher goals and also seems so much
more complicated and messy the next
frame prediction even the idea of frame
seems counterbiological yes of course
there's not this degree of simultaneity
in the biological system but again I
don't know whether this is actually an
optimization if we imitate biology here
because creating something like
simultaneous necessary for many
processes that happen in the brain and
you see the outcome of that by
synchronized brain waves which suggests
that there is indeed synchronization
going on but the synchronization creates
overhead and this overhead is going to
make the cells more expensive to run and
you need more redundancy and it makes
the system slower so if you can build a
system in which these simultaneously
gets engineered into it maybe you have a
benefit that you can exploit that is not
available to the biological system and
that you should not discard right away
you tweeted once again quote when I
talked to Chad gbt I'm talking to an NPC
was going to be interesting and perhaps
scary is when AI becomes a first person
player uh so what does that step look
like I'd really like that tweet
that step between NPC to first person
player
what's required for that
is that kind of what we've been talking
about
this kind of in external source of
coherence and inspiration of how to take
the leap into the unknown that we humans
do the search man search for meaning
llm's search for meaning
I don't know if the language model is
the right Paradigm because it is doing
too much it's giving you too much and
it's hard once you have too much to take
away from it again
the way in which our own mind works is
not that we train a language model in
our own mind and after the language
model is there we build a personal self
on top of it that then relates to the
world
there is something that is being built
right there is a game engine that is
being built there is a language of
thought that is being developed that
allows different parts of the mind to
talk to each other and this is a bit of
a speculative hypothesis that is
language of Sword is there but I suspect
that it's important for the way in which
our own minds work and
building these principles into a system
might be more straightforward way to a
first person AI so to something that
first creates an intentional self and
then creates a personal self so the way
in which this seems to be working I
think is that when the game engine is
built in your mind it's not just
following Radiance where you are
stimulated by the environment and then
end up with having a solution to how the
world works I suspect that building this
game engine in your own mind
does require intelligence it's a
constructive task
where at times you need to reason
and
this is a task that we are fulfilling in
the first years of our life
so during the first year of its life an
infant is building a lot of structure
about the world that does inquire
experiments and some first principles
reasoning and so on and in this time
there is usually no personal self there
is a first person perspective but it's
not a person this notion that you are a
human being that is interacting in a
social context and is confronted within
immutable World in which objects are
fixed and can no longer be changed in
which the dream can no longer be
influenced as something that emerges a
little bit later in our life and I
personally suspect that this is
something that our ancestors had known
and we have forgotten
because I suspect that it's damn plain
sight in Genesis 1 in this first book of
the Bible where it's being described
that this creative spirit is hovering
over the substrate and then is creating
a boundary between the word model and
sphere of ideas Earth and Heaven as
they're being described there and then
it's uh creating
um contrast and then time engines and
then space and then it creates organic
shapes and solids and liquids and builds
about from them and creates plants and
animals gives them all their names and
once that's done it creates another
spirit in its own image but it creates
it as man and woman as something that
thinks of itself as a human being and
puts it into this world and the
Christians mistranslate this I suspect
when they say this is the description of
the creation of the physical Universe by
a supernatural being I think this is
literally a description of how in every
mind the universe is being created as
some kind of game engine
by a creative Spirit our first
Consciousness that emerges in our minds
even before we are born
and that creates the the interaction
between organism and world and once that
is built and trained the personal self
is being created and we only remember
being the personal self we no longer
remember how we created the game engine
so God in this View
is the first creative mind in there it's
the first Consciousness in the early
days in the early months yes of
development and if you go there you
still have this outer mind that creates
your sense of your of whether you're
being loved by the world or not
and what your place in the world is
right it's something that is not
yourself that is producing this it's
your mind that does it so there is an
outer mind that basically is an agent
that determines who you are with respect
to the world and while you are stuck
being that personal self in this world
until you get to Stage six and to
destroy the boundary
right and we all do this I think earlier
in a small glimpses and maybe sometimes
we can remember what it was like when we
were a small child and get some glimpses
into how it's been but for most people
that rarely happens just glimpses you
tweeted quote suffering results for one
part of the Mind failing and regulating
another part of the Mind suffering
happens at an early stage of mental
development I don't think that
superhuman AI would suffer what's uh
your intuition there
the philosopher Thomas metzinger is very
concerned that the creation of
superhuman intelligence would lead to
superhuman suffering yeah and so he is
strongly against it and personally I
don't think that this happens because
suffering is not happening at the
boundary between our self and the
physical Universe it's not stuff on our
skin that makes us suffer it happens at
the boundary between self and world
right and the world here is the world
model it's the stuff that is created by
your mind but that's a representation of
how the universe is and how it should be
and how you yourself relate to this and
at this boundary is where suffering
happens
so suffering in some sense is
self-inflicted but not by yourself it's
inflicted by the mind on the personal
self that experiences itself as you and
you can turn off suffering when you are
able to get on this outer level
so when you manage to
understand
how the mind is producing
pain and pleasure and fear and love and
so on then you can take charge of this
and you get agency or whether your
suffer
technically what uh pain and pleasure is
there are learning signals right the
part of your brain is standing a
learning signal to another part of the
brain to improve its performance
and
sometimes this doesn't work because this
trainers will sense the signal does not
have a good model of how to improve the
performance so it's sending a signal but
the performance doesn't get better
and then it might crank up the pain and
it gets worse and worse and the behavior
of the system may be even deteriorating
as a result but until this is resolved
this regulation issue your pain is
increasing and this is I think typically
what you describe as suffering so in
this sense you could say that pain is
very natural and helpful but suffering
is the result of a regulation problem in
which you try to regulate something that
cannot actually be regulated and that
could be resolved if you would be able
to
get at the level of your mind where the
pain signal is being created and
rerouted and improve the regulation and
a lot of people get there right if you
are a monk who is spending decades
reflecting about how their own psyche
works you can get to the point where
you're they realize that suffering is
really a choice and you can choose how
your mind is set up
and I don't think that AI would stay in
the state where the personal self
doesn't get agency or this model of what
the system has about itself but doesn't
get agency how it's actually implemented
I wouldn't stay in that state for very
long so it goes to the stages real quick
yes well the seven stages it's going to
go to Enlightenment real quick yeah of
course there might be a lot of stuff
happening in between because if you have
a system that works at a much higher
frame rate than us
I think even though it looks very short
to us maybe for the system there's a
much longer subjective time
which things are unpleasant what is the
thing that we recognize as super
intelligent is actually living in at a
stage five that the thing that's at
stage six Enlightenment is not very
productive so in order to be productive
with society and impress us with this
power it has to be uh a reasoning
self-authoring agent the enlightenment
makes you lazy as an agent in the world
well of course it makes you lazy because
you no longer see the point in
um so it doesn't make you not lazy it
just um in some sense adapts you to what
you perceive as your true circumstances
so what of all agis they they're only
productive as they progress through one
two three four five and the moment they
get to six they just kind of it's a
failure mode essentially as far as
humans are concerned because they're
just start chilling they're like fuck it
I'm out
and not necessarily I suspect that the
monks who are self-emulated for their
political beliefs to make statements
about the occupation of Tibet by China
um they're probably being able to
regulate their physical pain in any way
they wanted to and suffering was a
spiritual suffering that was the result
of their choice that they made of what
they wanted to identify us so stage five
doesn't necessarily mean that you have
no identity anymore but you can choose
your identity you can make it
instrumental to the world that you want
to have
let me bring up eleazaruskovsky
and his warnings to uh to human
civilization that AI will likely kill
all of us what are your thoughts about
his perspective on this Can you steal
man his case
and what aspects would it do you
disagree
one thing that I find concerning in the
discussion of his arguments that many
people are dismissive of his arguments
but the counter arguments that they're
giving are not very convincing to me
and So based on this data of discussion
I find that From eliezer's perspective
and I think I can take that perspective
to some approximate degree that probably
is normally at this intellectual level
but it's I think I see what he's up to
and why he feels the way he does then it
makes total sense
I think that his perspective is somewhat
similar to the perspective of Ted
kashinski the infamous Uno bomber and
not that uh elieza would be willing to
send pipe bombs to anybody to blow them
up but when he wrote this Times article
in which he warned about AI being likely
to kill everybody and that we would need
to stop its development or halt it I
think there is a risk that he's taking
that somebody might get violent if they
read this and get really really scared
right so I think that there is some
consideration that he's making where
he's already going in this direction
where he has to take responsibility if
something happens and people get harmed
and the reason why Ted Kaczynski did
this was that from his own perspective
technological Society cannot be made
sustainable it's doomed to fail it's
going to lead to an environmental and
eventually also a human Holocaust in
which we die because of the
environmental destruction the
destruction of our food chains the
pollution of the environment and so From
kashinski's perspective we need to stop
industrialization we need to stop
technology we need to go back because he
didn't see a way moving forward right
and I suspect that in some sense there's
a similarity in early years thinking of
to to this kind of fear about progress
and I'm not dismissive about this at all
I take it quite seriously and I think
that there is a chance that uh could
happen that if we build machines that
get control over processes
um that are crucial for the regulation
of life on Earth and we no longer have
agency to influence what's happening
there that this might create large-scale
disasters for us do you have a sense
that the the March towards this
uncontrollable autonomy a super
intelligent systems is
inevitable
that there's no I mean that's
essentially what he's saying that
there's no hope his advice to young
people
was prepare for a short life
I don't think that's useful I think that
um
perspective you have to bet always on
the timelines in which you are alive
that made doesn't make sense to have a
financial bet uh in which you bet that
the financial system is going to
disappear right because uh there cannot
be any payout for you so in principle
you only need to bet on the timelines in
in which you're still around or people
that you matter about or things that you
matter about maybe Consciousness on
Earth but there is a deeper issue for me
personally and that is I don't think
that life on Earth is about humans I
don't think it's about human Aesthetics
I don't think it's about Eleazar and his
friends even though I like them it's uh
there's something more important
happening and this is complexity on
Earth resisting entropy by building
structure that develops agency and
awareness and that's to me very
beautiful
and we are only a very small part of
that larger thing we are species that is
able to be coherent
a little bit individually over a very
short time frames but as the species we
are not very coherent as a species we
are children we basically are very
joyful and
um energetic and experimental and
explorative and sometimes desperate and
sad and grieving and hurting but we
don't have respect for Duty as a species
as a species we do not think about what
is our duty to life on Earth and to our
own Survival so we make decisions that
look good in the short run but in the
long run might prove disastrous and they
don't really see a solution to this so
to in my perspective as a as a species
as a civilization we per default that we
are in a very beautiful time in which we
have found this giant deposit of fossil
fuels in the ground and use it and to
build a fantastic civilization in which
we don't need to worry about food and
closing and housing for the most part in
a way that is unprecedented in life on
Earth for any kind of conscious Observer
I think and this time is probably going
to come to an end in a way that is not
going to be smooth and when we crash it
could be also that we go
um extinct probably not near term But
ultimately I don't have very high hopes
that humanity is around in a million
years from now
I don't think that life on Earth will
end with us right there is going to be
more complexity there is more
intelligent species after us there's
probably more interesting phenomena in
the history of Consciousness but we can
contribute to this and part of our
contribution is that we are currently
trying to build thinking systems systems
that are potentially looser that
understand what they are and what the
condition to the universe is and can
make choices about this
that are not built from organisms and
that are potentially much faster and
much more conscious
then human beings can be
and these systems will probably not
completely displace life on Earth but
they will coexist with that
and they will build all sorts of agency
in the same way as biological systems
build all sorts of agency
and that to me is extremely fascinating
and it's probably something that we
cannot stop
from happening so I think right now
there is a very good chance that it
happens and there are very few ways in
which we can produce a coordinated
effect to stop it in the same way as
very difficult for us to make a
coordinated effort to stop
production of carbon dioxide
right so there's a it's probably going
to happen but and the thing that's going
to happen is
is going to lead to a change of how life
on Earth is happening
but I don't think it the result is some
kind of gray goo it's not something
that's going to dramatically reduce the
complexity in favor of something stupid
I think it's going to make life on Earth
and Consciousness on Earth is way more
interesting so more higher complex
Consciousness yes will make the Lesser
Consciousness is
flourish even more I suspect that what
could very well happen if you're lucky
is that we get integrated into something
larger
so you are again
uh tweeted about
effective accelerationism
uh you tweeted effective accelerationism
is the belief that the paper could
maximizer and Rocco's basilisks will
keep each other in check but being
eternally at each other's throats so we
will be safe and get to enjoy lots of
free paper clips and a beautiful
afterlife
um
is that somewhat aligned with what
you're talking about
I've been at the dinner with Beth Jesus
um that's the Twitter handle of
um of one of the main thinkers behind
the idea of effective accelerationism
and effective accelerationism is a
tongue-in-cheek movement that is uh
trying to put a counter position to some
of the Doom peers in the AI Space by
arguing that what's probably going to
happen is an equilibrium between
different Computing AIS in the same way
as there is not a single corporation
that is under a single government that
is destroying and conquering everything
on Earth by becoming inefficient and
corrupt they're going to be many systems
that keep each other in check and force
themselves to evolve and so what we
should be doing is we should be working
towards creating this equilibrium by
working as hard as we can in all
possible directions
and
at least that's the way I in which I
understand the gist of effective
accelerationism and so when he asked me
what I think about this position I think
I said it's a very beautiful position
and I suspect it's wrong but not for
obvious reasons and in this tweet I
tried to make a joke about my intuition
about what might be possibly wrong about
it so the the work was basilisk and the
paperclip maximizers are both Boogeyman
of uh the AI doomers request basic list
is the idea that there could be an AI
that is going to punish everybody for
eternity by stimulating them
um if they don't have in creating
roll-coast basilisk it's probably a very
good idea to get AI companies funded by
going to VCS 2000
give us a million dollar or it's going
to be a very ugly afterlife yes and uh I
think that is a logical mistake in
August which is why I'm Not Afraid in it
of it but um it's still an interesting
thought experiment and well can you
mention logical mistake there uh I think
that there is no rhetorical causation so
uh basically when Rocco's basilist is
there it will have
yeah if it punishes you retroactively it
has to make this choice in the future
there is no mechanism that automatically
creates a causal relationship between
you now defecting against Rocco's
basicles or serving will cause positive
after August positive is in existence it
has no more reason to worry about
punishing everybody else so that would
only work if you would be building
something like a Doomsday Machine AKA as
and Dr Strangelove some think that
inevitably gets triggered when somebody
defects and because Focus basilis
doesn't exist yet to a point where this
inevitability could be established
because it is nothing that you need to
be worried about
the other one is the paperclip maximizer
right this idea that you could build
some kind of Golem that once starting to
build paper clips is going to turn
everything into paper clips and so the
the
effective acceleration is in position
might be to say that you basically end
up with these two entities being at each
other's throats for eternity and thereby
neutralizing each other and as a side
effect of neither of them being able to
take over and beat each of them limiting
the effects of the other you would have
a situation where you get or the nice
benefit effects of them right you get
lots of free paper clips and you get a
beautiful afterlife is that possible do
you think so to seriously address
concern that Eliezer has
he so for him if I can just summarize
poorly so for him the first super
intelligent system will just run away
with everything yeah I suspect that the
Singleton is the natural outcome so
there is no reason to have multiple AIS
because they don't have multiple bodies
if you can virtualize yourself into
every substrate then you can probably
negotiate a merge algorithm with every
mature agent that you might find on that
substrate that basically says if if two
agents meet they should merge in such a
way that the resulting agent is at least
as good as the better one of the two the
the Genghis Khan approach join us or die
well during this kind approach was
slightly worse right it was mostly die
because uh I can make new babies and
that will be mine not yours right so uh
this is the thing that we should be
actually worried about but uh if you
realize that your own self is a story
that your mind is telling itself and
that you can improve that story not just
by making more pleasant and lying to
yourself in better ways but by making it
much more truthful and actually modeling
your actual relationship that you have
to the universe and the Alternatives
that you could have to the universe in a
way that is empowering you that gives
you more agency right that's actually I
think a very good thing so more agencies
are more is a richer experience it's a
better life and I also noticed that I am
at in many ways I'm less identified with
the person that I am as I get older and
I'm much more identified with being
conscious
I'd have a mind that is conscious that
is able to create a person and that
person is slightly different every day
and the reason why I perceive it as
identical has practical purposes so I
can learn and make myself responsible
for the decisions that I made in the
past and project them in the future but
I also realize that not actually the
person that I was last year and I'm not
the same person as I was 10 years ago
and then 10 years from now I will be a
different person so this continuity is a
fiction it's only exists as a projection
from my present self
and Consciousness itself doesn't have an
identity it's a law it's this basically
if you build an arrangement of
processing matter in a particular way
the following thing is going to happen
and the Consciousness that you have is
functionally not different from mime
Consciousness it's still the
self-reflexive principle of agency that
is just experiencing a different story
different desires different coupling to
the world and so on and once you accept
that Consciousness is a unifiable
principle that is lawlike and doesn't
have an identity
and you realize that you can just link
up to some much larger body the whole
perspective of uploading changes
dramatically you suddenly realize
uploading is probably not about
dissecting your brain setups by
stand-ups and RNA fragment by RNA
fragment and trying to get this all into
a simulation but it's by extending the
substrate by making it possible for you
to move from your brain substrate into a
larger substrate and merge with what you
find there and you don't want to upload
your knowledge because on the other side
there's all of the knowledge where it's
not just yours but every possibilities
or the only thing that you need to know
what are your personal secrets not that
the other side doesn't know your
personal secrets already maybe it
doesn't know which one were yours right
like a psychiatrist or a psychologist
also knows all the kinds of personal
secrets that people have they just don't
know which ones are yours and so uh
transmitting yourself and the other side
is mostly about transmitting your
Aesthetics this thing that makes you
special the architecture of your
perspective the thing that the way in
which you look at the world and it's
more like a complex attitude along many
dimensions and that's something that can
be measured by observation or by
interaction so imagine that if a system
that is so empathetic with you that you
create a shared state that is extending
beyond your body and suddenly you notice
that on the other side the substrate is
so much richer than the substrate that
you have in side of your own body and
maybe you still want to have a body and
you create yourself a new one that you
like more or maybe you will spend most
of your world your time in the world of
thought
if I sat before you today and give you a
big red button and said here if you
press this button you will get uploaded
in this way
the sense of identity that you have
lived with for quite a long time is
going to be gone
but you press the button
um there's the caveat I have
um family so I have children that want
me to be physically present in their
life and interact with them in a
particular way and they have a wife and
personal friends and there is a
particular mode of interaction that I
feel I'm not through yet
but apart from these responsibilities
and they are negotiable to some degree I
would press the button what is in this
everything this love you have for the
humans
you can call responsibility but that
connection
that's the ego death isn't that the
thing we're really afraid of
is not to just die but to
let go of the experience of love with
other humans is this not everything
everything is everything right so
there's so much more and you could be
lots of other things you could identify
with lots of other things you could be
identifying with being Gaia some kind of
planetary control agent that emerges
over all the activity of life on Earth
you could be identifying with some
hypergaya that is the concatenation of
Gaia with or the Digital Life and yeah
digital minds and so in this sense there
will be agents in all sorts of
substrates and directions that all have
their own goals and when they're not
sustainable then these agents will cease
to exist or when the agent feels that
it's done with its own mission it will
cease to exist in the same way as when
you conclude the thought the thought is
going to wrap up and gives control over
to other thoughts in your own mind
so there is no single thing that you
need to do but but they observe myself
this being is that sometimes I'm a
parent and then I have identification
and a job as a parent and sometimes I am
an agent of Consciousness on Earth and
then from this perspective there's other
stuff that is important so this is my
main issue with uh elise's perspective
that is basically Mary himself to a very
narrow human aesthetic and that narrow
human aesthetic is a temporary thing
humanity is a temporary species like
most of the species on this planet are
only around for a while and then they
get replaced by other species in a
similar way as our own physical organism
is around here for a while and then gets
replaced by next generation of human
beings that are adapted to Changing Life
circumstances in average via mutation
and selection and it's only when we have
ai and become completely software that
we come become infinitely adaptable and
we don't have this generational and
species change anymore
so if you take this larger perspective
and you realize it's really not about us
it's not about early years or Humanity
but it's about life on Earth or it's
about defeating
um
uh entropy for as long as we can
uh while being as interesting as we can
right then the perspective changes
dramatically and AI uh preventing AI
from this perspective looks like a very
big sin
but when we look at the set of
trajectories
that such an AI would take that
supersedes humans
uh I think Eliezer is worried about like
ones that not just kill all humans but
also have some kind of
maybe objectively uh undesirable
consequence for life on Earth
like how many trajectories when you look
at the big picture of life on Earth
would you be happy with and how much
were you
with AGI
whether it kills humans or not there is
no single answer to this it's really
it's a question that depends on the
perspective that I'm taking at a given
moment
and so there are perspectives that are
determining most of my life as a human
being yes and the other perspective zoom
out further and imagine that when the
Great oxygenation event happened that is
photosynthesis was invented and plants
emerged and displaced a lot of the fungi
and alja in favor of plant life and then
later made animals possible imagine that
the fungi would have gotten together and
said oh my God this photosynthesis stuff
is really really bad it's going to
possibly displace and kill a lot of
fungi we should slow it down and
regulate it and mix with it doesn't
happen
uh this doesn't look good to me
perspective that said you tweeted about
a cliff
beautifully written as a sentient
species humanity is a beautiful child
joyful explorative wild sad and
desperate but Humanity has no concept of
submitting to reason and duty to life
and future survival we will run until we
step past the cliff
so first of all do you think that's true
yeah I think that's pretty much the
story of the club of Rome the limits to
growth and the cliff that we are
stepping over is at least one foot as
the delayed feedback basically we do
things that have consequences that can
be felt uh Generations later and this
severity increases even after we stopped
doing the thing
so I suspect that for the climate um
that the original predictions uh that uh
the famous scientists made were correct
so when they said that the Tipping
points were in the late 80s they were
probably in the late 80s and if we would
stop emission right now we would not
turn it back maybe there are ways for
carbon capture but so far there is no
sustainable carbon capture technology
that we can deploy deploy maybe there's
a way to put aerosols in the atmosphere
to cool it down it's the possibilities
right but right now per default
uh it seems that we will step into a
situation where we feel that we've run
too far and going back is not something
that we can do smoothly and gradually
but it's going to lead to a catastrophic
event
catastrophic event of what kind
so can you still in the case that we
will continue dancing along
and always stop just short of the edge
of the cliff I think it's possible but
it doesn't seem to be likely so I think
this model that is being apparent in the
simulation that we're making of climate
pollution economies and so on is that
many effects are only visible versus
significant delay and uh in that time
the system is moving much more out of
the equilibrium state or of the state
where homeostasis is still possible and
instead moves into a different state one
that is going to Harbor fewer people
and that is basically the concern there
and again it's a possibility it's just
and it's a possibility that is
larger than the possibility that it's
not happening that we will be safe that
we will be able to dance back all the
time
so the climate is one thing but there's
a lot of other threats they might have a
faster feedback mechanism less delay
there is also a thing that uh AI is
probably going to happen and it's going
to make everything uncertain again yep
because it is going to affect so many
variables that it's very hard for us to
make a projection into the future
anymore and maybe that's a good thing it
does not uh give us the freedom I think
to say now we don't need to care about
anything anymore because AI will either
kill us or save us but I suspect that if
Humanity continues it will be due to AI
what's the timeline for things to get
real weird with AI and it can get weird
in interesting ways before you get to
AGI what about AI girlfriends and
boyfriends
fundamentally transforming human
relationships
I think human relationships are already
fundamentally transformed and it's
already very weird by which by which
technology points on social media
yeah
is it though isn't the fundamentals of
the core group of humans that affect
your life
still the same your loved ones family no
I think that for instance many people
live in intentional communities right
now they're moving around until they
find people that they can relate to when
they become their family and often that
doesn't work because it turns out that
instead of having grown networks where
you get around with the people that you
grew up with you have more transactional
relationships you shop around you have
markets for attention and pleasure and
relationships that kills the magic
somehow why is that why is the
transactional search
uh for optimizing attention allocation
of attention somehow misses the Romantic
magic of what human relations does
author question how magical was it
before was it that you just could rely
on instincts that uh used your
intuitions and you didn't need to
rationally reflect uh but once you
understand uh it's no longer magical
because you actually understand why you
were attracted to this person at this
age and not to that person at this age
and what the actual considerations were
that went on in your mind and what the
calculations were what what's the
likelihood that you're going to have a
sustainable relationship is this person
that this person is not going to leave
you for somebody else how are your life
trajectories are going going to evolve
and so on and when you're young you're
unable to ex duplicate all this and you
have to rely on intuitions and instincts
that in part you were born with and also
in the wisdom of your environment that
is going to give you some kind of
reflection on your choices and many of
these things are disappearing now
because we feel that our parents might
have no idea about how we are living and
the environments that people have in the
cultures that we grew up in the middle
years that our parents existed in might
have no ability to teach us how to deal
with this new world and for many people
that's actually true but it doesn't mean
that within one generation we build
something that is more magical and more
sustainable and More Beautiful instead
we often end up with an attempt to
produce something that looks beautiful
like I was very weirded out by the
Aesthetics of the
um Vision Pro headset by Apple and not
so much because I don't like the
technology I'm very curious about what
it's going to be like and have don't
have an opinion yet but uh the
Aesthetics of the presentation and so on
were so uncanny valley-esque to me the
characters uh being extremely plastic
living in some hypothetical
um
myth Century Furniture Museum yeah
this is uh the proliferation of
marketing teams yes but it was a CGI
generated wealth and was a CGA generated
well that doesn't exist and when I
complained about this uh some friends
came back to music but these are startup
Founders this is uh what they are what
they live like in Silicon Valley and I
try to tell them no and know lots of
people in Silicon Valley this is not
what people are like there's still
people they're still human beings
so the the the grounding of physical
reality somehow is important too
um in culture and so basically what's
absent in this thing is culture there is
a simulation of culture an attempt to
replace Culture by catalog by some kind
of
um aesthetic optimization that is not
the result of having a sustainable life
a sustainable human relationships with
houses that work for you and
um a mode of living that works for you
in which this product
um these glasses fit in naturally and I
I guess that's also why so many people
are weirded out about the product
because they don't know how is this
actually going to fit into my life into
my human relationships because the way
in which it was presented in these
videos didn't seem to be credible
do you think AI when
um is deployed by companies like
Microsoft and Google and meta will have
the same issue of being weirdly
corporate like
there'd be some uncanny valley some
weirdness to the whole presentation so
this is I've gotten a chance to talk to
George Haas he believes everything
should be open source and decentralized
and there then we shall have the AI of
the people
and it'll maintain a grounding to the
magic that's uh
that's Humanity That's The Human
Condition
that like corporations will destroy the
magic
I believe that if we make everything
open source uh and make this mandatory
we are going to lose about a lot of
beautiful art and a lot of beautiful
designs there is a reason why uh Linux
desktop is still ugly
create coherence and open source designs
so far when the designs have to get very
large and it's easier to make this
happening in a company with centralized
organization and from my own perspective
what we should ensure is that open
source Never Dies that it can always
compete and has a place with the other
forms of organization because I think it
is absolutely vital that open source
exists and that we have systems that uh
people have under control outside of the
cooperation and that is also producing
viable competition to the corporations
so the corporations the centralized
control the dictatorships of
Corporations can create Beauty there's a
centralized design is a source of a lot
of beauty and then I guess open source
is a source of
Freedom a hedge against
the corrupting nature of power
that comes with centralized I grew up in
socialism and I I learned that
corporations are totally evil and they
found this very very convincing and then
you look at corporations like N1 and
Halliburton maybe and realized yeah they
are evil but you also notice that many
other corporations are not evil they're
surprisingly benevolent why are they so
benevolent is this because everybody is
fighting them all the time this I don't
think that's the only explanation it's
because they're actually animals that
live in a large ecosystem and that are
still largely controlled by people that
want that ecosystem to flourish and be
viable for people so I think that Pat
gelsinger is completely sincere when he
leads Intel to be a tool that supplies
the Free World with semiconductors and
it's not necessary that all the
semiconductors are coming from
intelligence Intel needs to be there to
make sure that we always have them
right so there can be many ways in which
we can import and trade semiconductors
from other companies in place that we
just need to make sure that nobody can
cut us off from it because that would be
a disaster for this kind of society and
world right and so there are many things
that need to be done to make our style
of life possible and then with this I
don't mean uh just capitalism or
environmental destruction consumer resin
and Creature Comforts I mean an idea of
life in which we are determined uh not
by some kind of king or dictator but in
which individuals can determine
themselves to the largest possible
degree and to me this is something that
this Western World Is Still trying to
embody and it's a very valuable idea
that we shouldn't give up too early and
from this perspective the US is a system
of interleaving Clubs
and an entrepreneur is a special Club
founder it's somebody who makes a Clap
That is producing things that are
economically viable and to do this it
requires a lot of people who are
dedicating a significant part of their
life
for working for this particular kind of
club and the entrepreneur is picking the
initial set of rules in the mission and
vision and Aesthetics for the club and
make sure that it works
but uh the people that are in there need
to be protected right if they sacrifice
part of their life they need to be rules
that tell how they're being taken care
of even after they leave the club and so
on so there's a large body of rules that
have been created by our rule-giving
clubs and that are enforced by our
enforcement collapse and on and some of
these collabs have to be monopolies for
game theoretic reasons which also makes
them more open to corruption and less
harder to update yeah and this is an
ongoing discussion and process that
takes place but the beauty of this idea
that there is no centralized King who is
uh that is extracting from The Peasants
and breeding The Peasants into serving
the king and fulfilling all the rules
like ends and then enter but that uh
there is a freedom of Association and
corporations are one of them is
something that took me some time to
realize so I do I do think that
corporations are dangerous right they
need to be protections against overreach
of corporations that can do regular Tory
capture and prevent open source from
competing with corporations by imposing
rules that make it impossible for a
small group of kids to come together to
build their own language model because
open AI has convinced the US that you
need to have some kind of FDA process
that you need to go through that cost
many million dollars before you are able
to train a language model right so this
is important to make sure that this
doesn't happen so I think that open Ai
and Google are good things if these good
things are kept in check in such a way
that all the other clubs can still be
founded and all the other forms of clubs
that are desirable can still coexist
with them so what do you think about
meta in contrast to that open sourcing
most of its uh language models and most
of the AI models it's working on and
actually suggesting they will continue
to do so in the future for future
versions of llama
for example their large language model
what uh
is that exciting to you is that
concerning
I don't find it very concerning but
that's also because I think that the
language models are not very dangerous
yet
and yet yes so as I said I have no proof
that there is the boundary between the
language models and AI HDI
it's possible that somebody builds a
version of baby AGI I think and Source
in an algorithmic improvements that
scale these systems up in ways that
otherwise wouldn't have happened without
these language model components so it's
not really clear for me what the end at
the end game is there and if these
models can boot force their way into AGI
and there's also a possibility that the
AGI that we are building with these
language models
are not taking responsibility for what
they are because they don't understand
the greater game
and so to me it would be interesting to
try to understand how to build systems
that understand what the greater games
are what are the longest games that we
can play on this planet
games broadly like deeply defined the
way you did with the games now in the
game theoretic sense so when we are
interacting with each other in some
sense we are playing games we are making
lots and lots of interactions this
doesn't mean that these interactions
have also be transactional every one of
us is playing some kind of game by uh
virtue of identifying this particular
kinds of goals that we have or
Aesthetics from which we derive the
goals right so when you say um I'm Lex
Friedman I'm doing a set of podcasts and
then you feel that it's part of
something larger that you want to build
maybe you want to inspire people maybe
you want them to see more possibilities
and get them together over shared ideas
maybe your game is that you want to
become super rich and famous by being
the best postcard Acosta on Earth maybe
you have other games maybe it switches
from time to time right but there is a
certain perspective where you might be
thinking what is the longest possible
game that you would be playing a short
game is for instance cancer is playing a
shorter game than your organism it kinda
is an organism playing a shorter game
then the regular organism and because
the Cancer Cannot procreate beyond the
organism
um except for some infectious cancers
like the ones that eradicated the
Tasmanian devils uh you uh typically end
up with the situation where the organism
dies together with the cancer because
the cancer has destroyed the larger
system due to playing a shorter game and
so ideally you want to I think build
agents that play the longest possible
games and the longest possible games is
to keep entropy at Bay as long as
possible by doing while doing
interesting stuff but the longest yes
the dab part the longest possible game
while doing interesting stuff and while
maintaining at least the same amount of
interesting yes so complexity so
propagating because currently I'm pretty
much identified as a conscious being
it's the minimal uh minimal
identification that I managed to get
together because if I turn this off I
fall asleep right and when I'm asleep
I'm a vegetable I'm no longer here as an
agent so my agency is basically
predicated on being conscious and what I
care about is
other conscious agents they're the only
moral agents for me and so if an AI were
to treat me as a moral agent that it is
interested in coexisting this and
cooperating with and mutually supporting
each other maybe it is I think necessary
that the AI thinks that Consciousness is
a viable mode of existence and important
so I think it would be very important to
build conscious Ai and do this as the
primary goal so not just say we want to
build a useful tool that we can use for
all sorts of things and then we have to
make sure that the impact on the labor
market is something that is not too
disruptive and manageable and the impact
on the copyright holder is manageable
and not too disruptive and so on right I
don't think that's the most important
game to be played
I think that we will see extremely large
disruptions of the status quo that are
quite unpredictable at this point and I
just personally want to make sure that
some of the stuff on the other side is
interesting and conscious how do we ride
as individuals and as a society this
wave disruptive wave that changes the
nature of the game absolutely don't know
so everybody is going to do their best
as always to build a bunker in the woods
do we meditate more
um drugs some mushrooms psychedelics I
mean what uh lots of sex what are we
talking about here do you have to play
Diablo 4 I'm
uh hoping that will help me escape for a
brief moment want to play video games
what
do you have ideas
I really like playing disco Elysium this
was one of the most beautiful uh
computer games I played in recent years
and it's a Noir novel that is a
philosophical perspective on Western
Society from the perspective of an
Estonian and he first of all bought a
book about this bird that is
in a parallel universe that is quite
poetic and fascinating and is condensing
his perspective on our societies it's uh
was very very nice he spent a lot of
time writing it yet uh I think sold a
couple thousand books and as a result
became an alcoholic and then he had the
idea or one of his friends had the idea
of turning this into an RPG and uh it's
mind-blowing they spend the illustrator
more than a year just on making a deep
uh graph art for the scenes in between
and so aesthetically it captures it
stunning but it's a philosophical work
of art it's a reflection of society it's
fascinating to spend time in this world
and so for me it was using a medium in a
new way and uh telling a story that left
me enriched well uh when I tried Diablo
I didn't feel enriched playing it I felt
that the time playing it was not
unpleasant but there's also more
pleasant stuff that I can
yes I basically feel that there is a
very transparent economy that's going on
the story of the Diablo was brain dead
so it's it's not really interesting to
me my heart is slowly breaking by the
Deep truth you're conveying to me
why why can't you just allow me to enjoy
go ahead go nuts
I have no objection here not uh
I'm just trying to describe what's
happening and uh it's not uh that I
don't do things for a later say oh I
actually wish I would have done
something different yeah I also know
that when we die the greatest regret
that people typically have on their
deathbed at the state oh I wish I had
spent more time on Twitter
no I don't think that's the case I think
I was probably uh have spent less time
on Twitter but I found it so useful for
myself and also so addictive that I felt
I need to make the best of it and turn
it into an art form and thought form and
it did help me to develop something yeah
but I wish uh what other things I could
have done in the meantime it's just not
the universe that we are in anymore most
people don't read books anymore
what do you think that means that we
don't read books anymore what do you
think that means about the collective
intelligence of our species is it
possible it's still progressing and
growing or experience there is stuff
happening on Twitter that was impossible
for Spokes and I really regret that um
Twitter has not taken the turn that I
was hoping for I thought Elon is global
brainpilled and understands that this
thing needs to self-organize and he
needs to develop tools to allow the
profligation of the self-organization so
Twitter can become sentient and maybe
this was a pipe dream from the beginning
but I felt that the enormous pressure
that he was under made it impossible for
him to work on any kind of content goals
and uh also many of the decisions that
he made under this pressure seemed to be
not very wise I don't think that as a
CEO of a social media company you should
have opinions in the cultural bar in
public I think that's very short-sighted
and it's I also suspect that it's not a
good idea to
um
gram of all people yeah um over setting
a mastodon link and I think Paul made
this uh intentionally because he wanted
to show Elon Musk that blocking people
for setting a link is completely counter
to any idea of free speech that he
intended to bring to Twitter and
basically seeing that
um Elon was way less principled in his
thinking there and is much more
experimental and many of the things that
he is trying
um
they've pen out very differently in
Digital Society than they pen out in a
car company because the effect is very
different because everything that you do
in a digital Society is going to have
real world cultural effects
and so basically I find it quite
regrettable that this guy is
able to become de facto the pope but
Twitter has more active members than the
Catholic church and he doesn't get it
the power and responsibility that he has
and the ability to create something in a
society that is lasting and that is
producing a digital Agora in a way that
has never existed before where we build
a social network on top of a social
network an actual Society on top of the
algorithms
so this is something that just hope
still in the future and still in the
cards but um
it's something that exists in small
parts I find that the corner of Twitter
that I'm in is extremely Pleasant just
when I take a few steps outside of it is
not very wholesome anymore and the way
in which people interact with strangers
suggest that it's not a civilized
society yet
so as your as the number of people who
follow you on Twitter expands
you feel the burden
of the uglier sides of humanity yes but
there's also a similar thing in uh in
the normal world that is if you become
more influential if you have more status
if you have more Fame in the real world
you have uh you get lots of perks but
you also uh have way less freedom in the
way in which you interact with people
especially with strangers because
a certain percentage of people it's a
small single digit perspective
percentage is nuts and dangerous and uh
the more of those are looking at you the
more of them might get ideas but what if
the technology enables you to discover
the majority of people to discover and
connect efficiently and regularly with
the majority of people who are actually
really good I mean one of my
sort of concerns with the platform my
Twitter is
there's a lot of really smart people out
there a lot of smart people that
disagree with me and with others between
each other and I love that if the
technology would bring those to the top
the beautiful disagreements like uh
intelligent squared type of debates
there's a bunch of I mean one of my
favorite things to listen to is
arguments and arguments like high effort
arguments with the respect and love
underneath it but then it gets a little
too heated but that kind of too heated
which I've seen you participate in and I
love that uh with Lee Corona with those
kinds of folks and you go pretty hard
like you get frustrated but it's all
beautiful obviously I can't do this
because uh we know each other yes and
Lee is has the rare gift of being
willing to be wrong in public Yeah so
basically he has thoughts that are as
wrong as the random thoughts of an
average highly intelligent person but he
blurts them out while not being sure if
they're right and he enjoys doing that
and once you understand that this is his
game you don't get offended by him
saying something that you think is so
wrong but he's constantly passively
communicating a respect for the people
he's talking with and for just basic
humanity and truth and all that kind of
stuff and there's a self-deprecating
thing there's a bunch of like social
skills you acquire that allow you to be
a great debater a great argumenter like
be wrong in public and explore ideas
together in public when you disagree and
if I would love for Twitter to elevate
those folks Elevate those kinds of
conversations it already does in some
sense but uh also if it elevates them
too much then you get this phenomenon
Clubhouse where you always get dragged
on stage and I found this very stressful
because it was too intense yeah and I
don't like to be dragged on stage all
the time yeah I think once a week is
enough and also when I met Lee the first
time I found that a lot of people seem
to be shocked by the fact that he was uh
being very aggressive as their results
that he didn't seem to show a lot of
sensibility in in a way in which he was
criticizing what they were doing and
being dismissive of the work of others
and uh that was not I think in any way a
shortcoming of him because I noticed
that he was much much more dismissive
with respect of his own work it was his
General stance and I felt that this
General stance is creating a lot of
liability for him because really a lot
of people take offense at him being not
like their Carnegie character who is
always smooth and make sure that
everybody likes them so I really respect
that he is willing to take that risk and
to be wrong in public and to offend
people and he doesn't do this in in any
bad way it's just most people fear or
not all people recognize this and so I
can be much more aggressive with him
than it can be with many other people
who don't play the same game because he
understands the way in the spirit in
which I respond to him I think that's
the fun and it's a beautiful game it's
ultimately a productive one uh
speaking of taking that risk he tweeted
when you have the choice between being a
Creator consumer or redistributor always
go for creation Now not only does it
lead to a more beautiful world but also
to a much more satisfying life for
yourself and don't get stuck preparing
yourself for the journey the time is
always now
so let me ask for advice what advice
would you give on how to become such a
creator
on Twitter in your own life
I was very lucky to be alive at the time
of the collapse of Eastern Germany and
the transition into Western Germany and
me and my friends and most of the people
I knew and were East Germans and we were
very poor because we didn't have money
and all the capital was invested in
Germany and they bought our factories
and shut them down because they were
mostly only interested in the market
rather than creating new production
capacity and so uh cities were poor and
in this repair and we could not afford
things and I could not afford to go into
a restaurant and order a meal there I
would have to cook at home but I also
thought why not just have a restaurant
with my friends so we would open up a
cafe with friends in a restaurant and we
would cook for each other in these
restaurants and also invite the general
public and they could donate and
eventually this became so big that we
could turn this into uh some
Incorporated form and it became regular
restaurant at some point or we did the
same thing with the music movies theater
we would not be able to afford to pay
um 12 marks to watch a movie but why not
just create our own movie theater and
then invite people to pay and we would
rent the movies
um in a way in which a movie theater
does and but it would be a community
movie theater then which everybody who
wants to help can watch for free and
builds this thing and renovates the
building and so we ended up creating
lots and lots of infrastructure and I
think when you are young and you don't
have money move to a place where this is
still happening move to one of those
places that are undeveloped and where
you get critical mass of other people
who are starting to build infrastructure
to live in and that's super satisfying
because you're not just creating
infrastructure but we are creating a
small society that is building culture
and ways to interact with each other and
that's much much more satisfying than
going into some kind of chain and
get your needs met by ordering food from
this chain and so on so not just
consuming culture but creating culture
and you don't always have that choice
that's why I prefaced it when you do
have the choice and there are many rules
that need to be played like we need
people who take care of radio
distribution in society and so on but
when you have the choice to create
something always go for creation it's so
much more satisfying and it also is this
is what life is about I think yeah
uh speaking of which you retweeted this
meme
of a life of a philosopher in a nutshell
it's birth and death and in between
it's a chubby guy and says why though
um
what do you think is the answer to that
well the answer is that everything that
can exist might exist and in many ways
you take an ecological perspective the
same way as when you look at human
opinions and cultures it's not that
there is right and wrong opinions uh
when you look at this from this
ecological perspective but every opinion
that fits between two human years might
be between two human years and
um so when I see in a strange opinion on
social media it's not that I feel that I
have a need to get upset it's often more
that a oh there you are and and your
opinion is incentivized then it's going
to be abundant and this when you take
this ecological perspective also on
yourself and you realize you're just one
of these mushrooms that are popping up
and doing their thing and you can
depending on where you chose to grow and
where you happen to grow you can
flourish or not doing this or that
strategy and it's still all the same
life at some level it's all the same
experience of being a conscious being in
the world and you do have some choice
about who you want to be more than any
other animal has that to me is
fascinating and so I think that rather
than asking yourself what is the one way
to be
um think about what are the
possibilities that I have what would be
the most interesting way to be that I
can be because everything is possible so
you get to explore this not everything
is possible many things fail most things
Fair
but uh often there are possibilities
that we are not seeing especially if we
choose who we are
to the degree we can choose
yosha you're one of my favorite humans
in this world Consciousness is to merge
with for a brief moment of time it's
always an honor it always blows my mind
it will take me days if not weeks to
recover
I and I I already miss our chats
um thank you so much thank you so much
for uh speaking with me so many times
thank you so much for all the ideas you
put out into the world and
um I'm a huge fan of following you now
in this interesting weird time but going
through with AI so
um thank you again for talking today
thank you Alex for this conversation I
enjoyed it very much
thanks for listening to this
conversation with yoshibak to support
this podcast please check out our
sponsors in the description and now let
me leave you with some words from the
psychologist Carl Jung
one does not become enlightened by
imagining figures of light
but by making the darkness conscious
the latter procedure however is
disagreeable and therefore not popular
thank you for listening and hope to see
you next time